ADAS sensing and perception

Everything an assisted or automated vehicle does begins with the same question — what is out there, where, and how fast is it approaching. Four sensor types answer it, each in a way the others cannot, and each failing in a way the others do not.

What each sensor actually measures

The important discipline is separating what a sensor measures from what a system concludes. Everything a sensor measures directly is reliable. Everything inferred from it carries the assumptions of the inference.

FMCW radar

The transmitter sweeps its frequency continuously. A return arriving late is compared against what is being transmitted now, and the difference in frequency gives the range. Any relative motion adds a Doppler shift on top, giving closing speed.

Range and velocity from a single measurement. Nothing else on the vehicle measures relative velocity directly, which is why radar underpins anything judging a closing rate.

At 77 GHz the wavelength is about four millimetres. A fog droplet is a few micrometres across, so the wave passes as though the fog were not there.

The cost is angular resolution. The beam is wide, so two things at similar range and bearing merge. Radar knows how far and how fast far better than it knows where or what.

Lidar

Pulses of light, timed on their return. Each gives a true distance to whatever it struck, and the result is a point cloud — measured three-dimensional geometry rather than geometry inferred from a picture.

It supplies its own light, so midnight and midday are identical to it.

Its weakness is the mirror image of radar’s. At 905 or 1550 nanometres the wavelength is smaller than a fog droplet or a raindrop, so it scatters heavily. Fog, spray from a lorry and falling snow all degrade it badly, and snow in particular returns points that look like objects.

Camera

A camera records direction and appearance. It does not measure distance at all.

But an enormous amount of the road is information printed for human eyes — lane markings, signs, signals, brake lights, indicators — and nothing else can read any of it. Classification is the camera’s unique contribution. A lidar reports that something is there and precisely where; only a camera says it is a child.

Range must be inferred, from apparent size against an assumed real size, or from where an object meets an assumed flat ground plane. Both assumptions break — on a crest, on a slope, or with an object of unfamiliar size.

Ultrasonic

Sound pulses, timed on return. Slow and spreading rapidly, so useful over a few metres and no further. Within that they are cheap, robust and detect kerbs, posts and bollards that radar barely registers. Parking and low-speed manoeuvring, and nothing faster.

Where each one fails

Condition	Radar	Lidar	Camera	Ultrasonic
Dense fog	unaffected	badly degraded	badly degraded	n/a
Heavy rain and spray	slightly degraded	degraded	degraded	n/a
Darkness	unaffected	unaffected	degraded without lighting	unaffected
Direct low sun	unaffected	unaffected	can be blinded entirely	unaffected
Stationary object in lane	poor discrimination	good	good	n/a
Reading a sign or signal	impossible	impossible	the only option	impossible
Judging closing speed	measured directly	inferred over frames	inferred over frames	crudely

Why several of each

No single field of view serves every purpose. A long-range radar for cruise control looks far ahead through a narrow cone and will not see a car cutting in from the adjacent lane until late. A wide sensor sees that cut-in at once but cannot resolve anything at distance.

So a vehicle carries a mixture — long and narrow forwards, short and wide at the corners, ultrasonic close in — and the overlaps are deliberate. Where two sensors see the same thing by different physics, agreement is strong evidence and disagreement is a signal in its own right.

Calibration

A sensor’s measurement is worthless unless the system knows precisely where that sensor is pointing. An error of one degree in a forward radar’s alignment puts an object roughly two metres sideways at a hundred metres — the difference between your lane and the next one.

This is why a windscreen replacement or a bumper repair now requires a recalibration procedure against a patterned target at a measured distance, and why the bay must be level with room around the vehicle. It is not a formality; an uncalibrated system is confidently wrong.

What to take away

• Separate what a sensor measures from what the system infers. The first is solid, the second carries assumptions.
• Radar measures range and closing speed directly, and sees through fog, but resolves position and identity poorly.
• Lidar measures geometry directly and works in the dark, but scatters badly in fog, spray and snow.
• A camera measures no distance at all and is the only sensor that can read meaning.
• Ultrasonic works over a few metres, which suits parking and nothing else.
• Sensors fail silently — detecting the degradation is harder than handling the data.
• The failures are uncorrelated, and that is the whole reason for combining them.

Next, how those separate views are turned into a single picture — sensor fusion and autonomy levels.