AV: Sensor Fusion — Cameras, Radar, and LiDAR

# AV: Sensor Fusion — Cameras, Radar, and LiDAR

The sensor debate in autonomous vehicles isn't purely technical. It's also a commercial and strategic debate between companies with different resource positions and different theories about where the technology is going. Understanding why Tesla, Waymo, and Mobileye made different bets requires understanding what each sensor type actually does and where it fails.

## Cameras: Cheap, Dense, Passive

Cameras are the densest source of semantic information available to an AV system. A high-resolution camera at 60 fps generates enormous amounts of data that, with sufficient neural network capacity, can identify lane markings, traffic signals, pedestrians, vehicles, road surface conditions, and hundreds of other relevant scene elements.

The advantages: cost (high-resolution automotive cameras are now in the $50-200 range), data density, passive operation (they don't emit anything), and direct capture of the visual information that human drivers rely on. Road infrastructure was designed for vision — signs, markings, signals all encode information visually. A system that can read visual information well can leverage that entire infrastructure.

The disadvantages: cameras fail in low-light conditions (headlights help but don't fully compensate), fail in adverse weather (rain on the lens, snow obscuring markings, fog reducing range), and struggle with direct sun glare. Cameras also don't natively measure depth — range estimation requires stereo imaging, structure-from-motion algorithms, or fusion with other sensors.

## LiDAR: Accurate 3D, Expensive, Mechanical Complexity

LiDAR (Light Detection and Ranging) fires laser pulses and measures the return time to construct a 3D point cloud of the environment. The result is accurate depth measurement at 30+ meters in all lighting conditions (it works at night), with a spatial accuracy that cameras can't match without significant processing.

Early Waymo-era LiDAR units (the spinning dome sensor on the original Google self-driving car) cost $75,000 each. Solid-state LiDAR — with no mechanical moving parts — has reduced costs dramatically; Luminar's Iris and Innoviz's InnovizTwo are in the $1,000-3,000 range at volume. Still not cheap, but the cost trajectory is improving.

LiDAR's failure modes: rain and snow still create noise (though it's better than camera performance in weather), the point cloud doesn't natively include semantic information (you see a point cloud of an object, but you need additional processing to know if it's a pedestrian or a tree), and the resolution is still lower than camera imagery for detailed scene understanding.

## Radar: All-Weather, Low Resolution

Radar has been in automotive applications since the 1990s as adaptive cruise control sensors. It's robust in all weather conditions (radio waves pass through rain and snow), accurate for velocity measurement (Doppler radar), and inexpensive in its basic form.

The limitation is angular resolution. Standard automotive radar resolves vehicles as objects but doesn't provide the fine-grained spatial detail needed for AV decision-making. 4D imaging radar (which adds height/elevation to the traditional range-velocity-azimuth measurement) is improving this, and Arbe Robotics, Continental, and others are bringing higher-resolution radar to market.

Radar and camera complement well: camera provides semantic richness, radar provides all-weather robust range and velocity. This combination is what most ADAS systems use.

## Tesla's Camera-Only Bet

Tesla removed ultrasonic sensors in late 2022 and is camera-only. The strategic rationale:

**Cost**: eliminating LiDAR keeps vehicle cost competitive. A Waymo vehicle with full sensor suite costs significantly more per unit than a Tesla.

**Data**: Tesla has billions of miles of camera-only fleet data. Training neural networks requires data; Tesla has more relevant data than anyone, but it's all camera data. Adding LiDAR would require retraining on sensor-fused data sets at enormous cost.

**Philosophy**: Elon Musk's stated view is that LiDAR is a crutch that prevents the development of robust vision-based approaches. If the world was built for human vision, the solution should use vision. The argument is that camera-based neural networks, with sufficient data and compute, can exceed human-level performance in the same sense domain.

The counterargument from Waymo's perspective: redundancy is not a crutch; it's how you handle sensor failure. A camera that gets scratched, occluded by mud, or overexposed by direct sun is a safety failure mode. LiDAR doesn't have those failure modes.

## Waymo's Redundant Sensor Architecture

Waymo uses cameras, LiDAR, and radar in a sensor-fused stack. The philosophy is explicit: independent sensor modalities that each provide safety-relevant information mean that individual sensor failures don't create blind spots.

The trade-off is cost and complexity. The Waymo Driver is not going into $35,000 consumer vehicles in the near term because the sensor suite and compute make unit economics unfeasible at that price point.

Mobileye's EyeQ platform, used in thousands of production vehicles, has historically been camera-primary with radar support — closer to the Tesla philosophy than the Waymo philosophy. Their next-gen Chauffeur/Drive programs add LiDAR for L3/L4 applications.

Neither camp is obviously wrong given what we know today. The right answer likely depends on whether the AI/compute path can outperform the physical redundancy path at sufficient scale — and that question is still being empirically tested.

AV: Sensor Fusion — Cameras, Radar, and LiDAR

// COMMENTS

ON THIS PAGE