Humans drive with intuition honed over years. Autonomous vehicles drive with a stack of neural networks processing 5 GB/sec of sensor data and deciding steering, braking and acceleration 100 times per second. How does that stack think?
The 4 Layers of Autonomy
Transformers fuse camera, LiDAR and radar to detect objects, lanes, traffic lights and free space — 3D bounding boxes at 60 fps.
Forecasts where every agent will be 5 seconds ahead. Generative models simulate cyclist veering, pedestrian hesitating.
Chooses trajectory that is safe, comfortable and legal. Modern planners are diffusion models trained on 10M+ human drives.
Converts plan to actuator commands with model predictive control — correcting 100 Hz for wind, slope, tyre slip.
From Rules to Foundation Models
Pre-2023 stacks were rule-heavy (“if red light, stop”). 2026 stacks are end-to-end foundation models — like Tesla’s HydraNets or Wayve’s LINGO — that ingest raw video and output trajectory directly, learning nuance from fleet data. Reinforcement learning in simulation (CARLA, Waymax) then refines for rare, dangerous scenarios without real-world risk.
Explainability & Safety
The black-box worry is real. New methods provide post-hoc explanations: saliency maps show which pixels the AI fixated on; language models like DriveGPT narrate decisions: “I yield because pedestrian is looking at phone.” EU’s AI Act now requires such explainability for high-risk AI — including AVs.