Self-driving startup Waabi unveils Copilot4D, a generative AI model trained on lidar data to predict traffic flow around a vehicle 5-10 seconds into the future
Self-driving company Waabi is using a generative AI model to help predict the movement of vehicles, it announced today. The new system, called Copilot4D, was trained on troves of data from lidar sensors, which use light to sense how far away objects are. … X: Daryn Nakhuda / @daryn : Really excited to share this amazing work by the Waabi team! Like LLMs have done for text applications, Copilot4D enables a revolution for systems acting and interacting in the real world. #AI Lunjun Zhang / @zhanglunjun : Excited to share a new foundation model for self-driving, Copilot4D. Paper: “Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusion”, accepted to #ICLR2024 Arxiv: https://arxiv.org/... Blog: https://waabi.ai/... Video: https://waabi.ai/... Raquel Urtasun / @raquelurtasun : I'm thrilled to unveil @Waabi_ai latest research, Copilot4D: the first foundation model that explicitly reasons in both 3D space and time and leverages LiDAR. Big step forward in bringing generative AI to the physical world. Learn more here: https://waabi.ai/...
Context & Ripple Effects
Waabi was launched around a thesis that an autonomous-driving platform could handle complex reasoning, and it later appeared among startups pursuing end-to-end AI learning for self-driving. Copilot4D makes that strategy more concrete by applying a generative model to lidar-derived representations of nearby motion.
The development matters because it shifts attention from perception of the current scene to forecasting how a 3D traffic environment may evolve over the next several seconds—a core input to driving decisions.
First-order effects
- Waabi adds a lidar-trained generative model to its autonomous-driving stack for predicting surrounding vehicles and traffic flow five to 10 seconds ahead.
- The company can test whether a foundation-model approach to 3D space and time improves the motion forecasts available to its driving system.
Second-order effects
- Other autonomy developers using end-to-end learning will face a clearer benchmark: not just detecting objects, but modeling plausible future traffic states from sensor data.
- The move raises the value of lidar data pipelines and evaluation methods that can measure forecast quality in complex, changing road scenes.
Third-order effects
- If such models prove reliable, autonomous-driving competition may increasingly turn on proprietary sensor datasets and world-modeling capability rather than narrowly separated perception and prediction modules.
- Because forecast errors can propagate into vehicle behavior, wider deployment would strengthen the case for rigorous safety validation and oversight of AI-driven driving stacks.
The trend: Autonomous-vehicle developers are adopting foundation-model techniques to build predictive world models that connect sensor data to real-world action.