Experts say the lack of publicly available data makes it difficult to verify claims by Tesla and others about the safety of their autonomous driving tech
Automakers and technology companies say they are making driving safer, but verifying these claims is difficult. Tweets: @2plus2make5 , @parismarx , @davidzipper , @garymarcus , @noahgoodall , and @iihs_autosafety Tweets: Emma Pierson / @2plus2make5 : https://www.nytimes.com/... Tesla: drivers crash less when using Autopilot. Obvious issue: Autopilot is used on highway/good conditions. Tesla could easily report crash rates when controlling for these confounds, but they don't. This has always made me suspicious. 1/2 Paris Marx / @parismarx : “Publicly available data that reliably measures the safety of these technologies is scant. American drivers—whether using these systems or sharing the road with them—are effectively guinea pigs in an experiment whose results have not yet been revealed.” https://www.nytimes.com/... David Zipper / @davidzipper : “American drivers — whether using these [driver assistance] systems or sharing the road with them — are effectively guinea pigs in an experiment whose results have not yet been revealed.” https://www.nytimes.com/... Gary Marcus / @garymarcus : two paragraphs buried in @CadeMetz's newest article on autonomous driving that should make @tesla shareholders pretty nervous: source: https://www.nytimes.com/... https://twitter.com/... Noah Goodall / @noahgoodall : Measuring safety of vehicle automation features is hard, mostly due to lack of data. One needs crashes (of a defined severity threshold) and mileage (ideally w/ & w/o system engaged) over a consistent road type (ACC used on freeways, which are 2x safer) 1/ https://www.nytimes.com/... Iihs / @iihs_autosafety : While IIHS & HLDI research show definitive safety benefits from crash prevention tech like automatic emergency braking, a payoff is not immediately clear for newer partially automated driver assistance systems. The story from @nytimes @CadeMetz https://www.nytimes.com/...
Context & Ripple Effects
Tesla's safety marketing has leaned on official-sounding numbers before: back in 2018, [[a:929280|NHTSA distanced itself from its own finding, long cited by Tesla, that Autopilot reduces crash rates by up to 40%]], an early sign the company's headline statistics outran their evidentiary footing. This piece extends that thread to the present: researchers including Emma Pierson point out that Autopilot miles skew toward highways and good conditions, so raw crash-rate comparisons flatter the system — and Tesla publishes no figures controlling for those confounds.
The verification gap matters because the stakes are already enormous: investors have put an estimated ~$100B into self-driving startups with little to show for it, and regulators keep having to generate their own evidence, as in [[a:877728|NHTSA's probe of Full Self-Driving covering 2.4M vehicles after four reported low-visibility crashes]]. When companies control the data and outsiders can't check the claims, every safety assertion becomes a matter of trust rather than proof.
First-order effects
- Tesla's Autopilot safety claims are effectively unverifiable by outside researchers like Emma Pierson, Noah Goodall, and bodies such as IIHS, because the company reports crash rates without controlling for where and when the system is used.
- Regulators are pushed into resource-intensive self-investigation — the pattern behind NHTSA's 2.4M-vehicle Full Self-Driving probe — instead of being able to audit manufacturer-published data.
Second-order effects
- Rival automakers making similar driver-assistance claims face the same credibility discount, rewarding whoever volunteers independently checkable safety data first and pressuring laggards to follow.
- Investors weighing the ~$100B already sunk into autonomy startups must price in the absence of reliable third-party safety metrics, raising the effective cost of capital for companies whose claims can't be validated.
Third-order effects
- If unverified claims remain the norm, the industry's credibility may consolidate around the incremental advanced-driver-assistance pathway some experts favor over moonshot Level 4 bets, since gradual systems accumulate auditable real-world records faster.
- Sustained scrutiny points toward standardized, publicly reported safety telemetry as a regulatory requirement — turning safety-data disclosure from a marketing choice into a compliance obligation.
The trend: Autonomous-driving safety is shifting from company-authored statistics toward externally verifiable evidence, as regulators and researchers refuse to take manufacturers' crash-rate claims at face value.