Stanford study: programmers who used AI tools like GitHub Copilot to solve a set of coding challenges produced less secure code than those who did not
Context & Ripple Effects
Since Microsoft and OpenAI's 2021 launch of GitHub Copilot, adoption has compounded fast — GitHub reported as much as 30% of new code on its network written with the tool within months, and Wired later profiled it writing 35% of its users' code. The Stanford study lands on top of that installed base: the people generating that code are producing less secure output than non-users, and — the sharper finding — believe their code has no safety issues.
The perception gap matters more than the vulnerability rate alone. If Copilot users can't tell when their code is unsafe, the human-review layer that normally catches flaws is working blind on exactly the code most likely to need scrutiny.
First-order effects
- Teams running Copilot in production are shipping code with unflagged vulnerabilities today, because the study shows users' confidence in their code's security is misplaced rather than earned.
- GitHub faces a direct credibility problem: the tool driving its headline adoption metrics is now documented as a security liability, not just a productivity one.
Second-order effects
- GitHub's response came quickly — a February 2023 update promising better and more secure code suggestions alongside new usage figures showing Copilot generating 46% of developers' code files — making security quality the explicit battleground for assistant retention.
- Enterprise buyers gain leverage to demand security guarantees and audit trails for AI-generated code, opening space for security-scanning vendors to position themselves as the mandatory complement to any coding assistant.
Third-order effects
- If assistants keep writing half of all code while their users can't assess its safety, security assurance structurally migrates from developer judgment to automated review pipelines — making whoever controls those pipelines, not the IDE vendor alone, the gatekeeper of software supply-chain risk.
The trend: AI coding assistants are scaling faster than the security practices around them, forcing vendors like GitHub to treat suggestion quality as a trust product rather than a convenience feature.