Study: programmers who use GitHub Copilot and other AI tools produce less secure code than those who do not, despite believing their code has no safety issues
At the same time, tools like Github Copilot and Facebook InCoder make developers believe their code is sound
Context & Ripple Effects
GitHub Copilot had already gained traction after its launch, with GitHub saying the tool accounted for a meaningful share of new code and that many early users kept using it. The security finding therefore tests an adoption story built around faster code generation rather than merely a niche experiment.
The study’s central tension is not only weaker security outcomes but misplaced developer confidence. GitHub later emphasized more secure Copilot suggestions as Copilot-generated code expanded, making this an early signal that generation quality and assurance would become separate product requirements.
First-order effects
- Developers using Copilot, InCoder, and similar tools face a direct verification burden: the study found less secure code even when participants judged their output safe.
- GitHub’s efficiency narrative is complicated by evidence that assisted coding can reduce users’ ability to identify security flaws in generated code.
Second-order effects
- GitHub is pressured to compete on security-aware suggestions and review support, not just code-completion adoption, as reflected in its later security-focused Copilot update.
- Teams adopting AI coding tools must shift more security review toward generated output, limiting how much of the productivity gain can bypass human checking.
Third-order effects
- As AI-generated code becomes a larger share of development work, software delivery is likely to be constrained less by producing code than by establishing trustworthy validation for it.
- The durable divide in AI coding will be between tools that accelerate drafting and workflows that can demonstrate the resulting code is safe enough to ship.
The trend: AI-assisted programming is moving from a code-generation race toward a verification bottleneck, where developer confidence cannot substitute for security assurance.