Staff at AI companies are training/testing gun detection algorithms by recording staged “active shooter” events in their homes, raising concerns from ethicists
Companies are using bizarre methods to create algorithms that automatically detect weapons. Tweets: @vice , @gallagherfergal , @mtrc , @hypervisible , @oliviasolon , and @akinunver Tweets: @vice : Companies are using bizarre methods to create algorithms that automatically detect weapons. AI ethicists worry they will lead to more police violence. https://www.vice.com/... Fergal Gallagher / @gallagherfergal : “There is no dataset that would make this work. They are flawed, they are racist and they are being put into schools.” Gun Detection AI is Being Trained With Homemade ‘Active Shooter’ Videos https://www.vice.com/... Mike Cook / @mtrc : This is buck wild on several different levels, not least the fact that methodologically speaking this work should be rejected from any AI conference. But to train gun detection systems on footage of toy guns, in the US, in 2020, is a level of blinkeredness I cannot conceive of. https://twitter.com/... @hypervisible : There's little actual training data, so employees are running around cobbling together imaginary shooter scenarios and using that data instead. 🤯 These systems are going to get someone killed. https://twitter.com/... Olivia Solon / @oliviasolon : This is fascinating.... Although there's absolutely no evidence gun detection stops active shooters. https://twitter.com/... Akin Unver / @akinunver : “Even differences in the sun path between the northern and southern hemispheres and subtle differences in background scenery can cause the program to be less effective,” https://www.vice.com/...
Context & Ripple Effects
Gun detection vendors are repeating the playbook security contractors used when they marketed unproven facial recognition to US schools terrified of shootings — sell into fear first, validate later. VICE's reporting exposes how thin the foundation is: because no dataset of real active-shooter events exists, staff stage the scenarios themselves in their homes, which is why Fergal Gallagher's sources call the resulting systems flawed, racist, and already deployed into classrooms anyway.
The ethicists' warning about police violence lands where policing itself is automating: some California and Georgia departments have started flying autonomous drones to track vehicles and people, raising the same civil-rights objections. And the legal reckoning has a template — defense lawyers are already challenging Cybercheck's accuracy in thousands of US cases, showing how contested AI outputs become once they enter official decisions.
First-order effects
- Schools adopting these systems are relying on detectors whose training data is homemade staged footage, meaning every real-world alert rests on tests that never approximated an actual shooting.
- The named ethicists' critique shifts immediate scrutiny onto vendors to document error rates before a flagged 'weapon' triggers any response — misclassification is now an officer-use-of-force question, not just a technical one.
Second-order effects
- False positives funnel straight into policing: paired with the autonomous drones some departments now fly, an algorithmic detection can escalate to an armed response with little human verification in between.
- The Cybercheck dynamic is positioned to repeat — once gun-detection logs enter school discipline or criminal cases, defense attorneys gain a new class of AI records to challenge for accuracy and reliability.
Third-order effects
- The recurring pattern — safety-critical surveillance AI built on improvised or synthetic data and sold into schools and police before validation — points toward litigation-driven accountability rather than pre-deployment standards.
- If improvised training data remains standard for weapons detection, procurement contracts rather than model quality become the de facto safety guarantee, pressuring regulators to demand provenance for training data in public-safety AI.
The trend: Safety-critical surveillance AI is reaching schools and police on improvised training data, leaving courts, not validators, to test what vendors never did.