San Francisco sends legal notices to Apple and Google, demanding they take down 13 AI apps used to make deepfake nude images; Google says it deleted five apps
Context & Ripple Effects
This escalates a San Francisco enforcement campaign that previously targeted websites used to generate nonconsensual deepfake pornography. It also follows earlier platform removals: Apple took down three image-generation apps after reporting on their advertised capabilities, while Google had removed URLs following complaints.
The focus has shifted from individual content links and standalone sites to mobile-app distribution. That puts Apple and Google’s store-review and takedown processes directly in the enforcement path for explicit-AI harms.
First-order effects
- Apple and Google face a formal demand to remove 13 identified apps; Google says it has already deleted five of them.
- Developers of the named apps risk losing access to major mobile distribution channels, while users’ ability to obtain those apps through the stores is immediately constrained.
Second-order effects
- The notices pressure both app stores to reassess how they detect apps marketed for or capable of producing nonconsensual nude imagery, rather than relying only on complaints after listing.
- Other generative-image app developers may face stricter scrutiny of product claims, moderation controls, and store disclosures, particularly where their tools can be repurposed for intimate-image abuse.
Third-order effects
- If city-level action continues to reach app marketplaces as well as websites and search results, distribution intermediaries will become a central practical enforcement layer for explicit-deepfake rules.
- The pattern points toward governance based on access points—app stores, search, and hosting—while the effectiveness of that approach will depend on whether removals can keep pace with new or relisted services.
The trend: This is part of a broader shift from regulating explicit AI deepfakes in principle to pressing the platforms that distribute and surface the tools to operationalize enforcement.