The Pixel 9's AI features have made creating realistic and believable fake photos trivial, and Google's safeguards are astonishingly inadequate
Our basic assumptions about photos capturing reality are about to go up in smoke. — Photography by Chris Welch / The Verge; AI-altered images made with Google's Magic Editor.
The VergeSarah Jeong
Context & Ripple Effects
Google had already moved from face swaps in Pixel 8’s Best Take feature to a broader Pixel 9 camera toolkit that includes Reimagine and Autoframe. The story matters because it tests whether that product progression has outrun the controls needed when edits can look like ordinary photographs.
Pixel 9 users can more easily produce photorealistic altered images, while recipients lose a simple visual basis for deciding whether a photo records an event or has been materially changed.
Google faces immediate pressure to make its editing safeguards and disclosure signals more legible, since the reported weakness is not merely output quality but the ease of passing an edit off as authentic.
Second-order effects
Competing phone makers and photo-editing apps are pushed to balance increasingly capable generative camera features with clearer provenance and friction around deceptive uses.
As Magic Editor reaches a wider Google Photos audience, the trust issue shifts from a device-specific concern to an ecosystem problem for people sharing and viewing images.
Third-order effects
If consumer photo tools keep reducing the cost of convincing manipulation, visual authenticity will increasingly depend on provenance and platform disclosure rather than on a viewer’s ability to inspect pixels.
The durable competitive question becomes whether widely distributed AI editors can pair creative convenience with credible assurance; the corpus shows Google beginning to add image-level AI information, but its effectiveness remains unproven.
The trend: Consumer AI is turning photo manipulation into a default camera capability, making provenance and operational assurance as important as the editing feature itself.
The “Reimagine” feature on Google's new Pixel 9 lineup is incredible. It's so impressive that testing it has left me feeling uneasy on multiple occasions. With a simple prompt, you can add things to photos that were never there. And the company's Gemini AI makes it look astoni…
“The default assumption about a photo is about to become that it's faked, because creating realistic and believable fake photos is now trivial to do. We are not prepared for what happens after.” https://www.theverge.com/...
I've gotten a lot of pedantic feedback on this but I'm disappointed that no one has challenged my assertion that the library of alexandria would fit on my nintendo switch sd card. I ran the numbers and everything. I was so excited to argue about it. https://www.theverge.com/...
This tech itself is crazy but even more insane to me is how it's being shoved in front of us for no reason. I'm struggling to think of any actual use cases for an AI photo machine on my phone. https://www.theverge.com/...
I know things don't usually turn out as badly as tech pessimists predict, but in this case, I'm really not keen on the increasing ease with with reality can be transformed. — Easy to predict ‘saw your boyfriend with a different girl’ type use cases for this at the mild end of o…
Making photorealistic AI edits easy at scale isn't just the what-is-a-photo apocalypse. It's a truth apocalypse. @aliteralpsyduck on what happens when the default understanding of photos goes from “real” to “fake” -> https://www.theverge.com/... [images]
FWIW today I think Meta is doing a better job here wrt AI than most. But it's an arms race and other parties are definitely going straight to the bottom.
Things changed a lot internally after Cambridge Analytica. A lot of XFN pre-mortems to brainstorm on how any feature could possibly be manipulated and used for evil. But then, that was 2018 and people have short memories.
Remember, this stuff only needs to look “real” enough to all the people who still routinely post on Facebook. Made on Pixel 9 Pro XL in minutes. (AI tag added by me, not Threads.)
That's what I struggle with as well. Who is asking for this? Yes you can photoshop and adjust images but are masses of people begging to be able to modify photos like this? Just feels reckless.
There's a ton of super irresponsible Magic Editor features on the Pixel 9 but Add Me is a contained 2-image photo taking environment that only works with the people with you at the moment you're using the feature. I'd read this if you want to see the real irresponsible stuff, if…
There were a number of decisions made at Facebook about Sharing and Privacy defaults over a decade ago where PMs (including myself) said things like... “But no one would ever do that.” “Most people are good people and won't behave that way.” “Well, people already do that in oth…
Reading people claim that Google's Magic Editor isn't a problem reminds me of how Facebook was surprised at the suggestion that anyone would livestream a murder, only for the Christchurch shooting to happen just a few short years later https://www.theverge.com/...
Welcome to a watershed moment for generative AI. Right on your new phone. The Pixel 9 series can bend, twist, and “Reimagine” reality easier and faster than any consumer tech device I've ever used. Like I said, I hope we're ready for this. — https://www.theverge.com/... [im…
This article feels like alarmist journalism. Let's break it down: ‘Look what the AI did!’ ... after you explicitly told it to do that. At what point do we shift responsibility from tool makers to users? We can't bubble wrap every technology. That said, the ease of bypassing [imag…
No one's ready for this // Whenever someone says “no one is ready” they never mean themselves and are trying to speak for a mass of others who will not be as sophisticated at understanding risks and using that as a way to try to have technology controlled in some way. This goes
@c_r_5 @sarahjeong ok, go ahead and create all of them in Photoshop. Chances are you don't have the skills to make them this convincing, or the hours of effort it would require to source the proper photos, create realistic shadows, and other effects. Google's tools do this in sec…
No one's ready for this. Look at what Google's AI image tools are doing on the new Pixel 9 devices. These are very convincing fake images. As @sarahjeong puts it: “We are fucked.” https://www.theverge.com/... [image]