Google says the vast majority of AI Overviews provide high-quality information and many of the viral examples have been uncommon queries or have been doctored

Step 1: Google rolls out a new AI-powered product. Step 2: Users quickly find the product's flaws and point them out with social-media posts, which become news stories.

Business Insider 2024-05-25 Peter Kafka

Context & Ripple Effects

Google’s response frames the AI Overviews backlash as a problem of atypical or manipulated prompts rather than a broad failure of the product. That defense matters because it seeks to preserve confidence in a search feature being judged publicly through highly shareable edge cases.

The statement sits alongside Google’s swift removal of some AI Overviews and is followed by system changes aimed at nonsensical queries and satire. Together, the coverage shows a rollout moving from public rebuttal to more explicit operational controls.

First-order effects

Google must manage the gap between its claim that most results are high quality and the visible examples that have undermined trust in AI Overviews.
Users and publishers receive a clearer signal that some queries may be excluded or handled differently as Google applies detection for nonsensical queries and limits on satire or humor content.

Second-order effects

The episode raises the practical bar for search AI competitors: a broad rollout requires not only answer quality, but safeguards against adversarial, ambiguous, and context-poor prompts.
Product teams are pushed toward tighter query gating and faster correction loops, which can narrow early feature coverage in exchange for fewer public failures.

Third-order effects

If this pattern persists, consumer AI search will be differentiated as much by operational governance—testing, query eligibility, and rapid remediation—as by model capability.
The larger risk is that repeated, viral failures make confidence in AI-generated search answers contingent on visible controls rather than provider assurances alone.

The trend: AI search is shifting from a model-quality race toward governed deployment, where distribution scale makes edge-case handling a core product capability.

Discussion

@jbillinson Josh Billinson on threads
Google should obviously not be telling people to make pizza with Elmer's glue. Everyone knows the secret to a good pizza is using New York tap glue.
@andrewwebster Andrew Webster on threads
cool how google turned search into the worst system shock sequel
@fredericl Frederic Lardinois on threads
I'm trying to get worked up about the mistakes Google's AI Overview makes, but I'm failing. In day-to-day usage, the examples I see are alright. I'm more worried that it's actually quite good (or at least good enough and getting better) and takes away clicks from publishers.
@bzamayo Benjamin Mayo on threads
Google AI overviews aren't in my region yet I'm missing the opportunity to engagement farm off its stupidity
@alexhcranz Alex Cranz on threads
companies continue to poison any goodwill they have by pushing out products that aggressively lie to you by also (possibly) stealing all your content to lie to other people. the companies don't see it that way—but the general public sure as shit is
@technicallymims Christopher Mims on threads
the fact that nearly all of us into believing that Google's generative search results told people to jump off a bridge is a near-perfect illustration of the fact that AI has *already* confused us into not knowing what is true anymore, just by the fact if its existence
@quillmatiq Anuj Ahooja on threads
Microsoft wanted Google to dance but it turns out Google has two left feet tied at the ankles 💃🏼
@drewharwell Drew Harwell on threads
If there's one thing I know about Google, it's that people always search for normal stuff
@kylie.robison Kylie Robison on threads
Google confirms it manually removes odd AI search results, and I discussed the company's super-smart-search ambitions with Gary Marcus (who remembered a time when a Meta AI model told people to eat glass) https://www.theverge.com/...
@alex.stamos Alex Stamos on threads
I strongly recommend not amplifying any claims about Google's AI output without trying the query yourself, as lots of fake screenshots are circulating. Even The NY Times fell for it:
@wongmjane Jane Manchun Wong on threads
another crazy Google AI Overview result lol
@mhwilliams.bsky.social M.H. Williams on bluesky
Which is all well and good as a statement, but users can actually do the queries themselves and it's quite clear they still have the same problems that all LLMs do. The hallucinations are rampant and you're not really going to be able to get rid of them at scale. [embedded post…
@m33uhh.bsky.social Mia on bluesky
I thought one of the main benefits of AI generated results was being able to ask incredibly specific questions, aka “uncommon queries”
@dell.bsky.social Dell Cameron on bluesky
The translation is that Google knows it is supplying misinformation & that it has an acceptable failure rate. — What is a “vast majority”? Is it 80%? 90%? How many tens of millions of results does that leave? This is an intentional design flaw & when harm results it should …
@mattvallone.bsky.social Matt Vallone on bluesky
It's been odd watching them transition to “Do be evil”. [embedded post]
@monospace.com Hens Breet on bluesky
The problem is that we don't know which is which so it's safest to assume it's all compromised.
@ernie.tedium.co Ernie Smith on bluesky
More generally, I want an unadulterated experience when I look at Google. The level of visual noise has been rising, but this particular feature pushes it over the edge. [embedded post]
@ablum.bsky.social Andrew Blum on bluesky
For me, it's not just the inaccuracies. The viral examples we've seen remove any and all mystique AI has. I mean if you trace the mistake to a single Onion article... [embedded post]
@douglasmack.bsky.social Doug Mack on bluesky
The issue isn't that AI is sometimes wrong (per se), the issue is that all info is presented with the ILLUSION of expertise but stripped of all context or attention to details like “is this source reliable or is it complete bullshit that will end up killing me in five different w…
@brendansinclair.bsky.social Brendan Sinclair on bluesky
Oh, well that's ok if it's just edge cases then. If Google's AI only tells you to do incredibly dangerous things in .01% of cases, and Google handles 3 billion searches a day, that's only like 300,000 people a day being told to drink bleach or whatever. Seems fine. [embedded p…
@ernie.tedium.co Ernie Smith on bluesky
Did they say anything about &udm=14? [embedded post]
@grudgie.bsky.social @grudgie.bsky.social on bluesky
“Most of our information is good” simply is a generous way of saying “you cannot rely on our information.” [embedded post]
@calizaybak.bsky.social @calizaybak.bsky.social on bluesky
Who wants a search tool that can handle “uncommon queries”? — And doctored! Except all the examples I've seen here have a comment that says “OMG, I tried it myself!” which is a thing anyone can instantly do so doctored examples would get dunked-down instantly. [embedded post]
@cdarklock.bsky.social @cdarklock.bsky.social on bluesky
Uncommon queries are YOUR FUCKING JOB — Nobody is asking your search engine about things ANY JACKASS ALREADY KNOWS — What the entire fuck [embedded post]
@shevek.bsky.social @shevek.bsky.social on bluesky
you see this is fine and good because the big query machine doesn't *always* tell you formaldehyde is a great decongestant [embedded post]
@joshgrant.online Josh Grant on bluesky
Also, basically all of their customers of Google Cloud in Australia haven't seen any problems! [embedded post]
@braak.bsky.social Braak on bluesky
Google at long last providing the people what they want, answer to their questions that are mostly true [embedded post]
@beijingpalmer.bsky.social @beijingpalmer.bsky.social on bluesky
‘by 2024 Baidu will be better at search than Google.’ — 'wow! So Baidu's really going to improve?' — 'I didn't say that' [embedded post]
@zitron.bsky.social Ed Zitron on bluesky
oh okay no problem thank you!!!!!!!!! [embedded post]
@pkafka Peter Kafka on x
Oh and here's Google's newest statement re: Ai search. Full speed ahead! [image]
@mimbsy Caroline Mimbs Nyce on x
Google's new AI Overview told me that the one-year survival rate for chemotherapy is 52 percent. That statistic is from a real scientific paper about head and neck cancer, which it mistakenly applied it to *all* cancers. [image]
@pkafka Peter Kafka on x
I'm old enough to remember when Google bailed on an ai product after public backlash. In February. But I think they're fully locked into this one: gripes are one thing but backing down would be a disaster for management. Speaking of disaster: “let's discourage people from usin…
@ben8128 Ben Eidelson on x
The bad results are sometimes funny and, yes, embarrassing to Google. And I also find AI overviews to be a tremendously useful feature. Two things can be true at once. Especially for a beta product.
@willoremus Will Oremus on x
“With the generative AI from Google, OpenAI's ChatGPT and Microsoft's Copilot, you should assume that they're wrong until proved otherwise.” — @ShiraOvide on how to navigate a world in which Google tells you to eat rocks and glue https://www.washingtonpost.com/ ...
@chaykak Kyle Chayka on x
it's hard to overemphasize this: Google and OpenAI have no plan for how or why people will generate *new, correct information* in the age of generative AI search
@jjvincent James Vincent on x
it's completely damning of google's principles that its defense for the failure of AI search is that the queries being asked are “uncommon.” like a library telling you the more infrequently a book is taken out, the less you should trust it https://www.theverge.com/... [image]
@tomwarren Tom Warren on x
I know everyone is laughing at Google for its AI search results suggesting we should eat rocks. The funniest part is that the rocks thing also appeared on regular featured snippets, so it's not just AI results. Perhaps Google is more broken than we think [image]
@ethank Ethan Kaplan on threads
“Apple is late to AI”
@paul_rietschka Paul Rietschka on threads
“Google inadvertently reinvents expert systems.” This will end well.
@jonathangarelick Jonathan Garelick on threads
Google's generative AI team is writing a crazy amount of if statements right now
@reckless1280 Nilay Patel on threads
Google is manually removing weird AI answers in search - probably ruining whatever margin bringing the cost of an AI query down 80 percent over the last year saved. Lots in this @kylie.robison piece! https://www.theverge.com/...
@kylie.robison Kylie Robison on threads
Google promised a better search experience — now it's telling us to put glue on our pizza Last year, Google said that AI was the future of search. What's the point, though, if the search seems dumber than before? https://www.theverge.com/...
@laguettler.bsky.social @laguettler.bsky.social on bluesky
Thank goodness the company that created a dangerously unreliable product will be determining in how much danger is worth the financial and reputational investment they sunk into it. [embedded post]
@darthbluesky.bsky.social @darthbluesky.bsky.social on bluesky
will an AI be making these decisions over what will be removed [embedded post]
@darthbluesky.bsky.social @darthbluesky.bsky.social on bluesky
how dare users try to trip up our experimental AI that we are training on these same users to make a lot of money for ourselves where is the loyalty

Chronicles