A study of eight AI search engines found they provided incorrect citations of news articles in 60%+ of queries; Grok 3 answered 94% of the queries incorrectly
We Compared Eight AI Search Engines. They're All Bad at Citing News. — AI search tools are rapidly gaining in popularity …
Columbia Journalism Review
Context & Ripple Effects
An earlier review of AI search tools found uneven performance: weaker navigational results than Google, but potentially useful citations for exploratory queries. This study challenges whether those citations can reliably carry the evidentiary weight users may assign to them.
The finding also arrives days after a chatbot audit on pro-Kremlin false claims showed leading assistants could repeat coordinated misinformation. Together, the coverage puts source attribution—not just answer fluency—at the center of AI search quality.
First-order effects
Users of the eight tested AI search engines face a high risk that a cited news source does not support the answer, weakening citations as a shortcut for verification.
Grok 3 is especially exposed by the study’s result: its 94% incorrect-answer rate makes its news-citation experience difficult to treat as dependable on the tested queries.
Second-order effects
AI search providers will face pressure to improve retrieval, source matching, and citation display rather than compete primarily on concise synthesized answers.
Publishers whose reporting is cited incorrectly may need to treat AI answer surfaces as a distribution and attribution risk, alongside the existing problem of AI-generated sites appearing in Google News.
Third-order effects
If repeated audits continue to find attribution failures, AI search quality will increasingly be judged by verifiability and provenance, not merely whether an answer sounds plausible.
The pattern strengthens the case for publisher controls and clearer accountability around how news content is retrieved, represented, and linked in AI interfaces, though the effective standards remain unsettled.
The trend: AI search is shifting from an answer-generation race toward a trust-and-provenance test for news distribution.
Key detail: “We deliberately chose excerpts that, if pasted into a traditional Google search, returned the original source within the first three results.” #againstAI #noAI [embedded post]
Another good study showing that LLM chatbots are not providing correct results that could easily be found via search engine like Google. And these bots write their answers confidently, which fools people into believing the answer is correct. [embedded post]
Tow Center for Digital Journalism study found that “AI” chatbots provided incorrect answers to more than 60 percent of queries, with Musk's Grok 3 responding to 94 percent of the queries incorrectly. — “Premium chatbots provided more confidently incorrect answers than their fre…
Every day it's a new study that shows something like this, and then also every day is a parallel story of a media outlet absolutely just flat-out hardcore head-down maximum effort ultra-sprinting to deeply integrate this random sentence generator technology into their reporting […
Meanwhile, I think so far we're overly impressed by the confidence, rather than competence, of generative AI. Yet we're rushing ahead to rewire the web, and the ways we find and consume information, well before these tools have proven to be reliable and safe. — bsky.app/profil…
A few weeks ago I attended a conference where @upenn.edu Wharton professor Ethan Mollick confidently asserted to me and many other public K-12 education leaders that AI no longer makes up fake citations. When i quoted him on that, he called me a bunch of mean names. [embedded p…
Was toying with one the other day and asked it to provide proof that it didn't hallucinate any of the citations, it just reverted to circular logic “these are good citations because they are good” [embedded post]
Staff at @towcenter.bsky.social conducted tests on eight generative search tools with live search features to assess their abilities to accurately retrieve and cite news content, as well as how they behave when they cannot. — Result in brief: they're not very good at it!
🔍 AI Search Engines: popular, but potentially problematic. — A new study from the @towcenter.bsky.social reveals the hidden pitfalls in AI-powered search, like misattribution, misinformation and missing citations: — www.cjr.org/tow_center/w...
Chat-based search apps are often “confidently wrong” in attributing and citing news stories... particularly Grok, which performs abysmally. Terrific study from CJR: — www.cjr.org/tow_center/w...
NEW: We ran 1600 queries across eight generative AI search tools to see how they cite news content. They all did pretty poorly. (w/ @aisvarya17.bsky.social for @towcenter.bsky.social in @columjournreview.bsky.social) www.cjr.org/tow_center/w...