Experts explain why there's no magic formula to always distinguish human-written and AI-written text, meaning AI writing detectors can only make a strong guess
Can AI writing detectors be trusted? We dig into the theory behind them. — If you feed America's most important legal document …
Ars Technica Benj Edwards
Related Coverage
- Why AI Writing Detectors Find the US Constitution Written by An AI Tool Cryptopolitan · John Palmer
- Nice piece covering the futility of AI detectors at school. Don't use them. Seriosuly. — My quote at the end covers exactly how I feel … Ethan Mollick
- This is a resounding debunk of the emerging business of AI detecting tools. Too many false positives make it very difficult for good students … Rupert Breheny
- AI detectors are snake oil. If someone is trying to sell you software that claims it can tell if text is written by ChatGPT then you've met someone who thinks you're a fool who needs to be separated from your money. — https://arstechnica.com/... @carnage4life@mas.to · Dare Obasanjo
- @Techmeme — Adversarial training. — There are two groups: AI writers, who try to imitate human writing, and a group that's trying to distinguish between AI and human writers. — The group trying to separate gets a point if they choose the human-written samples. … @Chip_Unicorn@im-in.space · Chip Unicorn
- AI detectors for text are mostly snake oil due to how it's determining humanness: perplexity and burstiness measurements https://arstechnica.com/... @selenamarie@fiasco.social · Selena Deckelmann
- A must bookmark link for all students, just in case they come across any overzealous (or paranoid) profs https://arstechnica.com/... @koutropoulos@scholar.social · Apostolos K.
- Ars Technica does a great job of making the issues with AI detection for written texts understandable. — https://arstechnica.com/... I'm now waiting for Constitutional originalists to say “As the founding AIs intended” in an argument... @davidseidl@mstdn.social · David Seidl
- Anyone who comes to you saying they've created an AI detection system for checking if a given piece of text was written by AI or not and it's foolproof, don't believe a word of what they say. These detectors think the US Constitution was written by Chat GPT. I'd call that an abject failure. … @absamma@toolsforthought.rocks · Abraham
Discussion
-
@mmitchell.bsky.social
Margaret Mitchell
on bluesky
For this article from @benjedwards.bsky.social , I got to spend some time explaining the fundamental “AI” concept of “perplexity”. https://arstechnica.com/...
-
@benjedwards@mastodon.social
Benj Edwards
on mastodon
As educators panic about students using ChatGPT to write papers, many reach for AI detectors that perform slightly better than a random classifier — Many painful false accusations are the result. I wrote about why people should not rely on this imperfect tech: — https://arst…
-
@JMarkOckerbloom@mastodon …
John Mark Ockerbloom
on mastodon
A reminder that, just as you shouldn't rely on “AI” to write accurate prose, you also shouldn't rely on automated “AI detectors” to accurately flag cheating. Most of them, like the large language models they're trying to ID, are based on statistical pattern analysis rather than …
-
@chrismoranuk
Chris Moran
on x
1) If you're interested in LLMs but want to understand more, this is really useful. It's also important to hear how inaccurate these ‘detectors’ are because I still encounter people who think you can paste text into ChatGPT and ask it for an opinion... https://arstechnica.com/...
-
@mmitchell_ai
@mmitchell_ai
on x
For this article from @benjedwards, I got to spend some time explaining the fundamental “AI” concept of “perplexity”. =) https://arstechnica.com/...
-
@j_bertolotti
Jacopo Bertolotti
on x
Reminder that you can NEVER use a neural network to spot whether a text was written by a neural network. Corollary: You can never accuse anyone of cheating based on what a neural network said. https://arstechnica.com/...
-
@_alialkhatib
@_alialkhatib
on x
shouldn't be surprising, but we're in an era of bad reporting on AI, so i'm taking wins where i can: this article on detecting AI-generated text (and why it's practically impossible) investigated/reported by @benjedwards is really good https://arstechnica.com/... just a few riffs…
-
@benjedwards
Benj Edwards
on x
As educators panic about students using ChatGPT to write papers, many reach for AI detectors that perform slightly better than random chance Many painful false accusations are the result. I wrote about why people should not rely on this imperfect tech: https://arstechnica.com/...
-
@tobywalsh
Toby Walsh
on x
If you feed America's most important legal document—the US Constitution—into a tool designed to detect text written by AI models like ChatGPT, it will tell you that the document was almost certainly written by AI. https://arstechnica.com/...
-
r/HighStrangeness
r
on reddit
Simulation Universe?: AI detectors think the US Constitution was written by AI
-
r/offbeat
r
on reddit
Why AI detectors think the US Constitution was written by AI