Researchers extract business and personal email addresses for 30+ NYT staff by fine tuning GPT-3.5 Turbo, bypassing the safeguards for privacy-related queries
This is a visualization of a large email database from the Enron Corporation, which is often used to train artificial intelligence systems, like ChatGPT. Threads: @carnage4life and @vishvanands . Mastodon: @jeffjarvis@mastodon.social . X: @cbarrett . Forums: Slashdot Threads: Dare Obasanjo / @carnage4life : LLMs are just fancy autocomplete and are trained on data on the public internet. This means they can be asked to autocomplete people's personal information that is widely available on the public internet. Indiana University researchers gave GPT-3.5 Turbo a list of verified names and email addresses of New York Times employees, which caused the model to return similar results it recalled from its training data. 80% of the work addresses returned were correct. This data is also in Google search🤷🏾♂️ Vishvanand Subramanian / @vishvanands : NYT claims private information can be leaked during fine tuning of GPT 3.5. Initially, I was alarmed that researchers were able to bypass basic safeguards in LLMs against revealing private information in the training data. But perhaps the model was just guessing firstname.lastname corporate emails of popular journalists and happened to get a few right.. Mastodon: Jeff Jarvis / @jeffjarvis@mastodon.social : But the third paragraph says: “My email address is not a secret”... How Strangers Got My Email Address From ChatGPT https://www.nytimes.com/... X: Colin Barrett / @cbarrett : The idea that LLMs are anything other than data laundering machines, and not very good ones at that, is a fantasy Forums: Slashdot : ChatGPT Exploit Finds 24 Email Addresses, Amid Warnings of ‘AI Silo’
Context & Ripple Effects
The reported extraction extends an existing concern that large models can reveal training-data-like material: a previous divergence attack against ChatGPT induced copied sequences through prompting. Here, fine-tuning is reported to bypass privacy-query safeguards rather than merely test their limits.
Earlier coverage had already framed personal-data exposure as a risk of models trained on vast data troves, including a survey of privacy concerns around large language models. The NYT employee results make that abstract boundary concrete for organizations whose staff identities and contact details are publicly distributed.
First-order effects
- OpenAI's privacy safeguards appear less reliable once a GPT-3.5 Turbo model is fine-tuned, creating a direct exposure path for email addresses associated with named NYT staff.
- The New York Times and affected employees face a more actionable phishing, harassment, and impersonation risk because the reported outputs include both business and personal contact details.
Second-order effects
- Model providers will face pressure to test fine-tuned variants—not only base-chat behavior—for memorization and personal-data extraction, and to decide whether some fine-tuning access needs tighter controls.
- Organizations with public staff directories or broadly available contact traces may need to treat model-assisted aggregation as a distinct risk from ordinary web search, revisiting how easily personal and work identities can be connected.
Third-order effects
- If fine-tuning repeatedly defeats output-layer safeguards, privacy protection will shift toward training-data governance, memorization evaluation, and access design rather than relying chiefly on refusal behavior.
- The case reinforces the public-data permission boundary: information that is individually obtainable can become materially more harmful when a model makes it easy to retrieve and combine at scale.
The trend: Generative-AI privacy risk is moving from whether models contain public personal data to whether product controls can prevent that data from being efficiently extracted and recombined.