A summary of the draft paper co-authored by Timnit Gebru, which outlined the main risks of large language AI models and provided suggestions for future research

The company's star ethics researcher highlighted the risks of large language models, which are key to Google's business. — hide

MIT Technology Review 2020-12-06 Karen Hao

Context & Ripple Effects

This summary lands at the flashpoint of Google's AI ethics crisis: Timnit Gebru co-authored a draft arguing that large language models — the technology central to Google's search and assistant businesses — carry environmental, bias, and misinformation risks, and the company's handling of the paper precipitated her exit. Within months, Google's Ethical AI team had fallen apart following her departure and then that of Margaret Mitchell.

The paper's warnings aged into the field's mainstream agenda: two years later, MIT Technology Review was cataloguing the privacy risks of GPT-3, LaMDA, and OPT-175B in terms the draft anticipated, and over 500 researchers had rallied behind an open-source large language model explicitly designed for research independent of any company.

First-order effects

Google is placed in the position of a company whose own ethics researchers are documenting risks in the technology that drives its core business, forcing a choice between engaging the critique or managing it as a personnel matter.
The draft's publication gives outside researchers, journalists, and policymakers a concrete, citable catalogue of LLM risks — training data toxicity, environmental cost, and the unreliability of model outputs — rather than abstract concerns.

Second-order effects

The dispute triggers a wave of departures that guts Google's internal ethics capacity, pushing scrutiny of Google's models outward to academics, journalists, and regulators who no longer rely on the company's own review process.
Independent research infrastructure emerges as the workaround: the 500-plus-researcher open-source LLM effort exists precisely because corporate gatekeeping over model access makes in-company ethics review structurally compromised.

Third-order effects

If the pattern holds, LLM risk assessment migrates from corporate ethics boards to independent and open-source channels, and companies building frontier models face a standing legitimacy gap between their commercial stakes and their self-policing.
The specific risk categories the draft named — privacy in training data chief among them — become the basis for the field's later audit and documentation norms, shaping how regulators and the press evaluate every subsequent large model release.

The trend: As large language models become commercially central, the research examining their risks is being forced out of corporate labs and into independent, open-source, and journalistic channels.

Discussion

Vox Shirin Ghaffary on x
The controversy behind a star Google AI researcher's departure
@_karenhao Karen Hao on x
If you're wondering what Google was so terrified of @timnitGebru publishing, I got a copy of the paper. Here's my best summary of what it says. Thank you to @emilymbender who coauthored the work for sending it my way. https://www.technologyreview.com/ ...
@amywebb Amy Webb on x
I urge you to read this important, nuanced, smart piece in @techreview by @_KarenHao about what went down last week with @timnitGebru. The researchers' concerns are exactly what I wrote about in my book on AI's problems The Big Nine. Short thread... https://www.technologyreview.c…
@histoftech Mar Hicks on x
tbh this sounds like a brilliant paper and I hope we get the chance to see the whole thing soon https://twitter.com/...
@muna_mire @muna_mire on x
VERY troubling that Google's AI Ethics co-lead, Timnit Gebru, known for her work on AI and race (an interview guest when I was at @robinthede's show) has been forced out for raising the alarm on large language models! Read a summary of that research here: https://www.technologyre…
@bostonjoan Joan Donovan, PhD on x
Research should be an independent endeavor, but sadly industry has so many critical resources. We need independent and publicly funded research about the internet, AI, and computing now! https://twitter.com/...
@browserdotsys Bowser on x
imagine getting fired over a mishmash of hot topics that have been circulating for the past few years https://twitter.com/...
@jackclarksf Jack Clark on x
An analogy of this situation - imagine if particle physics was solely domain of private sector and @timnitGebru had a critique about some societal effect of running atom smashers - how would we feel if the company suppressed that research? Bad, I think!
@miles_brundage Miles Brundage on x
Hard to imagine a way that Google could have called more attention to this paper.
@nrrrdcore Julie on x
Don't think I've read a better explanation re: NLP's inability to effectively detect nuance and/or keep up with constantly evolving language norms—the picture these models paint is wildly distorted, but profitable enough not to warrant scrutiny apparently https://twitter.com/...
@cattsmall Catt Small on x
“We are working at a scale where the people building the things can't actually get their arms around the data, and because the upsides are so obvious, it's particularly important to step back and ask ourselves, what are the possible downsides?” https://twitter.com/...
@rosejackson3 @rosejackson3 on x
We have to start talking about the subtle-& not so subtle-hidden ways the tech industry makes it hard, then wounding, then impossible for women & people of color to succeed. @timnitGebru is but one GLARING example. But anyone in tech knows someone or has experienced it themselves…
@jackclarksf Jack Clark on x
I find it notable that the paper was about these models that have been developed in private industry (GPT3, BERT, etc). But what the paper is pointing at is the need for a far greater amount of study applied to these systems. That requires massive engineering resources.
@juliaxgulia Julia Kaganskiy on x
Perhaps the most surprising thing about the research paper Google suppressed and fired Gebru over is how well established its contents appear to be. AI's carbon footprint, tendency towards bias, and consolidation of power in the hands of major corps is not exactly news, is it? ht…
@omanreagan Michael on x
Corporations can't be in charge of and be gatekeeper to the research that examines the impact of what they do. This is proof of that at Google. Read it. https://www.technologyreview.com/ ...
@jtasioulas John Tasioulas on x
“Many of the top experts in AI ethics work at large tech companies because that is where the money is.” Interesting article. https://www.technologyreview.com/ ...
@azeem Azeem Azhar on x
the inconvenient truth seems to be that @timnitGebru & others highlighted real problems of prejudice and bias in the large scale language models Google is rolling out for its key search product. Cc @JeffDean / Cracking work from @_KarenHao cc @emilymbender https://twitter.com/...
@rachelcoldicutt Rachel Coldicutt on x
This is vital research for everyone deploying services based on large-scale language models - not just Google but governments, charities, businesses. Efficiencies aren't efficient if they deepen structural bias. Thanks @timnitGebru for doing, and standing by, this important work.…
@catherine_mayer Catherine Mayer on x
The best explainer yet on why Google forced out the amazing @timnitGebru - and how important her work is https://twitter.com/...
@ngkabra Navin Kabra on x
“Stochastic Parrots” is a great name for all machine learning (as opposed to artificial intelligence). Apparently, the paper that @TimnitGebru was fired over has a title that starts with “On the Dangers of Stochastic Parrots: ...” https://venturebeat.com/...
@eileenguo Eileen Guo on x
“We are working at a scale where the people building the things can't actually get their arms around the data,” says coauthor @emilymbender. https://twitter.com/...
@lisawshapiro Lisa Wood Shapiro on x
from @_KarenHao “Gebru's draft paper points out that the sheer resources required to build and sustain such large AI models means they tend to benefit wealthy organizations, while climate change hits marginalized communities hardest.” https://www.technologyreview.com/ ...
@superwuster Tim Wu on x
On Google firing Timnit Gebru — the whole idea of a vague “internal review” that Google researchers need pass crazy to me (if for anything but trade secrets/ IP) Risk of censoring research seems high. I'm trying imagine if Columbia U had that... https://www.technologyreview.com/ …
@techreview @techreview on x
We got a copy of the research paper that forced Timnit Gebru out of Google. In it, the company's star ethics researcher highlighted the risks of large language models, which are key to Google's business. https://www.technologyreview.com/ ...
@daiwaka Daisuke Wakabayashi on x
36 hours after @timnitGebru went public with her departure from Google, here is the company's first (carefully-lawyered) public comment. You can see from the replies that it's not being received well. https://twitter.com/...
@lexialex @lexialex on x
This is a powerful white man gaslighting a Black woman. Not cool. She said she was fired. I believe her. https://twitter.com/...
@kharijohnson Khari Johnson on x
“In summary, we advocate for an approach to research that centers the people who stand to be affected by the resulting technology, with a broad view on the possible ways that technology can affect people,” the paper reads. https://venturebeat.com/...
@marilink Mariln Gonzalo on x
“The research paper surrounding Gebru's exit questions the wisdom of building large language models and examines who benefits from them, and who is impacted by negative consequences of their deployment” https://venturebeat.com/...
@catherineadenle Catherine Adenle on x
Seriously?! The day Google was found to have illegally spied on and terminated activists was the day they fired their top #AI #ethics expert who is a black woman. Poor judgement/PR. #diversitynotdivision #DiversityMatters #bias #WIT #WomenWhoCode #inclusion https://twitter.com/..…
@grady_booch Grady Booch on x
I look forward to someday reading @timnitGebru's research paper. https://twitter.com/...
@biosbenk David Benkeser on x
This whole saga is hugely disappointing, but not unexpected. Tech has opened up a lot of great research opportunities for quantitative researchers. One thing those jobs lack that's critical for research in sensitive fields? Tenure. https://twitter.com/...
@nitashatiku Nitasha Tiku on x
There's much to say & learn still about Google firing a pioneer like @timnitGebru, who gave its AI ethics credibility you just can't replicate. But it's worth focusing for a min on the fact that the email she sent was to an ERG for women and allies in AI https://www.washingtonpos…
@kharijohnson Khari Johnson on x
“If a large language model... can manipulate linguistic form well enough to cheat its way through tests meant to require language understanding, have we learned anything of value about how to build machine language understanding or have we been led down the garden path?” https://…
@shanselman Scott Hanselman on x
Read and learn about @timnitGebru and the groundbreaking work she did in AI Ethics. It's unreal that she was fired in the middle of a disagreement about a paper. She asks hard questions and uses AI to shine a light on, and life up others. She should be supported. https://twitter.…
@math_rachel Rachel Thomas on x
I have long admired @timnitGebru for her brilliance, moral courage, clear voice in speaking up for what is right, & influential scholarship. It is truly terrible that Google would do this. In this thread, I want to share some of Timnit's work I love https://twitter.com/...
@sifill_ldf Sherrilyn Ifill on x
This is absolutely infuriating. I am a huge fan and supporter of the work and uncompromising voice of @timnitGebru. I have learned so much from her about AI bias. What a disaster. https://twitter.com/...
@kateconger O...K on x
“This shows how some large tech companies only support ethics and fairness and other A.I.-for-social-good-causes as long as their positive PR impact outweighs the extra scrutiny they bring.” https://www.nytimes.com/...
@ram_ssk Ram Shankar on x
So @GoogleAI policy says “Intellectual freedom is an essential ingredient in research progress” Bone chilling reporting @dinabass on @timnitGebru firing shows this policy, is but pablum All the perfumes of Arabia will not sweeten @JeffDean hands https://www.bloomberg.com/...
@itsafronomics Anna Gifty on x
Let me tell how bad this is: Dr. @timnitGebru is probably one of THE MOST prominent Black women in AI and in SilIcon Valley. The same Silicon Valley where Black women make up less than 0.5% of technology leadership and workforce. This is worse than bad. https://www.bloomberg.com/…
@juliasilge Julia Silge on x
Definitely interested 👀 in the upcoming research paper on ethical issues surrounding large language models that apparently precipitated all this: https://www.bloomberg.com/...
@josheidelson Josh Eidelson on x
“She has also been an outspoken critic of the lack of diversity & unequal treatment of Black workers at tech companies, particularly...Google, & said she believed her dismissal was meant to send a message to the rest of Google's employees not to speak up” https://www.bloomberg.co…
@ibjiyongi @ibjiyongi on x
Google fired their head of ethical AI for trying to be ethical https://twitter.com/...
@hughlangley Hugh Langley on x
Last night, @timnitGebru, co-lead of Google's Ethical AI team, said she was fired over an email she sent to one of Google's research groups. The email was related to an as yet unpublished research paper on the potential harms of language models https://www.businessinsider.com/ ..…
@juliaxgulia Julia Kaganskiy on x
Once again tech companies demonstrate that their AI & Ethics programs exist primarily as a PR front so they can avoid actually doing the work of developing ethical standards and practices. https://twitter.com/...
@histoftech Mar Hicks on x
In 2019 Google dissolved its AI ethics board rather than remove a xenophobic & transphobic member. Now they've fired an AI expert who helped make their products less racist & sexist, seemingly for doing her job well. This corporation keeps telling us who they are. Believe them. h…
@histoftech Mar Hicks on x
Timnit Gebru, technical co-lead of Google's ethical AI team, & leader in the field of AI ethics, was just fired by google for trying to do her job. “She is widely respected for her work examining bias in AI & is a rare senior Black woman in the field.” https://www.businessinsider…
@histoftech Mar Hicks on x
The company that dissolved its ethics board for being unethical has now fired a leading researcher they employed to ensure their products were not racist and sexist🙃🙃🙃 This corporation keeps telling us who they are. Believe them. https://twitter.com/...
@joewestby Joe Westby on x
Wow it was already bad that Google fired Timnit Gebru.... its even worse they literally fired her for protesting about *silencing marginalized voices*. WTAF https://twitter.com/...
@ekp Ellen K. Pao on x
Email from ethics and AI leader @timnitGebru details problems at Google: zero accountability, silencing, gaslighting, and dehumanization. Instead of learning from it, Google fired her for it https://twitter.com/...
@libshipwreck Librarianshipwreck on x
Nothing says Google is genuinely committed to ethics in AI quite like Google firing the co-lead of its Ethical Artificial Intelligence Team... https://twitter.com/...
@tri_becca90 Rebecca Rivers on x
This wasn't a resignation, @Google fired @timnitGebru in an extremely cruel way. This is Google's new motto. “And remember... don't worry about being evil, and if you see something that you think isn't right - shut up or we will torch your career!” https://twitter.com/...
@jeffdean Jeff Dean on x
I understand the concern over Timnit's resignation from Google. She's done a great deal to move the field forward with her research. I wanted to share the email I sent to Google Research and some thoughts on our research process. https://docs.google.com/...
@rossformaine Ross LaJeunesse on x
google used to tell employees to “act like owners” - i.e., speak up, ask tough questions, fix problems and challenge decisions - it's a key reason google's been so successful. now, speaking up gets you fired. #ISupportTimnit https://www.nytimes.com/...
@_karenhao Karen Hao on x
@MichaelSmithLL @mer__edith @timnitGebru @emilymbender @JeffDean Hi Michael, here's a good account from @tsimonite at @WIRED. Timnit tried to work with Google on a solution but management refused. https://www.wired.com/... https://twitter.com/...
@aprilaser April Glaser on x
One former employee who the NLRB ruled this week was unlawfully fired after labor organizing at Google, @laurenceb, had incisive comments in response to Dr. Gebru's firing https://www.nbcnews.com/... https://twitter.com/...
@dseetharaman Deepa Seetharaman on x
This paper sounds important & fascinating, especially the part about the near-impossibility of auditing these systems for bias because of their size. “A methodology that relies on datasets too large to document is therefore inherently risky.” https://twitter.com/...
@frankpasquale Frank Pasquale on x
“Though most AI researchers acknowledge that large language models don't actually understand language and are merely excellent at manipulating it, Big Tech can make money from models that manipulate language more accurately, so it keeps investing in them.” https://twitter.com/...
@bdsams Brad Sams on x
I genuinely wonder (and worry) about where Google is headed. https://twitter.com/...
@celiahodent Celia Hodent on x
All seems to be going well in terms of #inclusion at Google, as well as #ethics regarding their #AI work... What kind of a surreal move is this? https://twitter.com/...
@k8em0 Katie Moussouris on x
When people think it's “easy once you've made a name for yourself” they are thinking mainly of white men. The higher non-wm climb, the harder to fall, & the more it hurts because after what you've accomplished, you'd think you'd be safe. Racism & sexism ensure you're never safe. …
@safiyanoble Safiya Umoja Noble PhD on x
This. The Black, LGBTG, and women of color who do this work are my heroes. We put our careers on the line to take on these giant, powerful men snd companies. We are denied promotions we've earned, we are unsupported as grad students, or maligned by our peers. And still we rise. h…
@jenstirrup @jenstirrup on x
I put my career on the line by handing back the #MVPBuzz Microsoft MVP Award after harassment and bullying. It was a scary time but I could not continue being diminished anymore by takers who stood silent about my treatment. I feel for @timnitGebru today. https://twitter.com/...
@juliacarriew Julia Carrie Wong on x
It's really weird to keep insisting that someone resigned when she says she didn't. https://twitter.com/...
@foiafeed Foia on x
BuzzFeed News: One of the half-dozen Amazon employees fired following worker protests during the height of the coronavirus pandemic will get a hearing before the National Labor Relations Board. https://www.buzzfeednews.com/ ... https://twitter.com/...
@blackamazon @blackamazon on x
This email doesn't reference any of the research it believes is more efficacious versus Timnit's . It provides no counter to Timnit's timeline of events besides implying she was nit supposed to ask despite being head of AI The fact you sent this email doesn't answer concerns http…
@zittrain Jonathan Zittrain on x
Timnit Gebru's work on AI bias is necessary and pathbreaking. The prospect that she'd simply be instructed by her employer to withdraw her academic work-in-progress — with no discussion — implicates all the church/state issues of corporations hosting academic research units. http…
@digitallybones Sahil Mohan Gupta on x
Apple is cashing in, yet again https://twitter.com/...
@novallswift Novall Swift on x
@DigitallyBones This is a personal social account. I don't speak on behalf of Apple. I'm an engineer, not a recruiter. It comes from a good place.
@novallswift Novall Swift on x
Hi all. Apple has growing efforts in AI Ethics and we care about doing what's right. I've made personal connections with folks who are hiring in AI/ML and I would love to speak with anyone who worked with Dr. Gebru who might be looking for a change. My DMs are open.
@ainowinstitute @ainowinstitute on x
We stand with Dr. Timnit Gebru, and are grateful for her brilliant and tireless work. If you want to show your support, Dr. Gebru's colleagues have drafted this petition: https://googlewalkout.medium.com/ ... #IStandWithTimnit
@4sweb @4sweb on x
We stand with Dr Timnit Gebru, who was fired from her position at Google, after she co-authored a research paper that raised concerns about Google's AI/language models. This matter is a question of academic integrity, censorship, and Google's commitments to DEI. https://twitter.c…
@maddiestone Maddie Stone on x
Google is better with @timnitGebru. Googlers, please consider signing this letter in support of Timnit. It matters. https://googlewalkout.medium.com/ ...
@mmitchell_ai MMitchell on x
Today dawns a new horrible life-changing loss in a year of horrible life-changing losses. I can't well articulate the pain of losing @timnitGebru as my co-lead. I've been able to excel because of her—like so many others. I'm mostly in shock. What should I be doing right now?
@aprilaser April Glaser on x
Less than 1 percent of Google's tech employees are Black women https://www.nbcnews.com/...
@oliviasolon Olivia Solon on x
The research paper that Google said @timnitGebru had to retract looked ethical problems with large-scale machine learning models including their environmental impact and potential to encode biases. Google uses one such model to power its search engine. https://www.nbcnews.com/...
@oliviasolon Olivia Solon on x
The #IStandWithTimnit campaign is gathering momentum inside Google - more than 1,200 Googlers have signed the statement of solidarity within 24 hours https://www.nbcnews.com/...
@brookeobie @brookeobie on x
Corporate heads love your courage until you use it with them. https://twitter.com/...
@ericajoy Erica Joy on x
💖💖💖 for the support displayed here. but dear god i wish it wasn't required so frequently. https://twitter.com/...
@caseynewton Casey Newton on x
Was talking to a manager yesterday about the Timnit Gebru situation and they said that if an employee threatens to resign in writing, employers can legally treat that as a resignation. Anyone else hear this / have experience here? Feel free to DM
@timnitgebru Timnit Gebru on x
You know these last 2 weeks were supposed to be my vacations and we're currently on a road trip. Trying to find my email with my conditions that were so unacceptable to them, but one of them is basically point #1 here: https://googlewalkout.medium.com/ ...
@vcastillo630 Vivianne Castillo on x
What has happened to @timnitGebru is infuriating & triggering. Her firing shows me that no matter how world-renowned, respected & brilliant you are in tech, it'll never be enough to protect you from White supremacy's belief that Black folks are to be submissive. #IStandWithTimnit…
@bobbyallyn Bobby Allyn on x
200+ Google employees now demanding explanation from executives about why AI scientist @timnitGebru was fired over research paper into the ethics and environment impact of Google's AI systems https://www.npr.org/...
@drewharwell Drew Harwell on x
@nitashatiku @timnitGebru Hard to overstate Gebru's influence - both for the AI field and for researchers of color who saw her as a Silicon Valley inspiration. Her research (like https://gendershades.org/) shaped global understanding of algorithmic bias and facial recognition's r…
@timnitgebru Timnit Gebru on x
EXACTLY. I was just saying. It is called brain women and ALLIES. And the allies were like hell naw get out of here. And I was told so many people were responding to it and I didn't even know cause I didn't have access by then. Its crazy. https://twitter.com/...

Chronicles