Google says poor image tuning and Gemini becoming more cautious than intended made the model “overcompensate in some cases, and be over-conservative in others”

Google announced yesterday that it disabled image generation of people in Gemini following criticism about its historical accuracy.

9to5Google 2024-02-24 Abner Li

Context & Ripple Effects

Google had already acknowledged historical inaccuracies in Gemini image outputs and paused people-image generation pending an improved release. This explanation narrows the failure from a single accuracy complaint to an interaction between image tuning and overly cautious model behavior, echoing earlier reports that Gemini was reluctant to address some sensitive topics.

First-order effects

Gemini users lose access to people-image generation while Google revises the system, rather than receiving a narrow fix to individual prompts.
Google publicly identifies overcorrection and excessive caution as contributing factors, raising the bar for the promised re-release beyond restoring the disabled feature.

Second-order effects

The pause makes evaluation of image-generation safeguards more central to Gemini's product rollout: fixes must avoid both inaccurate depictions and blanket refusals or distorted outputs.
Competitors offering people-image tools face added pressure to demonstrate that their bias and safety tuning preserves historical and contextual accuracy.

Third-order effects

The episode points toward governed generation in which providers may need feature-level shutdowns and staged re-enablement when safety tuning changes core output quality.
If such trade-offs recur across models, differentiation will depend less on whether an image model has safeguards than on whether its controls can be audited and adjusted without broad capability loss.

The trend: Generative-AI providers are moving from broad safety policies toward more operational control of the quality trade-offs those policies create.

Discussion

@jacaranda7 Rob Leathern on threads
The correct take by @dwillner. Google execs are rushing to market, teams skipping steps/lacking support in 🔑 areas: “..much more nuance will be needed to both accurately depict history (with all of its warts) and to accurately depict the present (in which all CEOs are not white …
@goldman Jason Goldman on threads
Google ran this as a show your ass test.
@vishwanathsarang Vishwanath Sarang on threads
This is good comms work.
@crumbler Casey Newton on threads
I suspect this week's Gemini ‘racially diverse Nazi’ issues resulted more from bugs than policy. That said, it offers a good reason for chatbot makers to look at how their tendency to censor is undermining user trust: https://www.platformer.news/ ...
@daveleebbg Dave Lee on threads
The Google Gemini race row is precisely what the company feared when it held back its public AI products for so long. There's a lot of “I told you so"s happening over in Mountain View right now, I suspect.
@glinden.bsky.social Greg Linden on bluesky
These systems aren't going to be fixed by applying yet more bandaids. The underlying cause is that the models have no ability to determine what is true. They're inherently unreliable and can't be used in any application where users expect the output to be reliable, badly narrow…
@elonmusk Elon Musk on x
A senior exec at Google called and spoke to me for an hour last night. He assured me that they are taking immediate action to fix the racial and gender bias in Gemini. Time will tell.
@yishan @yishan on x
This event is significant because it is major demonstration of someone giving a LLM a set of instructions and the results being totally not at all what they predicted.
@yishan @yishan on x
Google's Gemini issue is not really about woke/DEI, and everyone who is obsessing over it has failed to notice the much, MUCH bigger problem that it represents. (1/n)
@benhylak Ben on x
haven't seen this sort of clarity in writing from google in a long time: https://blog.google/...
@willoremus Will Oremus on x
it is interesting how AI erasing or negatively stereotyping people of color was dismissed as a minor/fixable bug or nonissue by some of the same folks who see AI erasing white men as somewhere between a crime against humanity and the downfall of civilization
@googlepubpolicy @googlepubpolicy on x
Our AI image generation feature got it wrong. Here's more on what happened and how we're working to fix it. https://blog.google/...
@modestproposal1 @modestproposal1 on x
imagine if Google had used humans to tune its search engine when it was fighting for market share instead of building a machine to organize the worlds information and blowing away the competition
@tomcoates Tom Coates on x
This *genuinely isn't fucking hard* guys. To believe you wouldn't need to do this work is to argue that you don't believe that the vast trove of documents used to train these systems have *any* documents in them that reflect disproven and ignorant views of the period.
@illneil Neil Hartner on x
Interesting explanation that highlights that even well-intentioned inclusiveness can lead to unintended exclusions. If you train AI this way, it clearly becomes overly biased against caucasians. So fair to assume the same risk exists if we train humans the same way.
@grimezsz @grimezsz on x
I am retracting my statements about the gemini art disaster. It is in fact a masterpiece of performance art, even if unintentional. True gain-of-function art. Art as a virus: unthinking, unintentional and contagious. offensive to all, comforting to none. so totally...
@stevesi Steven Sinofsky on x
At this point Google owes the internet the release of the complete list of training materials. We have to see for ourselves what nonsense Gemini was fed.
@natesilver538 Nate Silver on x
Sorry, but this thread defies logic. If you program your LLM to add additional words ("diverse" or randomly chosen ethnicities, etc.) whenever you ask it to draw people, then *of course* it's going to behave this way. It is incredibly predictable, not some emergent proprerty.
@chatgptapp @chatgptapp on x
alright lay off my buddy gemini
@tomcoates Tom Coates on x
TL:DR - (a) LLMs are trained on human content. Humans can be bigoted and ignorant (b) Some humans will use LLMs to spread bigotry and ignorance (c) Therefore accurate and responsible LLMs will require corrective prompts (d) Some correctives will be poorly implemented. The end.
@natesilver538 Nate Silver on x
There are also many examples of it inserting strong political viewpoints even when not asked to draw people. Fundamentally, this *is* about Google's politics “getting in the way” of its LLM faithfully interpreting user queries. That's why it's a big deal. https://twitter.com/...
@carnage4life Dare Obasanjo on x
Google has confirmed that Gemini refusing to generate white people was due to it being tuned to always generate a range of races and them not accounting for the fact that sometimes generating one race of people is actually accurate. No explanation for why just white people tho🙃 […
@jlgolson Jordan Golson on x
To be insanely generous, basically Google tried to make it “diverse” the way normal people think of it and got insane Harvard DEI instead.
@elonmusk Elon Musk on x
The problem is not just Google Gemini, it's Google search too
@natesilver538 Nate Silver on x
Gemini is behaving exactly as instructed. Asking it to draw different groups of people (e.g. “Vikings” or “NHL players") is the base case, not an edge case. The questions are all about how it got greenlit by a $1.8T market cap company despite this incredibly predictable behavior.
@maxwinebach Max Weinbach on x
People will complain but this is a good response and an honest blog. Good job Google.
@daniel_271828 Daniel Eth on x
I want to add to this - a major reason corps have such blunt “safety measures” is their alignment techniques suck. Google doesn't *actually* want to prevent all images of Caucasian males, but their system can't differentiate between “don't be racist” and “be super over the top PC…
@stevesi Steven Sinofsky on x
Gemini: A Family of Highly Capable Multimodal Models // This is the Gemini technical paper from 12/23—one with 10 pages of contributors—worth looking at sections 7.3-7.4 where with hindsight we can begin to discern where the reinforcement went haywire. https://storage.googleapis.…
@ylecun Yann LeCun on x
Indeed, my remarks on a paper from Duke on GAN-based portrait super-resolution were met with an unusual level of vitriol, back in 2020. I merely pointed out the obvious fact that image reconstruction is heavily biased by the statistics of the training dataset. As it turns out, a.…
@lutherlowe Luther Lowe on x
If the Republicans in Congress this blog is designed to diffuse fallout from were merely alarmed by Gemini's absurd outputs, they'd be missing a much bigger problem: self-preferencing of the house LLM by G distorts the free market & would hurt open source-powered AI startups.
@yishan @yishan on x
@ESYudkowsky Again, I will say this: any time you see a comedic large-scale error by AI, it is evidence that we do not know how to align and control it, that we are not even close.
@daniel_271828 Daniel Eth on x
“Guys, there's nothing to worry about regarding alignment - the much-hyped AI system from the large tech company didn't act all bizarre due to a failure of alignment techniques, but instead because the company didn't really test it much for various failure modes” 🤔
@tomcoates Tom Coates on x
I get really fucked off with this kind of stupid position. Let's be clear. LLMs are self-evidently trained on documents and images *from the past*. Human beings in the past have been wildly sexist, racist and homophobic.
@kane @kane on x
incredible that despite inventing the original transformer in 2017, google has managed to step on rakes of its own making at every step in productization since then
@aravsrinivas Aravind Srinivas on x
4 years ago, @ylecun was harassed for saying bias in model outputs comes from bias in data. Google went too far in the opposite direction to the extent that they made these models factually inaccurate in Gemini. Good to see someone like Prabhakar take over the mantle finally. [im…
@fchollet François Chollet on x
Bias in ML systems can come from bias in the training data. But that's only one possible source of bias among many. Arguably, prompt engineering can be an even worse source of bias. Literally any part of your system can introduce biases. Even non-model parts, like your...
r/Bard r on reddit
“Gemini image generation got it wrong. We'll do better.”

Chronicles