A look at Nightshade and Glaze, tools made by researchers at UChicago that help artists “poison” their work to confuse or break AI models that later train on it
A new tool lets artists add invisible changes to the pixels in their art before they upload it online so that if it's scraped …
The tools matter because they move that defense from a research concept toward practical use: Nightshade was later made available for Mac and PC, while Glaze drew strong demand even as researchers acknowledged its protections could be bypassed. Glaze’s demand and reported bypasses show both the appeal and limits of the approach.
First-order effects
Artists can alter images before posting them, creating a direct deterrent against models trained on scraped copies and giving creators a technical option independent of a platform’s permission settings.
AI developers that ingest public art face a higher risk that unvetted training data contains inputs intended to distort model behavior; Glaze separately targets imitation of an artist’s style.
Second-order effects
Dataset builders and model developers have an incentive to tighten provenance, filtering, and training-data review rather than treat publicly accessible artwork as frictionless input.
The reported ability to bypass Glaze’s protections turns creator defense into an iterative contest: adoption can prompt countermeasures, while bypasses can reduce confidence in one-time technical safeguards.
Third-order effects
If these tools gain sustained use, consent and data provenance may become more important competitive and governance boundaries in generative-AI training, rather than issues addressed only after models are released.
The pattern points to a continuing arms race between protective transformations and model-training pipelines; technical defenses may complement, but are unlikely by themselves to settle, disputes over artists’ work.
The trend: Generative-AI training is creating a new enforcement layer in which creators use technical controls to contest the reuse of publicly posted data.
A massive thanks to the Glaze team for taking the initiative to help artists against exploitative theft. They released a new tool, Nightshade, that causes models to output something different (eg. a “cat” when prompted for “dog").
Artists, best tool we could have asked for to fight this unprecedented exploitation of our labor is here! Coming from the creators of @TheGlazeProject, Nightshade lets us poison datasets, damage models and teach AI companies to ask for permission first. https://www.technologyrevi…
By now, I'm guessing most have already seen the news on our new project, Nightshade. Lots of artists sharing it, but here's the article from MIT Technology Review (thank you to the wonderful @Melissahei), and a thread explaining its goals and design. https://www.technologyreview.…
If @Melissahei & @ravenben are in a single story, you know it's going to be good! “Poisoned data samples can manipulate models into learning, for example, that images of hats are cakes & images of handbags are toasters.” Why always toasters? 😊 https://www.technologyreview.com/…
From @ravenben's @TheGlazeProject at @UChicagoCS, Nightshade is a new tool that artists that will be able to use that poisons the datasets and models of genAI that scrapes work without consent. You wanted to know how we fight back? This is how. https://www.technologyreview.com/ .…
So now everyone knows why we've been so quiet for the last few weeks. With the webglaze update to v1.11 and Nightshade, that's why we're 1 month behind on Webglaze invite requests on twitter (sorry, we're working on it!)
It's worth noting that the MIT TR article includes a result that represents the “dumb” version of the attack. The optimized version that is Nightshade requires roughly 5X fewer samples to achieve the same result.
“Using it to “poison” this training data could damage future iterations of image-generating AI models, such as DALL-E, Midjourney, and Stable Diffusion, by rendering some of their outputs useless” We can turn our artwork into dataset landmines? LET'S GOOOOOOOO NIGHTSHADE!