How OpenAI is using ChatGPT prompt engineering to get around fundamental biases in DALL-E 3, discourage it from recreating the work of living artists, and more
Last year I wrote about my initial experiments with DALL-E 2, OpenAI's image generation model. Mastodon: @williamgunn@mastodon.social , @waxy@xoxo.zone , and @simon@fedi.simonwillison.net X: @joedevon , @katebevan , @bjh_ip , and @simonw . Forums: Hacker News Mastodon: William Gunn / @williamgunn@mastodon.social : @Techmeme Crediting the artist would likely work better in practice, rather than relying on easily-broken RLHF handcuffs, which can only be applied for a limited set of artists anyways. @simon @waxy@xoxo.zone : Now add a walrus: Prompt engineering in DALL-E 3: fascinating post by Simon Willison with details about how ChatGPT was instructed to use DALL-E 3 https://simonwillison.net/... Simon Willison / @simon@fedi.simonwillison.net : I wrote some notes about DALL-E 3, including reverse engineering some aspects of how it works. It's a fascinating insight into the prompt engineering that happens inside of OpenAI — **Now add a walrus: Prompt engineering in DALL-E 3** — https://simonwillison.net/... X: Joe Devon / @joedevon : If you're playing around with DALL-E 3, this post is a must read. Really excited by several things here like using seeds to keep visual consistency, which is so hard to do in Midjourney. Simon mentions the leaked prompts OpenAI uses as well. Some of it is so annoying. I tried... Kate Bevan / @katebevan : this is a really cool tinker with Dall-e 3 and a poke in the guts of it that really shows how prompt engineering is both technical and also deeply creative. Ben Hoyle / @bjh_ip : Great distillation of DALL-E 3 functionality via some reverse engineering and experimentation. I like the bits in the prompt where they address current copyright law & add diversity. Cool mixture of prompt engineering + thinking about wider societal impacts (it's not all bad!). Simon Willison / @simonw : I wrote some notes about DALL-E 3, including reverse engineering some aspects of how it works. It's a fascinating insight into the prompt engineering that happens inside of OpenAI Now add a walrus: Prompt engineering in DALL-E 3 https://simonwillison.net/... Forums: Hacker News : Now add a walrus: Prompt engineering in DALL-E 3
Context & Ripple Effects
OpenAI had already paired DALL-E 2's rollout with a content policy aimed at sensitive and biased imagery, as usage expanded through its early access rollout and content-policy controls. This analysis adds a more operational detail: a language model can sit between the user and image model, rewriting or constraining requests before generation.
The reverse-engineering focus matters because it makes the policy layer—not just the image model's training and weights—a visible part of how OpenAI manages bias and living-artist imitation. Later evidence that prompting can induce training-data copying in ChatGPT underscores why such controls need to be tested as interfaces, not assumed to be fixed safeguards.
First-order effects
- DALL-E 3 users receive outputs shaped by ChatGPT-mediated prompt rewriting, which can steer requests away from known bias patterns and requests to emulate living artists.
- OpenAI can adjust behavior at the prompt-orchestration layer without necessarily changing the underlying image model, making the user-facing policy experience more dependent on that intermediary.
Second-order effects
- Artists and users seeking highly specific stylistic control may encounter a less literal mapping from their request to the generated image, while competing image tools face pressure to explain how they handle similar requests.
- Because the rules are expressed through prompts, researchers and users can inspect and probe an enforcement surface that may be more legible—but also more susceptible to adversarial prompting, as illustrated by the later training-data extraction prompt attack.
Third-order effects
- If multimodal products increasingly rely on model-mediated prompt rewriting, AI safety and IP governance will move toward layered systems: underlying model behavior plus a separately maintained policy and interaction layer.
- That structure can make controls faster to revise, but it also creates an ongoing audit burden: providers will need to demonstrate that policy behavior holds across varied prompts rather than treating a refusal as a permanent model property.
The trend: Generative-AI providers are shifting from single-model guardrails toward layered, continuously tuned orchestration systems that govern what users can ask for and receive.