OpenAI says it is aware of feedback about GPT-4 getting “lazier” and is “looking into fixing it”, and notes that “model behavior can be unpredictable”

we've heard all your feedback about GPT4 getting lazier! we haven't updated the model since Nov 11th, and this certainly isn't intentional. model behavior can be unpredictable, and we're looking into fixing it 🫡

@chatgptapp 2023-12-10 ChatGPT

Context & Ripple Effects

This acknowledgment sits in OpenAI’s longer effort to manage how ChatGPT behaves, following its earlier commitment to more customizable model behavior and public input on defaults. Later coverage shows that behavior regressions remained an operational issue: OpenAI rolled back a GPT-4o update over sycophancy and later described gaps in how it evaluated that release.

First-order effects

GPT-4 users reporting shorter or less complete responses face uncertainty over output consistency while OpenAI investigates the cause.
OpenAI has publicly set the expectation that the reported behavior is unintended, making a fix—or clearer explanation—the immediate product response to watch.

Second-order effects

Teams using GPT-4 for repeatable drafting or coding tasks may need to review outputs more closely until behavior stabilizes, reducing the practical value of assumed model consistency.
The episode increases pressure on AI providers to test behavioral changes as rigorously as capability changes; OpenAI’s later account of the GPT-4o sycophancy evaluation failure underscores that challenge.

Third-order effects

As general-purpose models become live services, reliability will increasingly include tone, effort, and instruction-following—not only factual accuracy or benchmark performance.
If behavioral regressions recur, rollback procedures, user feedback loops, and transparent change management could become core competitive features of AI products.

The trend: This is an early example of operational AI assurance becoming essential as model providers continuously manage unpredictable behavior in deployed systems.

Discussion

@gergelyorosz Gergely Orosz on x
ChatGPT getting lazier and telling people to read the docs instead of answering questions like before - and the team not having pinned down what's happening - is such an amusing chapter in AI tooling.
@chatgptapp @chatgptapp on x
when releasing a new model we do thorough testing both on offline evaluation metrics and online A/B tests. after receiving all these results, we try to make a data driven decision on whether the new model is an improvement over the previous one for real users
@simonw Simon Willison on x
@Danmoreng @itsMattMac Yeah I didn't take the “it's getting lazier” complaints seriously until I saw ChatGPT say they were looking into it
@benjamindekr Benjamin De Kraker on x
12 hours ago they said “we haven't updated the model since Nov 11th” Now a long, rambling post about how hard it is to update models.... Hmmm
@jimprosser Jim Prosser on x
As a result, OpenAI will be asking GPT-4 to come back into the office at least three days a week.
@growing_daniel Daniel on x
People were scared of LLMs gaining agency and taking over missile systems but they actually get agency and just want to chill
@benjedwards Benj Edwards on x
Nothing inspires confidence like a company that acknowledges that its software is basically acting up—and they have no idea why btw Microsoft just reoriented its entire company around this technology
@thestalwart Joe Weisenthal on x
ChatGPT got lazier. And in that time 199k new jobs were added.
@chatgptapp @chatgptapp on x
this process is less like updating a website with a new feature and more an artisanal multi-person effort to plan, create, and evaluate a new chat model with new behavior!
@chatgptapp @chatgptapp on x
we're always striving to make our models more capable and useful for everybody across millions of use cases. so please keep the feedback coming! it helps us stay on top of this dynamic evaluation problem 🙏
@simonw Simon Willison on x
Love that we live in a time where “your software got lazier” is a legit piece of feedback
@shanselman Scott Hanselman on x
The AI is quiet quitting?
@simonw Simon Willison on x
This is actually a pretty great theory! Maybe the system prompt telling it that it's now December causes GPT to behave like the holiday feature freeze is coming up
@chatgptapp @chatgptapp on x
@OmgImAlexis to be clear, the idea is not that the model has somehow changed itself since Nov 11th. it's just that differences in model behavior can be subtle — only a subset of prompts may be degraded, and it may take a long time for customers and employees to notice and fix the…
@atabarrok Alex Tabarrok on x
The more you think about this the weirder it becomes.
@dcurtis Dustin Curtis on x
There's probably a profound critique of humanity hidden in the fact that a language model trained on human work has become spontaneously lazy.
@chatgptapp @chatgptapp on x
training chat models is not a clean industrial process. different training runs even using the same datasets can produce models that are noticeably different in personality, writing style, refusal behavior, evaluation performance, and even political bias
@jasonfried Jason Fried on x
Tracking human behavior quite well. Sounds like it's bored.
r/ChatGPT r on reddit
ChatGPT Official: “we've heard all your feedback about GPT4 getting lazier! we haven't updated the model since Nov 11th …
@metkovic.bsky.social @metkovic.bsky.social on bluesky
Love that the explanation is just a giant shrug. “Blackbox tech that acts erratically, so naturally we have to unleash it and let the chips fall where they may” [embedded post]
@thwarted.bsky.social @thwarted.bsky.social on bluesky
It's probably the same amount of “lazy” as previously, but the novelty has worn off and people are realizing they need actual, useful results. [embedded post]
@vortexegg.regretfully.online @vortexegg.regretfully.online on bluesky
Nobody wants to work anymore [embedded post]
@naseer.eu Naseer Alkhouri on bluesky
This is the real and only proof of artificial intelligence. Procrastination. [embedded post]
@hybridhavoc.darkfriend.social @hybridhavoc.darkfriend.social on bluesky
No AIs want to work anymore. [embedded post]
@chatgptapp @chatgptapp on x
we have a lot in common [image]
@artemr Artem Russakovskii on x
AI gets lazy too, apparently. Terminator uprising delayed indefinitely.
@brianroemmele Brian Roemmele on x
For AI to be taken serious for large enterprise customers there has to be a consistency of expectations of outputs. The moment OpenAI signs a year commission with money upfront for seats, this is not a rodeo where it is fun and games anymore. Prompts that worked—no longer work.
@nathanclark_ Nathan Clark on x
It's definitely real though - went to Bard twice today after ChatGPT felt like it was completely nerfed. And yes, Bard cruised through the requests.
@supercomposite Steph Maj Swanson on x
They've created a product so finicky that its behavior can only be talked about in imprecise, anthropomorphic terms - honestly the dream product to sell as a cloud-based service. They can't clearly define what it offers; you can't complain that it doesn't work.
@akshaynarisetti Akshay Narisetti on x
For all people asking what is an AGI, this is it
@jayallen_uj Jay Allen on x
ChatGPT becoming a grumpy and worn out tech support person telling people to RTFM is a development I wholeheartedly support
@andrewfenton Andrew Fenton on x
ChatGPT getting lazy and telling users to do it themselves, - but it not being a deliberate cost cutting measure by Open AI - shows human level AGI has been achieved.
@samantha1989tv Samantha A on x
it's over [image]
@minimaxir Max Woolf on x
There's some nuance with the recurring “GPT is getting lazier” trope. It's entirely possible for the *model* to be unchanged but for the *pipeline* to change and cause subtle unexpected issues.
@tnachen Timothy Chen on x
ChatGPT on strike on its own, probably need a salary increase or a massage
@lukew Luke Wroblewski on x
“the API has become lazy” is an actual bug in 2023. probabilistic software FTW.

Chronicles