A study finds that asking LLMs to be concise in their answers, particularly on ambiguous topics, can negatively affect factuality and worsen hallucinations
Turns out, telling an AI chatbot to be concise could make it hallucinate more than it otherwise would have.
Context & Ripple Effects
The corpus has tracked hallucination as a foundational limitation of chatbots since early warnings that they can generate plausible output without regard for truth, including the earlier warning about chatbots' hallucination problem. This study adds a concrete interface-level variable: brevity instructions can alter reliability rather than merely presentation.
It also reinforces the earlier critique that effective LLM use depends on non-obvious prompting choices, documented in analysis of why chatbot prompts are hard to get right. That matters as concise answers become a common product default and expectation.
First-order effects
- Teams deploying LLMs on ambiguous questions may need to treat “be concise” as a reliability-affecting instruction, not a harmless formatting preference.
- Users seeking short answers face a sharper trade-off: less visible qualification or reasoning can coincide with less factual output.
Second-order effects
- Model providers and application builders will have reason to test response-length settings alongside factuality, especially where interfaces default to terse summaries.
- Workflow designers may add options for fuller answers, uncertainty cues, or verification on ambiguous requests rather than optimizing solely for response speed and brevity.
Third-order effects
- If repeated across models and tasks, this shifts AI quality evaluation from a single accuracy score toward measuring how interface and prompt constraints change reliability.
- The finding supports a broader move away from one-size-fits-all chatbot behavior toward context-sensitive controls for AI embedded in work processes.
The trend: AI products are increasingly being judged on whether their interaction design preserves reliability, not just on the underlying model’s raw capabilities.