Anthropic releases prompt caching, which lets developers cache frequently used context between API calls, in public beta on its API
Anthropic introduced prompt caching on its API, which remembers the context between API calls and allows developers to avoid repeating prompts.
VentureBeatEmilia David
Context & Ripple Effects
Anthropic had already added tools for developers to generate, test, and evaluate prompts; prompt testing and evaluation tools addressed how applications shape inputs, while caching addresses how they reuse them across requests.
The release makes context handling a first-class API concern. Later API additions, including extended prompt caching alongside agent-building tools, show how reusable context became part of a broader developer platform.
First-order effects
Developers can retain frequently reused context between API calls rather than resending the same prompt material each time.
Anthropic’s API becomes better suited to applications with stable instructions or recurring reference context, with caching available in public beta.
Second-order effects
Application teams can separate durable shared context from request-specific inputs, making prompt architecture and cache management part of implementation work.
Competing model APIs face pressure to offer comparable mechanisms for persistent or reusable context as developers weigh operational overhead alongside model capability.
Third-order effects
If widely adopted, reusable context will shift LLM APIs from stateless request endpoints toward context-aware platforms, making context engineering a durable product and infrastructure discipline.
The boundary between performance-oriented caching and longer-lived agent memory may become a key design choice, especially as providers add more tools for stateful agent workflows.
The trend: LLM providers are turning context management from an application-side workaround into a core API layer for building more capable, repeatable agentic software.
This is huge. You can now dump tons of data into the context window, with fast speeds and minimal cost. Examples: - show the LLM your entire codebase and ask for new features - instead of just RAGging in the top 5 docs, give the LLM the top 1000 - show hundreds of examples
🆕 Prompt caching with Claude. Caching lets you instantly fine-tune model responses with longer and more instructive prompts—all while reducing costs by up to 90%. Available in beta on the Anthropic API today. https://www.anthropic.com/...
Anthropic added context caching with significant latency reductions & up to 90% cheaper tokens input cost Claude 3 Haiku, Opus and 3.5 Sonnet high perf koding at low cost for all 😌 [image]
This is such a big deal. Imagine caching entire books worth of content, any coding documentation you want, and retrieving it with multiple API calls. Imagine the sophisticated agents you can now build with this.
Caching is one of the most underused and valuable ways to decrease latency and cost, and increase consistency of model responses. Use caching if you're building with agents or production-level use cases.
This is effectively the most important feature possible outside of just making models better. Cheaper prompts (10x) and faster (30-80%) first token on cache hits, incredible when doing few shot prompting with images or text
@simonw These non-standard, fine-grained pricing models are already starting to give off that awful AWS pricing vibe. This makes me somewhat reluctant to use the feature.
Anthropic (@AnthropicAI) just announced a game-changer for their API: Prompt caching. Think of prompt caching like this: You're at a coffee shop. The first time you visit, you need to tell the barista your whole order. But next time? Just say “the usual.” That's prompt [image]
Prompt caching is clearly very exciting but isn't as much of a silver bullet as meets the eye. My first thought was that one would be able to effectively “compile” a massive prompt with lots of embedding context, and get massively improved performance over traditional RAG, at a
Looks similar to Gemini's context caching, but the Anthropic pricing model is different Gemini charge $4.50/million tokens/hour to keep the context cache warm Anthropic charge for cache writes, and “cache has a 5-minute lifetime, refreshed each time the cached content is used”
@AnthropicAI People have enough trouble understanding the difference between fine-tuning and prompting already, it really doesn't help if you misuse the term “fine-tune” in a tweet like this one!
Very interesting to see differences in feature announcement: - GDM: corpo blog - OAI: weird rumors for a week - Anthropic: dude posts docs and it's out
Prompt caching is akin to instant fine-tuning, unlocking application possibilities not possible before due to constraints on cost and latency. I'm stoked to see what you all can build now with this supercharged way of accessing Claude.