Discussion

• @mattshumer_ Matt Shumer on x

  This is huge. You can now dump tons of data into the context window, with fast speeds and minimal cost. Examples: - show the LLM your entire codebase and ask for new features - instead of just RAGging in the top 5 docs, give the LLM the top 1000 - show hundreds of examples

• @anthropicai @anthropicai on x

  🆕 Prompt caching with Claude. Caching lets you instantly fine-tune model responses with longer and more instructive prompts—all while reducing costs by up to 90%. Available in beta on the Anthropic API today. https://www.anthropic.com/...

• @legit_rumors @legit_rumors on x

  Anthropic added context caching with significant latency reductions & up to 90% cheaper tokens input cost Claude 3 Haiku, Opus and 3.5 Sonnet high perf koding at low cost for all 😌 [image]

• @skirano Pietro Schirano on x

  This is such a big deal. Imagine caching entire books worth of content, any coding documentation you want, and retrieving it with multiple API calls. Imagine the sophisticated agents you can now build with this.

• @matthewberman @matthewberman on x

  Caching is one of the most underused and valuable ways to decrease latency and cost, and increase consistency of model responses. Use caching if you're building with agents or production-level use cases.

• @abacaj Anton on x

  This is effectively the most important feature possible outside of just making models better. Cheaper prompts (10x) and faster (30-80%) first token on cache hits, incredible when doing few shot prompting with images or text

• @alexalbert__ Alex Albert on x

  We just rolled out prompt caching in the Anthropic API. It cuts API input costs by up to 90% and reduces latency by up to 80%. Here's how it works:

• @rednafi @rednafi on x

  @simonw These non-standard, fine-grained pricing models are already starting to give off that awful AWS pricing vibe. This makes me somewhat reluctant to use the feature.

• @danshipper Dan Shipper on x

  Anthropic (@AnthropicAI) just announced a game-changer for their API: Prompt caching. Think of prompt caching like this: You're at a coffee shop. The first time you visit, you need to tell the barista your whole order. But next time? Just say “the usual.” That's prompt [image]

• @ianwoodfill Ian Woodfill on x

  Prompt caching is clearly very exciting but isn't as much of a silver bullet as meets the eye. My first thought was that one would be able to effectively “compile” a massive prompt with lots of embedding context, and get massively improved performance over traditional RAG, at a

• @simonw Simon Willison on x

  Looks similar to Gemini's context caching, but the Anthropic pricing model is different Gemini charge $4.50/million tokens/hour to keep the context cache warm Anthropic charge for cache writes, and “cache has a 5-minute lifetime, refreshed each time the cached content is used”

• @simonw Simon Willison on x

  @AnthropicAI People have enough trouble understanding the difference between fine-tuning and prompting already, it really doesn't help if you misuse the term “fine-tune” in a tweet like this one!

• @chheplo Pratik Desai on x

  Anthropic is focused on builders and not killing your startup. This will significantly reduce the cost and latency. I'm becoming Claude stan.

• @mvpatel2000 Mihir Patel on x

  Very interesting to see differences in feature announcement: - GDM: corpo blog - OAI: weird rumors for a week - Anthropic: dude posts docs and it's out

• @yomaggievo Maggie Vo on x

  Prompt caching is akin to instant fine-tuning, unlocking application possibilities not possible before due to constraints on cost and latency. I'm stoked to see what you all can build now with this supercharged way of accessing Claude.

• r/singularity r on reddit

  New Prompt caching with Claude.  Coding with claude + cursor gonna become even more lit!

• r/singularity r on reddit

  Anthropic adds prompt caching (Talk directly to books, papers, etc)

• r/OpenAI r on reddit

  Prompt caching with Claude