What is prompt caching?
A support bot resending the same 6,000-word policy document on every message pays to reread it each time, unless the prompt's front is cached.
Prompt caching
context caching
Prompt caching stores the processed state of a prompt's unchanged prefix, such as a system prompt or a long document. A later request reusing that prefix skips reprocessing it, so the model reads only the new part of the prompt, which lowers both cost and response time.
Most prompts sent to an LLM have a stable front and a changing tail. A coding assistant resends the same tool definitions and codebase context on every message; only the user's latest question changes. A support agent resends the same policy document; only the customer's message is new. Without caching, the model reprocesses the stable part every time, at full price.
Caching breaks that prefix out. The first call processes it normally and writes it to a cache. Every call after that, within the cache's time window, reads the stored version instead of reprocessing it. Only new tokens at the end of the prompt cost full price.
Anthropic's documentation, verified on 2026-08-14, prices a cache hit on Claude at 0.1x the base input token rate, a 90% discount. A 5-minute cache write costs 1.25x the base rate, and a 1-hour write costs 2x. The minimum prefix length that can be cached ranges from 512 to 4,096 tokens depending on the model. OpenAI's documentation, checked the same day, prices cached input tokens at 0.1x the uncached rate too. Automatic caching triggers at 1,024 tokens or more, and cache lifetimes vary by model generation.
It cuts cost and latency together
Most cost optimizations trade something away. A smaller model is cheaper and less capable. A shorter context window is cheaper and forgets more. Prompt caching does not make that trade. You send the same prompt, get the same answer, and pay less for the part that did not change.
Latency improves for the same reason cost does. A cached prefix skips the processing step that reading long input normally requires, so the model starts producing the answer sooner. When the stable prefix runs thousands of tokens, that gap decides whether a response feels instant or makes someone wait.
In agents we run, the biggest wins come from tool definitions and system instructions. Those sit at the front of nearly every call an agent makes. They rarely change mid-session, and an agent can make dozens of calls in one task. Caching that prefix once and reusing it across the run keeps a long session cheap instead of expensive.
The saving depends entirely on prompt structure. Put the stable content first and the changing content last, or the cache breaks on every call and you pay the write cost for nothing.
- Stable prefixSystem prompt, tool definitions, a long document.
- First call: cache writeFull processing cost, plus a write premium.
- Later calls: cache readStable prefix reused at a fraction of the price.
- New suffix onlyThe changing part of the prompt is billed normally.
- Cache expiresTTL runs out or content changes; the next call rewrites it.
The cache holds a prefix, not the whole prompt. Reordering a prompt so the stable part comes first is what makes caching work at all.
Common questions
01How much does prompt caching actually save?
The saving scales with how much of your prompt is stable and how many calls reuse it. Anthropic and OpenAI both price a cache hit at 0.1x the normal input token rate, a 90% discount. A prompt that is 90% stable prefix can cut its input cost by close to that share on every repeat call. A prompt with almost nothing stable saves little.
02Does prompt caching change the model's answer?
No. Caching only changes how the prefix is processed internally, not what tokens the model sees or how it responds. The same prompt produces the same distribution of answers whether the prefix was cached or processed fresh.
03How long does a prompt cache last?
It expires on a fixed time window rather than staying forever. Anthropic offers a 5-minute default and a 1-hour option at a higher write cost. OpenAI's newer models use a 30-minute fixed window, and older models vary by retention policy. Every provider refreshes the timer when the cache is reused, so an active session can keep its cache alive well past the base window.
Related terms
- Context window →The total space a prompt has to fit in. Caching changes what you pay for that space, not its size.
- Claude Agent SDK →Where a caching strategy for tool definitions and system prompts gets built into an agent's request loop.
- OpenAI API →OpenAI's own caching behavior, for teams comparing providers on cost per repeated call.
