Prompt caching explained: how it works on OpenAI, Anthropic and Gemini, and when it pays

What is prompt caching?#

Prompt caching is a provider-side discount for repeated input. Before a model writes anything, it processes your whole prompt into internal state (the attention “keys and values” for every token). If the next request starts with exactly the same tokens, the provider can reuse that saved state for the shared start, the prefix, instead of computing it again. You pay a reduced price for those cached tokens, and the response starts sooner.

Three details matter in practice:

• It matches prefixes, not whole prompts. Everything up to the first token that differs can be reused; everything after it is processed normally. One changed character near the top of a prompt makes the rest of it uncacheable.
• It doesn’t change the answer. OpenAI’s documentation is direct: “Prompt caching does not change how the model generates output tokens.” Output is billed at the normal price.
• It isn’t response caching. The provider doesn’t store or replay answers. Caching whole responses for repeated questions (sometimes called semantic caching) is something you build in your own app.

Caches are private. Anthropic says caches are never shared across organisations and are isolated per workspace on its own API; OpenAI says caches are not shared across organisations.

How much does prompt caching save?#

It depends on how much of each prompt is a repeated prefix and how often it repeats. Take a document assistant on Claude Sonnet 5.5: a 10,000-token prefix (instructions plus a document) followed by a 200-token question, with a 300-token answer. The first request writes the prefix to the cache; the next five read it:

The write costs $2.50 per million tokens instead of $2, so request 1 is dearer than it would be without caching. From request 2 the prefix costs $0.10 per million, and what remains is mostly the answer: output at $10 per million is now the largest part of each request. That is the general pattern. Caching can remove most of the cost of repeated input, but it does nothing for output. The cost estimation guide applies the same maths to a whole 10-turn conversation.

How prompt caching works on Anthropic, OpenAI and Google#

All three providers cache prefixes, but they differ in whether you have to ask for it, how long entries live and how writes are charged. As documented on 8 October 2026:

To compare cached prices across every model, filter the model comparison to models with prompt caching and sort by the cached price column.

Anthropic: you choose what to cache

Claude caches nothing unless you ask. The simplest way is a top-level cache_control: {"type": "ephemeral"}, which places the breakpoint on the last cacheable block and moves it forward as a conversation grows. For finer control you mark up to four blocks yourself. The prefix is built in a fixed order, tools then system then messages, and a change at one level invalidates that level and everything after it. Prompts below the model’s minimum are silently not cached, so check the usage fields. Anthropic also notes that cache hits are not deducted from your rate limit.

OpenAI: automatic, with write charges on the newest models

OpenAI caches automatically. On GPT-5.6 and later, implicit mode places a breakpoint at the end of the latest eligible message, explicit mode lets you mark breakpoints, and entries last at least 30 minutes after the latest write or reuse. Those models charge 1.25× the input price to write. Earlier models have no write charge and keep entries for about 5 to 10 minutes of inactivity, with an optional 24-hour retention on some. Unlike Anthropic, OpenAI counts cached tokens toward your tokens-per-minute limit. Cached state lives on individual machines, and OpenAI notes that traffic above 15 requests per minute can be routed to overflow machines. On models before GPT-5.6, a stable prompt_cache_key helps send related requests to the same cache; from GPT-5.6, routing is automatic.

Google: implicit by default, explicit for guaranteed savings

Gemini 2.5 and newer cache implicitly with nothing to set up, but Google describes implicit caching as having no cost-saving guarantee. Explicit caching lets you create a cache object (a document, a video, a long system instruction), refer to it in later requests and get a guaranteed discount. You pay the reduced rate for cached tokens each time they’re used, plus storage for as long as the cache lives. Google’s tips: put large, common content at the start of the prompt and send requests with a similar prefix close together.

When does prompt caching pay off?#

Caching pays off once the savings on reads cover the extra cost of the write. If a write costs W times the input price and a read costs R times, a prefix needs more than (W − 1) ÷ (1 − R) reads within its lifetime to come out ahead. On Claude Sonnet 5.5, W is 1.25 and R is 0.05:

• 5-minute cache: (1.25 − 1) ÷ (1 − 0.05) = 0.26, so a single read pays for the write.
• 1-hour cache: with Anthropic’s 2× write price, (2 − 1) ÷ (1 − 0.05) = 1.05, so it takes 2 reads. Anthropic’s pricing page reaches the same answer for its standard 0.1× read price: caching “pays off after one cache read for the 5-minute duration (1.25x write), or after two cache reads for the 1-hour duration (2x write)”.

The lifetime is the catch. Suppose the same 10,000-token document gets one question every 10 minutes. A 5-minute cache has always expired, so every request pays the write premium: $0.15 an hour for the prefix, against $0.12 with no caching at all. A 1-hour cache writes once and reads five times, which brings the hour to $0.045. Match the cache lifetime to the gap between your requests, not to the length of a session.

Google’s explicit caches follow different maths: storage is charged per token per hour, so the cache pays off when the hits in an hour save more than an hour of storage. Divide the storage price by the per-token saving on each hit to get the hits per hour you need, using the prices on Google’s pricing page for your model.

How to structure prompts for cache hits#

1. Put stable content first. Tool definitions, the system prompt, long documents and few-shot examples go at the top; the user’s question and anything per-request go last.
2. Keep the prefix byte-for-byte identical. A timestamp, request ID or user name near the top breaks every hit. So does serialising the same JSON with keys in a different order.
3. Don’t change settings mid-conversation. On Claude, changing tool definitions invalidates every cache level, and changing thinking or effort settings typically invalidates the cached messages.
4. Warm the cache before a burst. Anthropic notes that a cache entry only becomes available after the first response begins, so send one request and wait for it before firing parallel ones.
5. Append, don’t rewrite. In chats and agents, add new turns at the end. Summarising or trimming old history changes the prefix and forces a fresh write.

Counting the stable part of your prompt with the token counter tells you whether it clears the provider’s minimum and what share of each request it is.

When prompt caching doesn’t help#

• Short prompts. Below the minimum (512 to 4,096 tokens depending on provider and model) nothing is cached.
• Slow or bursty-then-idle traffic. If the prefix is reused less often than the cache lives, you pay write premiums for nothing.
• Prompts that differ early. Per-user data or retrieved documents at the top leave little shared prefix. Move them below the fixed instructions.
• Output-heavy work. When most of the cost is the answer (long drafts, code generation, heavy reasoning), caching the input moves the total only a little. A model with cheaper output does more; see the cheapest LLM APIs.
• Batch jobs, unless prepared. Anthropic says cache hits in batches are best-effort because requests run concurrently in any order; it suggests writing the prefix to a 1-hour cache first.

How to check that caching is working#

Don’t assume hits; read them from the response. Anthropic returns cache_creation_input_tokens (written) and cache_read_input_tokens (read) alongside input_tokens, which counts only the tokens after the last breakpoint. OpenAI reports cached tokens under usage.input_tokens_details.cached_tokens, inside the input_tokens total. Gemini reports cached tokens in the response’s usage_metadata. Pricing a Claude response from its usage:

def claude_cost(usage, input_price, output_price, write=1.25, read=0.1):
    """Prices in USD per 1M tokens. write/read: the model's cache multipliers."""
    return (
        usage["input_tokens"] * input_price
        + usage["cache_creation_input_tokens"] * input_price * write
        + usage["cache_read_input_tokens"] * input_price * read
        + usage["output_tokens"] * output_price
    ) / 1_000_000

# A cache hit: 10,000 prefix tokens read, a 200-token question, a 300-token answer.
usage = {"input_tokens": 200, "cache_creation_input_tokens": 0,
         "cache_read_input_tokens": 10_000, "output_tokens": 300}

# Example prices (not a real model): $1 in, $5 out per 1M tokens.
hit = claude_cost(usage, 1.00, 5.00)
miss = (10_200 * 1.00 + 300 * 5.00) / 1_000_000
total_in = sum(usage[k] for k in ("input_tokens", "cache_creation_input_tokens", "cache_read_input_tokens"))
print(f"with cache ${hit:.4f}, without ${miss:.4f}, "
      f"cached share {usage['cache_read_input_tokens'] / total_in:.0%}")

# Output: with cache $0.0027, without $0.0117, cached share 98%

Track the cached share over a day of traffic. If it’s far below the share of your prompt that is meant to be stable, something in the prefix is changing between requests. Once you know your real cached share, enter it in the LLM cost calculator to see the monthly effect.