Every step of a six-step task sends the same six tool definitions and the same policy document again, ahead of the one new tool result. Input tokens are the bill and the prefix is most of them. The invoice shows no cache reads, because a timestamp in the system prompt changes the prefix on every call, and the caching you switched on is charging you for writes.
What you get
You will end up with an agent loop whose stable prefix is cached across steps, and a stand-in provider that applies the documented caching rules offline. A script prices each run against a baseline. This is for you if a multi-step agent sends its tool list again on every call.
Short answer
Send tool definitions, the system prompt, and any reference document first, in the same bytes on every step, and append the tool results after them. Mark the end of that prefix with a cache breakpoint, or let automatic caching move one along the conversation. Then read the usage fields: cache reads that stay near zero mean something in the prefix changes on every call, and the bill is higher than with no caching at all.
You will need
Node 22 or later, Python 3.11 or later, and an API key for the provider once you leave the
stand-in. Verified 2026-09-25 against Node 22.22.2 and @anthropic-ai/sdk 0.128.0. The
samples never call a model. The SDK sends its real request body to a fetch stand-in that
applies the rules in
Anthropic’s prompt caching documentation
and answers with the same usage fields the API returns. The minimums and lifetimes on that
page change, so read it before you trust a number here.
Approaches compared
| Approach | When it fits | What it costs you | When to pick something else |
|---|---|---|---|
Anthropic prompt caching with cache_control | You want to choose what is cached: up to four explicit breakpoints, or one automatic one, at a five-minute or one-hour lifetime | Breakpoints to place and keep correct as the prompt evolves, and a write priced at 1.25 times a plain input token | You never send the same prefix twice inside five minutes |
| Gemini context caching | Implicit caching on by default, or an explicit cache object with a TTL when the saving has to be guaranteed | A minimum of 4,096 tokens on the 3.x models, and a cache object to create, name, and delete when you go explicit | Your prompts sit under the minimum, or you use the Interactions API, which has no explicit caching |
| OpenAI automatic prompt caching | Nothing to place: on by default from 1,024 tokens, with implicit breakpoints on older models and explicit ones from GPT-5.6 | Little to tune on older models, a prompt_cache_key to keep stable for routing, and a prefix that has to match exactly | You want to choose the lifetime: the default is 30 minutes, and longer retention is a separate option |
| Trimming the tool list | The task needs three of the tools and the rest are along for the ride | A selection step per task, and a tool the model needed that is no longer there | The tool list is the product, and any step may need any of it |
The four rows are two decisions. Explicit breakpoints are yours to place and yours to break; automatic caching places them for you and leaves nothing to tune when the hit rate is low. Trimming is not caching at all. It makes every step cheaper whether or not the cache hits, and it stacks with the other three.
Put the stable prefix first and mark where it ends
The cache hashes the request in one order: tools, then system, then messages, and a hit needs every byte up to the breakpoint to match a prefix an earlier request wrote. So the request is built in that order, with the parts that never change first and the conversation last.
export function buildRequest({ tools, system, messages, cache }) {
const body = {
model: MODEL,
max_tokens: 512,
tools: tools.map((t) => ({ ...t })),
system: system.map((s) => ({ ...s })),
messages,
}
if (cache) {
body.tools[body.tools.length - 1].cache_control = { type: 'ephemeral' }
body.system[body.system.length - 1].cache_control = { type: 'ephemeral' }
body.cache_control = { type: 'ephemeral' }
}
return body
}Two explicit breakpoints, on the last tool and on the last system block, and a top-level
cache_control that asks for automatic caching. The documentation allows four breakpoints and
the automatic one takes a slot, so this leaves one spare. The two explicit ones cover what the
automatic one cannot. The tool list and the system prompt are shared by every task, so an entry
written at the end of them serves the next task as well as the next step.
The stand-in provider is the same rules written down. It hashes the positions in order and writes an entry only where a breakpoint sits. It looks back at most twenty positions for an entry an earlier request wrote, and it caches nothing under the model’s minimum. An entry lives for five minutes from the request that wrote or read it.1
let hit = -1
if (cacheable) {
for (const bp of breakpoints) {
for (let i = bp, looked = 0; i >= 0 && looked < LOOKBACK; i--, looked++) {
const expires = entries.get(positions[i].hash)
if (expires !== undefined && expires > t) { hit = Math.max(hit, i); break }
}
}
for (const bp of breakpoints) if (positions[bp].upTo >= minimum) entries.set(positions[bp].hash, t + TTL_MS)
if (hit >= 0) entries.set(positions[hit].hash, t + TTL_MS)
}
const read = hit >= 0 ? positions[hit].upTo : 0
const write = cacheable ? positions[last].upTo - read : 0
const input = total - (cacheable ? positions[last].upTo : 0)It counts a token as four characters of JSON. The real count differs, and every comparison on this page is between runs against the same stand-in, so the ratios hold and the absolute numbers do not.
Measure reads against a baseline
Six runs of the same six-step task. The first sends no cache_control at all. The second is
the request shown earlier. The third puts a timestamp in the first system block and the fourth rotates the
tool list on every call. The last two trim the six tools to the three the task uses, with and
without caching.
node agent.mjs@anthropic-ai/sdk 0.128.0, model claude-opus-5, stand-in provider, tokens counted as JSON length over four
run steps input write read read share
baseline, no caching 6 11481 0 0 0%
prefix cached 6 0 2592 8889 77%
timestamp in the prefix 6 0 8126 3415 30%
tools reordered per call 6 0 11481 0 0%
tool list trimmed to three 6 9501 0 0 0%
trimmed and cached 6 0 2262 7239 76%
prefix cached, step by step
step input write read
1 0 1326 0
2 0 89 1326
3 0 363 1415
4 0 363 1778
5 0 88 2141
6 0 363 2229The baseline pays for 11,481 input tokens across six steps. The cached run pays for none as plain input: 77 percent of what it sent was read from cache, and the rest was written once. The step table shows the shape. Step one writes the whole prefix. Step two reads it back and writes the two new blocks, and every later step reads a little more and writes a little.
The timestamp run is the pitfall in its polite form. The tool list sits ahead of the system prompt in the hash order, so the entry at the end of the tools still hits, and the read column is not zero. That entry exists because the tool list alone clears the model’s minimum, 512 tokens on Claude Opus 5. On Claude Sonnet 5 the minimum is 1,024, which the tool list does not reach, so the same run reads nothing. Everything after the timestamp is written again on every step. A dashboard that shows any cache reads at all reports this as caching that works.
The rotated tool list is the pitfall in full. The first position differs on every call, so nothing
behind it can match, and the run writes 11,481 tokens at the write price without a single read. That is more expensive than sending no cache_control, and the bill puts a number on it.
Price it
Reads and writes are priced as multiples of a plain input token, and the multiples come from the providers’ documentation rather than from the code.2 The script reads the recorded runs and prices each one.
def cost_units(steps, write=1.25, read=0.1):
"""Total cost in units of one uncached input token."""
total = 0.0
for s in steps:
total += s["input_tokens"]
# The cache fields are typed integer or null, and a null has to count as zero.
total += (s.get("cache_creation_input_tokens") or 0) * write
total += (s.get("cache_read_input_tokens") or 0) * read
return totalpython3 bill.pymodel claude-opus-5, priced with the anthropic multipliers, in units of one uncached input token
run cost units vs baseline hit rate
baseline, no caching 11481 0% 0%
prefix cached 4129 64% 77%
timestamp in the prefix 10499 9% 30%
tools reordered per call 14351 -25% 0%
tool list trimmed to three 9501 17% 0%
trimmed and cached 3551 69% 76%The cached run costs 36 percent of the baseline. The rotated run costs 125 percent of it: it writes every token at the 1.25 price without reading one back, which is the bill for believing caching is on. The timestamp run saves 9 percent, all of it from the tool list. It looks like a working cache to anyone who checks for a non-zero read count instead of comparing against a baseline. Trimming alone saves 17 percent, and trimming plus caching is the cheapest run of the six, because the prefix it writes and reads back is shorter.
Read the fields rather than the flag. On the Claude API, cache_read_input_tokens and
cache_creation_input_tokens sit beside input_tokens, which counts only what came after the
last breakpoint. Both cache fields at zero means the prompt was under the model’s minimum, so
nothing was cached, without an error to tell you. OpenAI reports
usage.input_tokens_details.cached_tokens,3 and Gemini reports cached tokens in
usage_metadata.4 In every case the number to watch is the share of input read from cache,
against the run that cached nothing.
Check it worked
The stand-in’s rules are tested one by one, and the last two tests drive the agent loop through the real SDK and assert on what reached the wire.
node --test cache.test.mjs1..10
# tests 10
# suites 0
# pass 10
# fail 0
# cancelled 0
# skipped 0
# todo 0The Python suite holds the bill to the recorded runs, including the claim that the rotated tool list costs more than no caching.
python3 -m unittest -v test_bill 2>&1test_a_timestamp_in_the_prefix_costs_more_than_a_stable_one (test_bill.BillTests.test_a_timestamp_in_the_prefix_costs_more_than_a_stable_one) ... ok
test_a_write_costs_more_than_input_and_a_read_much_less (test_bill.BillTests.test_a_write_costs_more_than_input_and_a_read_much_less) ... ok
test_hit_rate_is_zero_without_caching (test_bill.BillTests.test_hit_rate_is_zero_without_caching) ... ok
test_null_cache_fields_count_as_zero (test_bill.BillTests.test_null_cache_fields_count_as_zero) ... ok
test_reordering_the_tools_costs_more_than_not_caching_at_all (test_bill.BillTests.test_reordering_the_tools_costs_more_than_not_caching_at_all) ... ok
test_report_lists_every_run (test_bill.BillTests.test_report_lists_every_run) ... ok
test_the_cached_run_is_cheaper_than_the_baseline (test_bill.BillTests.test_the_cached_run_is_cheaper_than_the_baseline) ... ok# pass 10 and seven ok lines. Test six is the one to read: a timestamp in the first system
block leaves exactly the tool list cacheable, measured in tokens.
When it goes wrong
Both cache fields are zero on every call. The prefix up to the breakpoint is under the model’s minimum. The documentation puts that at 512 tokens for Claude Fable 5.1 and Claude Opus 5.5, 1,024 for Claude Sonnet 5, and 4,096 for Claude Haiku 4.5. Move more stable content ahead of the breakpoint, or accept that a short prompt is not worth caching.
Reads stop after a pause in the conversation. The five-minute lifetime is measured from the start of the request that last wrote or read the entry, not from the end of its response, so a long generation eats into it. Use the one-hour lifetime for a task whose steps are minutes apart. Its write costs 2 times a plain input token, against 1.25 for the five-minute one.
Reads drop in the middle of a long task. The automatic breakpoint looks back twenty positions for an earlier write, and a step that adds more blocks than that pushes the last write out of reach. Add an explicit breakpoint at the end of the stable prefix, as the request on this page does, so the search back has an entry to find.
The hit rate is high and the bill barely moved. The prefix is small and the tool results are large, so most of each step’s tokens come after the last breakpoint. Cut the tool results down before they enter the conversation; caching prices the prefix and nothing else.
When not to do this
Do not put a timestamp, a request id, a user id, or a random example into the system prompt or the tool list. Every one of them is a different prefix per call, and the writes it causes cost more than the caching you meant to get. Put per-request context in the last user message, after the breakpoint.
Do not order tools by relevance per step or let a serializer emit their keys in a different order. The documentation warns that some languages randomize key order when they produce JSON, and a tool list that differs in one byte is a tool list the cache has never seen.
Do not reach for the one-hour lifetime when the prompt is reused every minute. The five-minute entry is refreshed for free on every read, and the longer one costs 2 times a plain input token to write, against 1.25.
Related how-tos
Last verified
Verified 2026-09-25 against Node 22.22.2, Python 3.11.15, and @anthropic-ai/sdk 0.128.0.
Every output block is what the command preceding it printed. No request reached a provider: the
usage numbers come from the stand-in in provider.mjs, which implements the rules as the
documentation states them and counts tokens as JSON length over four.
Footnotes
-
The documentation measures the lifetime from the start of the request that writes or reads the entry, not from the end of its response. It gives the arithmetic: a response that streams for four minutes leaves about one minute for the follow-up to arrive. Time spent generating the answer counts against the cache of the question. The stand-in here answers at once, which is the one respect in which it is more generous than the service. ↩︎ Back to text
-
The pricing table gives the multipliers as 1.25 for a five-minute write, 2 for a one-hour write, and 0.1 for a read. Two footnotes carry the exceptions: reads cost 0.025 times the base price on Claude Fable 5.1 and Claude Mythos 5.1, and 0.05 on Claude Opus 5.5. The script uses 0.1, the rate on the model in the runs. On Fable 5.1 and Mythos 5.1 a read is one fortieth of an input token, and on Opus 5.5 one twentieth. The rotated tool list is exactly as expensive on all of them as it is here. ↩︎ Back to text
-
OpenAI’s guide says that on models before GPT-5.6 the reported
cached_tokensis computed by subtracting the hidden system tokens from the last matched breakpoint and rounding down to a multiple of 128. GPT-5.6 and later report the exact eligible boundary. A counter that rounds to a power of two describes the cache’s own bookkeeping rather than your prompt, and the guide says as much by listing the two behaviors side by side. ↩︎ Back to text -
Gemini’s explicit caching is a cache object you create with a model, contents, and a time to live. The time to live defaults to one hour when unset, and the whole feature is in beta under
v1beta. The implicit minimum on the same page is 4,096 tokens for the 3.x models and 2,048 for the 2.5 ones. The threshold doubled between generations while the advice stayed put: large and common contents at the beginning of the prompt, and similar requests close together in time. ↩︎ Back to text