Prompt caching
Every model we serve caches prompts, every one of them reports cache hits in the same field, and that field is the one your bill is computed from. On a real agent loop it was the difference between 13,433 weighted tokens and about 2,700.
You pay less for the part that did not change
An agent resends its whole context on every turn. Caching means the unchanged prefix is charged at a cache rate instead of the input rate, and the cache rate is a fraction of it.
Nothing needs turning on. Send the same prefix again within the provider's cache window and the hit happens upstream. There is no cache header to set, no beta flag, and no separate endpoint.
What you can control is whether your prefix stays stable. Anything that changes early in the prompt, a timestamp in a system message being the usual culprit, invalidates everything after it. Put the volatile part last.
What it was worth on a real session
One customer's agent loop on 2026-09-12, read off our own metering. The context grows as the agent works, which is the normal shape and the one caching is built for.
| Call | Input | Cached | Weighted tokens |
|---|---|---|---|
| first, cold | 250,555 | 0 | 13,433 |
| second | 250,652 | 250,496 | 2,709 |
| twentieth | 257,343 | 257,024 | 2,844 |
The context grew by seven thousand tokens across those calls and the cost per call stayed flat, because almost all of the growth was already cached by the time it was sent again. Without caching, every one of those turns would have been charged like the first.
How to see a cache hit
Cache hits come back on the usage object of your own response. We did not invent the field or move it: we checked each upstream and they all report it in the same place.
"usage": {
"prompt_tokens": 251462,
"completion_tokens": 101,
"prompt_tokens_details": {
"cached_tokens": 250752
}
}That is the number our metering reads, so what you can see and what you were charged for come from one source. The per-model detail is on where each model reports cached tokens, and the arithmetic that turns it into a bill is on weighted tokens.