Kimi K3 prompt caching

It's automatic, and cold is always cold

There is no opt-in. Caching keys off a repeated prefix, and you can see it in usage.prompt_tokens_details.cached_tokens.

We generated prefixes that had never been sent to Moonshot before — freshly randomised content, so a cold call is genuinely cold — and sent each one twice:

// first call, fresh 12,504-token prefix
"usage": {
  "prompt_tokens": 12504,
  "completion_tokens": 32
  // no cached_tokens field at all
}

// second call, byte-identical prefix, ~2s later
"usage": {
  "prompt_tokens": 12504,
  "completion_tokens": 32,
  "prompt_tokens_details": {"cached_tokens": 12288}   // 96 blocks of 256
}

Across every prefix we tested, from 1.3k to 29k tokens, the cold call cached exactly zero. There is no partial credit on first contact. The discount exists only from request two onward — which means a workload that never repeats a prefix can never benefit, no matter how large its prompts.

The block size is 256 tokens

Cache hits are not granted per token. In nine tests, cached_tokens came back as an exact multiple of 256 every single time, leaving an uncached remainder of up to 255 tokens:

The uncached tail never reaches one block

Tokens left uncached per warm prefix · nine prefixes, 1.3k–29k · measured 2026-07-16

The practical consequence: caching pays off in proportion to prefix size. A 300-token prefix has at most one cacheable block and up to 255 tokens of uncacheable tail — most of your prompt, at full price. A 50k prefix has nearly 200 blocks and the tail is rounding error. Do not bother engineering for cache hits on small prompts.

prompt_cache_key did nothing we could measure

Moonshot's API docs list a prompt_cache_key parameter, described as being for session-based caching and "recommended" for multi-turn. That reads like the lever you're supposed to pull. It isn't — at least not for prefix reuse.

We ran a clean test: a brand-new prefix, first call already carrying an explicit prompt_cache_key, then a second identical call. The result was indistinguishable from the no-key case:

Same miss on the cold call, same hit on the warm one. Automatic prefix caching had already done the work. For repeated-prefix workloads, you do not need this parameter — and if you added it expecting a discount, that is not where your discount came from.

It saves money, not time

This one surprised us. The intuition is that a cache hit skips work, so it should be faster. Comparing cold and warm calls on the same prefix, wall-clock time did not consistently improve:

Run-to-run variance swamped any cache effect at these sizes — and remember K3 spends seconds reasoning on every call regardless, which dominates the timeline. Treat prompt caching as a billing optimisation, not a latency one. If you need speed, see what we measured on latency.

How long a warm prefix stays warm

A warmed prefix was still returning the same 12,032 cached tokens 210 seconds later, with no decay across probes at 30s, 90s and 210s.

That is a floor, not the TTL — we stopped probing there. Moonshot documents no TTL, no minimum prefix length, and nothing about whether cache writes carry a surcharge, so we are not going to invent numbers. What we can say: a prefix reused within a few minutes stays warm, which covers the agent-loop case that matters most.

How to stay in the cheap row

• Put the stable bytes first. System prompt, tool definitions, long documents at the front; the varying user turn last. The cache matches a prefix — one early edit invalidates everything after it.
• Keep drift out of the prefix. A timestamp, a request ID, a re-serialized JSON blob with reordered keys — any of these change the bytes and silently drop you to $3.00 per 1M. This is the most common way teams lose the discount without noticing.
• Batch against one prefix. Twenty questions about the same document want one warm prefix and twenty short tails.
• Don't engineer caching for small prompts. Under ~256 tokens there is nothing to cache; under ~1k the tail is most of your bill.
• Accept the cold call. The first request of every cold period pays full price. Budget for it rather than trying to design it away.
• Watch cached_tokens in production. It's the only way to know your hit rate is what you think. If it's absent, you missed.

What the spread is worth

An agent with a 50,000-token fixed prefix serving 200 requests a day — 10M input tokens daily:

Arithmetic on the official 2026-07-16 rates, input only. The cheap row is idealised — cold calls bill full price and the tail never caches. Output bills at $15.00/1M either way, and reasoning tokens may exceed your entire input bill on this workload. Ceiling and floor, not a quote.