Kimi K3 prompt caching

It's automatic, and cold is always cold

There is no opt-in. Caching keys off a repeated prefix, and you can see it in usage.prompt_tokens_details.cached_tokens.

We generated prefixes that had never been sent to Moonshot before — freshly randomised content, so a cold call is genuinely cold — and sent each one twice:

// first call, fresh 12,504-token prefix
"usage": {
  "prompt_tokens": 12504,
  "completion_tokens": 32
  // no cached_tokens field at all
}

// second call, byte-identical prefix, ~2s later
"usage": {
  "prompt_tokens": 12504,
  "completion_tokens": 32,
  "prompt_tokens_details": {"cached_tokens": 12288}   // 96 blocks of 256
}

Across every prefix we tested, from 1.3k to 29k tokens, the cold call cached exactly zero. There is no partial credit on first contact. The discount exists only from request two onward — which means a workload that never repeats a prefix can never benefit, no matter how large its prompts.

The block size is 256 tokens

Cache hits are not granted per token. In nine tests, cached_tokens came back as an exact multiple of 256 every single time, leaving an uncached remainder of up to 255 tokens:

The uncached tail never reaches one block

Tokens left uncached per warm prefix · nine prefixes, 1.3k–29k · measured 2026-07-16

The practical consequence: caching pays off in proportion to prefix size. A 300-token prefix has at most one cacheable block and up to 255 tokens of uncacheable tail — most of your prompt, at full price. A 50k prefix has nearly 200 blocks and the tail is rounding error. Do not bother engineering for cache hits on small prompts.

One caveat from the 2026-07-16 raw data: two tiny repeated prompts cached end to end — prompt_tokens 90 returned cached_tokens 90, and 86 returned 86. So a ~90-token prompt repeated verbatim can hit in full; the 256-token block accounting above is what we measured at the 1k–70k prefix sizes. 256-block granularity re-confirmed 2026-07-18 via EvoLink (hits of 1,024/4,352/17,408/70,400).

prompt_cache_key did nothing we could measure

Moonshot's API docs (read 2026-07-20) list a prompt_cache_key parameter, described as being for session-based caching and "recommended" for multi-turn. That reads like the lever you're supposed to pull. It isn't — at least not for prefix reuse.

We ran a clean test: a brand-new prefix, first call already carrying an explicit prompt_cache_key, then a second identical call. The result was indistinguishable from the no-key case:

Same miss on the cold call, same hit on the warm one. Automatic prefix caching had already done the work. For repeated-prefix workloads, you do not need this parameter — and if you added it expecting a discount, that is not where your discount came from.

It saves money, not time

This one surprised us. The intuition is that a cache hit skips work, so it should be faster. Comparing cold and warm calls on the same prefix, wall-clock time did not consistently improve:

Run-to-run variance swamped any cache effect at these sizes — and remember K3 spends seconds reasoning on every call regardless, which dominates the timeline. Treat prompt caching as a billing optimisation, not a latency one. If you need speed, see what we measured on latency.

Everything above carries through gateways

All of the mechanics on this page were measured against api.moonshot.ai on 2026-07-16. Two days later we re-ran the core protocol through EvoLink (direct.evolink.ai) at four fresh prefix sizes — and through OpenRouter at one — to check that nothing is lost in the hop:

Every finding held: cold calls cached zero, warm hits landed in exact 256-token blocks with the tail uncached, and the money-not-time result repeated (one size faster, three not). OpenRouter's single test hit identically — 4,352 cached of a 4,501-token prefix — with its usage.cost receipts confirming the 10× billing difference in dollars. The gateway comparison has those numbers. One wrinkle worth knowing either way: on a cache miss the response omits prompt_tokens_details entirely — code that asserts on the field's presence will break on first calls.

How long a warm prefix stays warm

A warmed prefix was still returning the same 12,032 cached tokens 210 seconds later, with no decay across probes at 30s, 90s and 210s.

That is a floor, not the TTL — we stopped probing there. Moonshot documents no TTL, no minimum prefix length, and nothing about whether cache writes carry a surcharge, so we are not going to invent numbers. What we can say: a prefix reused within a few minutes stays warm, which covers the agent-loop case that matters most.

How to stay in the cheap row

• Put the stable bytes first. System prompt, tool definitions, long documents at the front; the varying user turn last. The cache matches a prefix — one early edit invalidates everything after it.
• Keep drift out of the prefix. A timestamp, a request ID, a re-serialized JSON blob with reordered keys — any of these change the bytes and silently drop you to $3.00 per 1M. This is the most common way teams lose the discount without noticing.
• Batch against one prefix. Twenty questions about the same document want one warm prefix and twenty short tails.
• Don't engineer caching for small prompts. Under ~256 tokens there is nothing to cache; under ~1k the tail is most of your bill.
• Accept the cold call. The first request of every cold period pays full price. Budget for it rather than trying to design it away.
• Watch cached_tokens in production. It's the only way to know your hit rate is what you think. If it's absent, you missed.

What the spread is worth

An agent with a 50,000-token fixed prefix serving 200 requests a day — 10M input tokens daily:

Arithmetic on the official 2026-07-20 rates, input only. The cheap row is idealised — cold calls bill full price and the tail never caches. Output bills at $15.00/1M either way, and reasoning tokens may exceed your entire input bill on this workload. Ceiling and floor, not a quote.

Prompt caching FAQ

How does prompt caching work on Kimi K3?

Automatically, on a repeated prefix, with no parameter required. Send a byte-identical prefix again and the repeated portion bills at $0.30 per 1M instead of $3.00. We measured hits granted in exact multiples of 256 tokens, with the remaining tail never cached.

Do I need prompt_cache_key to get the Kimi K3 cached price?

No. We tested a fresh prefix with and without an explicit prompt_cache_key and the cached token counts were identical: zero on the cold call, 9,984 on the warm one in both cases. Automatic prefix caching already does the work.

Does prompt caching make Kimi K3 faster?

Not measurably. Comparing cold and warm calls on the same prefix, wall-clock time did not consistently improve across five prefix sizes: some warm calls were slower than their cold counterparts. Treat prompt caching as a billing optimisation, not a latency one.

How long does the Kimi K3 prompt cache last?

At least 210 seconds. A warmed prefix returned the same 12,032 cached tokens with no decay across probes at 30, 90 and 210 seconds. Moonshot documents no TTL, so this is a measured floor rather than the actual limit.