Kimi K3 latency

Time to first token

Five streaming runs, trivial prompt ("Count to three"), 128-token ceiling:

Median TTFT is 2.82s. But look at the middle column: three of five runs never produced a content token at all. That is not a latency result — it is the reasoning-budget trap caught in the wild. A 128-token ceiling was entirely consumed by thinking, so the stream ended having emitted only reasoning. If you are streaming K3 to a user, this is what a silent failure looks like.

The gap between first token and first content token is the part that matters for perceived speed: on run 1, reasoning started at 2.74s but the answer only began at 4.62s. If you render reasoning_content, users see motion at ~2.8s. If you don't, they stare at a spinner for ~4.6s.

Long context

Non-streaming, trivial output, so this is essentially "how long until K3 has read your prompt and thought about it":

Wall clock grows with the prompt

Seconds per call, trivial output · measured 2026-07-16

Half a million tokens is 52 seconds and $1.49 for a single request, before K3 writes anything. The million-token window is real, but it is neither fast nor free — and at ~5× the tokens, we saw ~5× the wall clock, so the ~1M ceiling plausibly lands near two minutes. We did not test it; that would be inference, not measurement.

Speed comes from design, not caching

A reasonable hope: warm the prefix, skip the work, get the latency back. We tested it. It does not hold. Comparing cold and warm calls on identical prefixes, wall clock did not consistently improve — 3.56s→3.80s, 3.02s→4.29s, 3.99s→3.54s, 4.38s→4.27s, 5.22s→3.78s. Run-to-run variance swamped any cache effect.

The reason is structural: K3 spends seconds reasoning on every call regardless of whether the prompt was cached, and that dominates the timeline. Caching is a billing optimisation, not a latency one.

What to do about it

• Stream, always. Non-streaming at long context means a minute of silence. stream: true plus stream_options: {"include_usage": true}.
• Set client timeouts in minutes. Default SDK timeouts will kill legitimate long-context calls. We measured 52s on a single request that succeeded.
• Decide what to do with the reasoning gap. ~2.8s to first reasoning token, ~4.6s to first content token. Either show thinking, or show a spinner that expects to run for several seconds.
• Don't put K3 in a synchronous request path that a human is waiting on with a short budget. It thinks. That is the product.
• Trim your context. The latency scales with prompt size, and unlike cost, there is no cache discount to blunt it.

What these numbers are not

They are not a benchmark. Single client, single network path, single day, sample sizes of one to five. They carry our route to Moonshot's servers and whatever load Moonshot was under on launch day — a date when a brand-new flagship model is presumably busy.

What dominates every number here is K3 thinking, measured in seconds. Network routing — including the hop through a pass-through gateway — is milliseconds against that, which is why we treat these as K3's timings rather than any one endpoint's. Where we ran them.

Read them as magnitudes, not measurements of the model: seconds not milliseconds to first token; tens of seconds not seconds at 100k; a minute not ten seconds at 500k. That is the shape of the thing, and it is enough to design around. We will re-run and re-stamp as the launch-day load settles.