Kimi K3 latency

Time to first token

Five streaming runs, trivial prompt ("Count to three"), 128-token ceiling:

Median TTFT is 2.82s. But look at the middle column: three of five runs never produced a content token at all. That is not a latency result — it is the reasoning-budget trap caught in the wild. A 128-token ceiling was entirely consumed by thinking, so the stream ended having emitted only reasoning. If you are streaming K3 to a user, this is what a silent failure looks like.

The gap between first token and first content token is the part that matters for perceived speed: on run 1, reasoning started at 2.74s but the answer only began at 4.62s. If you render reasoning_content, users see motion at ~2.8s. If you don't, they stare at a spinner for ~4.6s.

Re-measured 2026-07-18: real prompts are a different regime

Two days later we ran a larger pass: ten streaming runs per endpoint, each with a unique realistic prompt (a two-sentence explanation task — something K3 actually thinks about), no cache hits, through both of EvoLink's endpoints:

Three things this adds to the 2026-07-16 picture:

• Prompt complexity moves TTFT by 3–4×. The same day, the same client, a trivial prompt still returned its first token in ~3.35s. Ask for actual thinking and the median first token lands at 11.3s. Budget for the prompt you'll really send, not the benchmark one.
• The swing is enormous. Individual runs ranged 2.9s to 40s to first content. A p95 of ~36s means one call in twenty makes your user wait half a minute for the first visible character — unless you stream reasoning or show progress.
• The primary endpoint earns its name. direct.evolink.ai beat the fallback api.evolink.ai on every median, by roughly 4s at the middle of the distribution. Use direct in production; keep api as the fallback it's documented to be.

Same-day throughput was steady where TTFT was not: 15–18 tokens/sec median once content starts, both endpoints, both days. The wait is front-loaded; the stream itself is fine. Comparing gateways? The same protocol ran against OpenRouter too, in paired rounds across two windows — no stable winner. EvoLink's own median moved from 21.4s in the first window to 31.1s an hour later — time-of-day drift as large as the gap between gateways.

Long context

Non-streaming, trivial output, so this is essentially "how long until K3 has read your prompt and thought about it":

Wall clock grows with the prompt

Seconds per call, trivial output · measured 2026-07-16

Half a million tokens is 52 seconds and $1.49 for a single request, before K3 writes anything. The million-token window is real, but it is neither fast nor free — and at ~5× the tokens, we saw ~5× the wall clock, so the ~1M ceiling plausibly lands near two minutes. We did not test it; that would be inference, not measurement.

Speed comes from design, not caching

A reasonable hope: warm the prefix, skip the work, get the latency back. We tested it. It does not hold. Comparing cold and warm calls on identical prefixes, wall clock did not consistently improve — 3.56s→3.80s, 3.02s→4.29s, 3.99s→3.54s, 4.38s→4.27s, 5.22s→3.78s. Run-to-run variance swamped any cache effect.

The reason is structural: K3 spends seconds reasoning on every call regardless of whether the prompt was cached, and that dominates the timeline. Caching is a billing optimisation, not a latency one.

Re-confirmed 2026-07-18 at four prefix sizes from 1.2k to 70k tokens: every warm call hit the cache (billing-verified), and TTFT still did not consistently improve — one size got faster, three did not.

What to do about it

• Stream, always. Non-streaming at long context means a minute of silence. stream: true plus stream_options: {"include_usage": true}.
• Set client timeouts in minutes. Default SDK timeouts will kill legitimate long-context calls. We measured 52s on a single request that succeeded.
• Decide what to do with the reasoning gap. ~3s to first reasoning token on trivial prompts, 11s median on real ones, and first content arrives seconds to tens of seconds after that. Either show thinking, or show progress UI designed for a wait measured in tens of seconds at p95.
• Don't put K3 in a synchronous request path that a human is waiting on with a short budget. It thinks. That is the product.
• Trim your context. The latency scales with prompt size, and unlike cost, there is no cache discount to blunt it.

What these numbers are not

They are not a benchmark. Single client, single network path, two days, sample sizes of one to ten. They carry our route to Moonshot's servers and whatever load Moonshot was under — on 2026-07-16 and again two days later, when a brand-new flagship model is presumably busy.

What dominates every number here is K3 thinking, measured in seconds. Network routing — including the hop through a pass-through gateway — is milliseconds against that, which is why we treat these as K3's timings rather than any one endpoint's. Where we ran them.

Read them as magnitudes, not measurements of the model: seconds not milliseconds to first token on trivial prompts, tens of seconds on real ones; tens of seconds not seconds at 100k; a minute not ten seconds at 500k. That is the shape of the thing, and it is enough to design around. We re-ran once already — the 07-18 pass above — and will keep re-stamping.

Kimi K3 latency FAQ

How fast is the Kimi K3 API?

It depends on the prompt. On a trivial prompt we measured ~3 seconds median to first token (2.8s on 2026-07-16, 3.35s on 07-18). On realistic prompts, the 2026-07-18 median was 11.3s to the first reasoning token and 21.4s to the first content token (n=10, p95 35.7s), with throughput of 18.4 tok/s (median) once content starts. That first token is always reasoning rather than the answer. These are single-client measurements, not a benchmark.

How long does Kimi K3 take with a long context?

We measured 10.5 seconds for a 98,625-token prompt and 52.0 seconds for a 497,718-token prompt, both with trivial output. Scaling looked roughly linear, so the full 1,048,576-token window plausibly approaches two minutes, though we did not test it.

Does prompt caching make Kimi K3 faster?

No. Comparing cold and warm calls on identical prefixes, wall clock did not consistently improve, and some warm calls were slower. K3 reasons on every call regardless of caching, and that dominates the timeline. Caching is a billing optimisation, not a latency one.