Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving

Serving a language model splits into two jobs: reading the prompt (prefill) and writing the reply one token at a time (decode). Mooncake runs them on separate machines and pools the KV cache between them.

The serving platform behind Moonshot AI's Kimi assistant reads prompts on one pool of GPUs, writes replies on another, and keeps the attention keys and values that pass between them in a cache built from the cluster's spare CPU memory and SSDs. A scheduler routes each request by its predicted latency.

In the one-hour sample of Kimi traffic the authors released, the average prompt is 7,590 tokens and the average reply is 182, about 42 prompt tokens for every token generated. At that ratio, reading prompts takes most of the compute, and holding their keys and values takes most of the GPU memory.

A model-as-a-service provider such as Moonshot AI is paid per request served, so it wants as many requests through its GPUs as possible. Users, meanwhile, care about two latencies. The time to first token (TTFT) runs from pressing enter to the first word of the reply. The time between tokens (TBT) is the gap between consecutive words as the rest of the reply streams out. A service-level objective (SLO) bounds both, and the paper writes the bound as a multiple of an unloaded baseline: $\text{TTFT}_{P90}=10\times$ means 90% of requests must reach their first token within ten times the TTFT that the same request would have if it ran alone, with no other request interfering. The end-to-end experiments use 10× for TTFT and 5× for TBT. Because the bound is a ratio, a 100k-token prompt gets a proportionally longer TTFT allowance than a 1k-token one.

Mooncake's design follows from a handful of facts, taken in this order below: a request is two workloads with opposite hardware profiles; the attention keys and values (the KV cache) that link them are large; separating the two workloads makes that cache something to move, store and share; sharing works by matching prompt prefixes; a scheduler then routes each request to minimize its predicted TTFT; one very long prompt can be split across several machines; and when the cluster is overloaded, the scheduler has to reject requests before it spends GPU time on them.

Prefill and decode: two phases with opposite costs

A decoder-only Transformer (the LLaMA family, GPT and the model behind Kimi are all this kind) handles a request in two stages. Prefill runs the whole prompt through the network in one pass. All the prompt tokens are processed in parallel, the pass produces the first output token, and along the way every layer computes a key and a value vector for every prompt token. Decode then writes the reply one token at a time: each step feeds the newest token in, produces the next one, and repeats until the model emits a stop token.

Prefill is compute-bound: with thousands of tokens in flight, each weight read from memory is used thousands of times, so the GPU's arithmetic units are the bottleneck (except for short prompts, which do not have enough tokens to fill them). Its cost also grows faster than the prompt. The linear layers (the attention projections and the MLP) cost the same per token, about $2\times 68.45$ billion floating-point operations (FLOPs) per token for the LLaMA2-70B shape. Attention compares each token with every earlier token, so its cost grows with the square of the length. The paper calls the sum superlinear, which is more accurate than "quadratic": for LLaMA2-70B the two terms are equal at about 104,000 tokens, so below that the linear layers still do most of the work. Going from an 8k prompt to a 128k prompt multiplies the tokens by 16 and the FLOPs by about 33.

Decode is memory-bound. One step of one request does very little arithmetic, but it must read all 138 GB of weights and that request's whole KV cache out of GPU memory to do it. The GPU spends the step waiting for bytes, and the relevant limit is memory bandwidth (bytes per second), not memory capacity. Servers therefore batch decode: they run one step for many requests together, so that one read of the weights serves all of them. Decode throughput then rises with batch size, but the gain per doubling shrinks, because each added request brings its own cache to read. In the figure, look for the prefill curve bending upward and the decode curve flattening out.

A server that runs both stages on the same GPUs has to interleave them. When a 100k-token prompt arrives, its prefill occupies the GPUs for seconds, and every reply being decoded on those GPUs stops streaming until it finishes. Each of those pauses is a TBT violation for a request that was otherwise fine.

What the KV cache is and how big it gets

Inside each attention layer, every token produces a query, a key and a value vector. To predict the next token, the newest token's query is compared with the keys of all earlier tokens, and the resulting weights mix their values. Because attention is causal, a token's key and value depend only on the tokens before it, so they never change once computed. The KV cache keeps them. With it, a decode step computes the key and value of the one new token, appends them, and attends over the stored rest; without it, every step would recompute the keys and values of the entire prefix. Caching keys and values is an inference-time technique that later serving systems adopted; the original Transformer paper does not describe it.

Mooncake's experiments use a model with the LLaMA2-70B architecture: 80 layers, 64 query heads, and 8 key/value heads of dimension 128, stored in 16-bit floats. The cache for one token holds a key and a value for each layer and each key/value head:

That is 320 KiB per token. LLaMA2-70B uses grouped-query attention, in which eight query heads share one key/value head, a generalization of multi-query attention; with a separate key and value per query head (plain multi-head attention, $n_{kv}=64$) the cache would be eight times larger, 2.6 MB per token. The paper states none of these sizes; they follow from the model configuration. At the trace's average prompt of 7,590 tokens, one request carries 2.49 GB (2.3 GiB) of cache. An 8-GPU node has 640 GB of GPU memory, the 16-bit weights take about 138 GB of it, and the remaining 502 GB holds about 200 such caches if nothing else used memory. Decode batch size, and with it decode throughput, is capped by how many caches fit.

The cache also explains why batching cannot make decode compute-bound. Whether an operation is compute-bound or memory-bound depends on its arithmetic intensity, the FLOPs it performs per byte it reads from memory. A GPU runs at full speed only when an operation's intensity exceeds the ratio of its peak FLOP rate to its memory bandwidth. For the A800 GPUs in the paper's cluster (the export version of the A100, with the same 312 TFLOP/s of 16-bit tensor throughput and 2,039 GB/s of memory bandwidth) that ratio is about 153 FLOPs per byte. Decode has two kinds of operation with very different intensities. In the weight multiplications, each 2-byte weight is read once and used for 2 FLOPs per request in the batch, so the intensity equals the batch size. Attention is different: for each earlier token and layer, a request reads its own $2 n_{kv} d_h$ cached numbers and spends $4 n_h d_h$ FLOPs on them.

The batch size does not appear. Adding a request adds its attention FLOPs and its cache bytes in the same proportion, so attention stays at 8 FLOPs per byte, about 5% of the A800's peak, at any batch size. The paper's §10 makes this point (decode attention's intensity is "only proportional to" the head ratio and "cannot be increased by increasing the batch size"); the value 8 is our arithmetic from the model configuration. In the figure, move the batch slider and watch the teal point slide right while the amber one stays put, then switch the attention type.

Two things follow for serving. Every byte of cache a GPU does not have to hold leaves room for another request in the decode batch, and every cached key and value that can be loaded instead of recomputed saves prefill compute.

How Mooncake works: separate pools and a shared KV cache

Mooncake divides the cluster into a prefill pool and a decode pool. A long prefill runs on prefill machines and can no longer pause anyone's decoding, because no decoding happens there. This split, called disaggregation, did not originate with Mooncake. Splitwise (November 2023) proposed it first, and the Mooncake paper credits it as having "further motivated our progress"; DistServe and TetriInfer (January 2024) are concurrent work that the authors say corroborates their findings. Mooncake's own contributions are the four pieces described in the rest of this page: the cache pool, the scheduler, a way to spread one long prefill across machines, and an admission policy for overload.

Splitting the pools means the KV cache built during prefill has to travel to a decode machine, so Mooncake gives the cache its own storage tier. A GPU cluster has a large amount of CPU memory (DRAM) and SSD capacity next to the GPUs that inference barely uses, and fast RDMA network cards between the nodes (up to 800 Gbps per node in the paper's cluster, about 100 GB/s). Mooncake gathers that spare DRAM and SSD into a disaggregated KV cache, a cluster-wide store of key/value blocks far larger than GPU memory. In CPU memory the cache is kept as paged blocks. Each node runs a process called Messenger that moves blocks between machines with GPUDirect RDMA, which lets the network card read and write GPU memory without copying through the CPU.

A global scheduler, Conductor, assigns each incoming request a prefill instance and a decode instance. The request then goes through four steps, and the cache is involved in each:

1. KV cache reuse. The prefill instance loads whatever leading part of this prompt is already in the pool into GPU memory.
2. Incremental prefill. It computes the rest of the prompt, and stores the newly produced keys and values back into CPU memory. If more than a threshold of tokens (prefill_chunk, "typically larger than 1000 tokens") is uncached, the prefill is split into chunks and pipelined, as described below.
3. KV cache transfer. Messenger streams the cache to the decode instance's CPU memory layer by layer, overlapping with step 2 rather than waiting for it to finish.
4. Decoding. The decode instance loads the cache into GPU memory and the request joins the next batch.

Step 4 uses continuous batching, from Orca: instead of waiting for a fixed batch to start and finish together, the decode scheduler revisits the batch before every step, adding newly arrived requests and removing finished ones, so a short reply does not wait for a long one. The baseline in every experiment is vLLM, which combines continuous batching with paged KV memory and runs prefill and decode on the same GPUs.

Reusing the cache by prefix

Many prompts start the same way. A system prompt is prepended to nearly every call, a user asks several questions about one pasted document, and a chat resends its whole history every turn. Two prompts that share their first $k$ tokens have identical keys and values for those tokens, because each token's key and value depend only on the tokens before it. The cache for a shared prefix can therefore be computed once and loaded for every later prompt that starts with it; only the part after the prompts diverge needs prefill.

To find shared prefixes, Mooncake cuts each prompt into blocks of 512 tokens and gives each block a key: a hash of the block's tokens concatenated with the previous block's key. Since every key folds in the key before it, block $i$ of two prompts has the same key only if the two prompts agree on all tokens up to the end of block $i$. Comparing keys from the start until the first mismatch gives the length of the shared prefix in blocks. Git commit IDs work the same way: a commit's hash covers its parent's hash, so equal IDs imply equal histories. (The 512-token pool block is unrelated to vLLM's 16-token pages, which organize memory inside one GPU.)

def prefix_hash(tokens, B=512):
    keys, prev = [], b""
    for i in range(0, len(tokens), B):
        prev = H(tokens[i:i + B], prev)    # this block's tokens + previous key
        keys.append(prev)
    return keys

def shared_prefix_blocks(keys_new, keys_cached):
    n = 0
    while n < min(len(keys_new), len(keys_cached)) and keys_new[n] == keys_cached[n]:
        n += 1
    return n                               # reusable tokens = n * 512

The released trace stores these keys as integer IDs instead of text. Two of its requests:

# two requests from Mooncake's open trace (no user text is released)
{ "timestamp": 27482, "input_length": 6955, "output_length": 52,
  "hash_ids": [46,47,48,49,50,51,52,53,54,55,56,57, 2353, 2354] }
{ "timestamp": 30535, "input_length": 6472, "output_length": 26,
  "hash_ids": [46,47,48,49,50,51,52,53,54,55,56,57, 2366] }
# the first 12 hash_ids match: a shared prefix of 12 x 512 = 6,144 tokens

The second request reuses 6,144 tokens of cache and prefills its remaining 328 (6,472 − 6,144). At 320 KiB per token the reused part is 2.01 GB, which takes about 20 ms to load over a 100 GB/s link; computing those 6,144 tokens would take about 0.9 s at an assumed 1 PFLOP/s per node. The figure puts a cached prompt above an incoming one that differs by a single edited token.

Two properties are visible there. Reuse is prefix-only: one changed token voids the cache for every block after it, so a prompt layout that puts the varying text (a timestamp, a user name) at the start wastes the cache for everything that follows. And prefill compute falls a little more slowly than the token count, because the remaining tokens still attend to the cached ones: with 12 of 16 blocks reused, prefill handles 25% of the tokens and about 26% of the FLOPs. Reuse saves prefill only; each decode step still reads the whole cache.

The paper measures how much reuse the trace offers by simulating a single global pool of 512-token blocks under three eviction policies (LRU, LFU, and a length-aware variant of LFU that favours blocks appearing late in requests) at capacities from 1,000 blocks to unbounded. With LRU the hit ratio rises from 30% at 1,000 blocks to 50% at 50,000 blocks, and an unbounded pool reaches only 51%. One block of this model's cache is 512 × 327,680 bytes, 168 MB, so 50,000 blocks are 8.4 TB of KV, an amount that fits in a cluster's combined DRAM and SSD but in no GPU (an A800 has 80 GB).

So in this workload about half of the KV cache can be reused, and the curve flattens well before the pool is unbounded. The authors caution against reading the flat end as a sign that larger pools are unnecessary, since the trace covers one hour and real capacity needs scale with traffic, and they note in their related-work section that reuse can reach 90% in specific applications such as their own chat-with-papers service. Popularity is also very uneven: more than half of the cached blocks are never used again, while a few (a common system prompt, for instance) are read tens of thousands of times. If only one node held such a block, every request needing it would wait on that node or on its network link. Mooncake copies hot blocks to other nodes, using a heuristic tied to the scheduler instead of a model that predicts future block usage.

How Conductor picks a prefill node

Reuse creates a routing conflict. The node holding a popular prefix saves the most prefill compute, but because that prefix is popular, the node also has the longest queue. Two simple policies both fail. Always routing to the longest cached prefix piles traffic onto the hot node; routing to the shortest queue ignores the cache and recomputes prefixes that were already available.

Conductor instead estimates, for every prefill instance, the TTFT the request would get there, and picks the smallest. The estimate has three parts:

$T_{\text{queue}}$ is the sum of the predicted prefill times of the requests already queued on that instance. $T_{\text{prefill}}$ comes from a model fitted to offline measurements of prefill time as a function of prompt length and cached-prefix length; Transformer prefill runs a fixed sequence of operations, so the authors report the fit is accurate "as long as enough offline data is available". $T_{\text{transfer}}$ is nonzero only when the instance fetches a longer prefix from another node, and it is harder to predict because it depends on how congested the sending node's network is. Equation (1) is our summary of the paper's Algorithm 1, which the paper does not number:

# Conductor, Algorithm 1 of the paper (simplified): pick a prefill instance
keys = prefix_hash(prompt)
best_len, holder = find_best_prefix_match(prefill_pool, keys)
ttft, chosen = INF, None
for inst in prefill_pool:
    have = inst.prefix_len(keys)               # tokens of THIS prompt cached here
    t_q = estimate_queue_time(inst)            # sum of queued prefill times
    if best_len / have < balancing_threshold:  # local cache is close enough
        t = t_q + prefill_time(len(prompt), have)
    else:                                      # fetch the holder's longer prefix
        t_x = transfer_time(holder, inst, best_len - have)
        t = t_q + t_x + prefill_time(len(prompt), best_len)
    if t < ttft:
        ttft, chosen = t, inst
dec, tbt = select_decode_instance(decode_pool)  # load-balancing on decode side
if ttft > TTFT_SLO or tbt > TBT_SLO:
    return reject(429)                          # HTTP 429 Too Many Requests
if best_len / chosen.prefix_len(keys) > balancing_threshold:
    copy_kv(holder, chosen)                     # the hot prefix gains a replica
return chosen, dec

A worked example with illustrative numbers. An 8,192-token prompt arrives. Node A holds its first 6,144 tokens and has 2.0 s of queued work; node D holds nothing and has 0.3 s queued. Taking prefill times from the FLOP count above at an assumed 1 PFLOP/s per node, the full prompt costs 1.21 s and the last 2,048 tokens alone cost 0.32 s. Routing to A gives 2.0 + 0.32 = 2.32 s. Recomputing on D gives 0.3 + 1.21 = 1.51 s. Having D fetch A's 2.01 GB of prefix over an idle 100 GB/s link (0.02 s) gives 0.3 + 0.02 + 0.32 = 0.64 s, so Conductor picks D with a fetch, and A's prefix now also lives on D. The balancing_threshold decides when that fetch branch is tried: an instance recomputes from its own cache unless the best remote prefix is longer than its own by more than this factor. The authors tune it by hand and name adaptive tuning as future work.

The figure runs the same rule on four instances. A holds the whole prefix but sits behind the longest queue; B holds half; C and D are cold. Start with "recompute only" and drag the load, then allow prefix fetches.

That copy is the hot-spot migration mentioned earlier. Whenever Conductor sends a request to a node that fetches a prefix, the prefix gains a replica, so popular blocks spread to more nodes in proportion to how often load pushes requests away from their holders, with no explicit popularity forecast. The decode instance is chosen separately by plain load balancing, and if either the best predicted TTFT or the decode instance's predicted TBT exceeds its SLO, Conductor rejects the request immediately with HTTP 429 (Too Many Requests). In the paper's experiment (Figure 8, values given only in the plot), this cache-aware scheduling achieves lower average TTFT and higher TTFT-SLO attainment than both random routing and load-balancing routing.

Prefilling one long prompt across several nodes

At 128k tokens one 8-GPU node needs tens of seconds of compute for a single prefill (about 40 s in our model at 1 PFLOP/s), so meeting a TTFT target requires spreading one request across several nodes. The standard way to split a model across GPUs is tensor parallelism from Megatron-LM, which shards every weight matrix and combines the partial results with two all-reduce operations per layer. Inside a node those all-reduces run over NVLink at 400 GB/s on the A800. Across nodes they run over RDMA at about 100 GB/s, four times slower, and each one blocks the next layer. The paper says this lowers MFU, the fraction of peak arithmetic a prefill node achieves. Sequence parallelism (Ring Attention and similar methods, which split the prompt's tokens across nodes) still communicates at least once per layer, and the authors add that keeping a separate sequence-parallel group would need frequent resizing.

Mooncake groups $X$ prefill nodes into a pipeline. Each node holds a contiguous slice of the layers, the prompt is cut into chunks of at most prefill_chunk tokens, and chunk 1 enters node 1; when node 1 finishes its layers for chunk 1 it passes the activations to node 2 and starts chunk 2. Causal attention makes this possible: chunk 2's tokens only need keys and values from chunk 1 and themselves, and node 1 already holds chunk 1's keys and values for its own layers. The paper calls this chunked pipeline parallelism (CPP). Nodes communicate only at stage boundaries, one chunk's activations at a time, and that transfer can overlap with the next chunk's compute.

The figure models both options for LLaMA2-70B with the assumptions listed under the chart. Look at the network readout as well as the times.

At 128k tokens and four nodes, CPP cuts TTFT from 40.5 s to 13.0 s (3.1×) and tensor parallelism to 15.3 s. The larger difference is network traffic: each pipeline node sends one copy of the prompt's activations, about 2.1 GB, while each tensor-parallel node sends about 515 GB of all-reduce traffic, and that traffic competes with KV cache transfers on the same network cards. CPP has its own overhead: with few chunks, the pipeline spends much of its time filling and draining, so at 16k tokens (two chunks) tensor parallelism comes out ahead in this model, and an 8k prompt is a single chunk that gains nothing from extra nodes. The paper lists the other benefit as operational: prompts below the threshold run unchunked and pay no overhead, so the same node groups serve short and long prompts without being reconfigured. The authors call CPP, to their knowledge, "the first application in the inference stage" of pipeline-style acceleration. Inter-layer pipelining for inference existed earlier, so the new part is narrower: pipelining a single long prompt's prefill across nodes to cut its TTFT (see the provenance note).

The second prefill technique concerns GPU memory. The paper defines a request's occupation cost as its KV size $S$ times the time $T$ it holds that cache in GPU memory. A 128k-token prompt's cache is 42.9 GB, 5.4 GB on each of the node's eight GPUs, and while it sits there the node has that much less room for other requests. If the prefill node loads the whole cached prefix, then computes, then stores the whole fresh cache, the loads and stores add to $T$. Prefill runs one layer at a time, so Mooncake loads each layer's cached KV just before that layer runs and starts storing a layer's fresh KV as soon as the layer finishes:

# layer-wise prefill: KV load and store run beside each layer's compute
launch(load[0])
for i in range(L):
    wait(load[i])                # layer i's cached KV is now in VRAM
    if i + 1 < L:
        launch(load[i + 1])      # prefetch the next layer's cached KV
    h = layer(h, i)              # attention + MLP over the uncached tokens
    launch(store[i])             # ship layer i's fresh KV to CPU memory
wait_all(store)                  # only the last stores outlast compute

The paper says the prefill instance's execution time then becomes "roughly equivalent to either the KVCache loading time or the standard prefilling time, depending on the prefix cache proportion relative to the input length". The figure shows when each case applies. At the trace's 50% hit ratio and 100 GB/s, compute dominates by a factor of about 50 and the transfers disappear behind it. Raise the cached share to 90% and lower the bandwidth to 10 GB/s, and the 242 ms of loading exceeds the 129 ms of compute, so loading sets the pace.

Because the cache streams out of VRAM as fast as it is produced, the prefill scheduler can disregard GPU memory "as long as it can contain a single request" and plan only around where cache blocks live and how much DRAM is free. The same argument is the paper's answer to chunked prefill (splitting a prompt into pieces that ride along inside decode batches, as in SARATHI): interleaving a prefill with decoding stretches $T$, which raises the occupation cost, and long prompts need cross-node parallelism that decode machines do not. Mooncake does inline a prefill into a decode batch, but only when the prompt is short enough to run in one piece without breaking the TBT SLO.

Overload, early rejection and load oscillation

Moonshot AI cannot add GPUs as fast as Kimi's traffic grows, so at peak hours the cluster is overloaded and some requests must be turned away. The paper's objective is goodput: throughput counting only requests that complete within the SLO. Mooncake's definition is strict: a request that is prefilled but then misses its TBT target counts for nothing, and all the compute spent on it is wasted. So a request that will not finish in time should be rejected before its prefill starts.

In a coupled server, load is usually measured as the number of running requests over capacity. Mooncake measures it directly against the SLO: an instance's load is its predicted maximum TTFT (prefill) or TBT (decode) relative to the corresponding limit. The first policy the paper tries, early rejection, checks the decode pool's load at arrival, before prefill begins, and rejects if the larger of the two pool loads is over the threshold. That stops the waste, but the cluster then oscillates. The paper's Figure 9 shows prefill and decode load swinging in anti-phase over 20 minutes on 20 machines.

A request admitted now adds decode load only after its prefill finishes, roughly one prefill duration later. Suppose both pools are lightly loaded: Conductor admits many requests until the prefill pool is full. Their prefills finish, and the decode pool fills up. Conductor now sees high decode load and rejects arrivals, so the prefill pool empties. The decode batch drains as replies finish; Conductor sees low decode load, admits a burst, and the cycle restarts. The pools are separate machines, so this is not contention for a shared resource; it is the same overshoot as a shower whose hot water takes a few seconds to arrive, where the user keeps turning the tap further than needed because the temperature lags the setting. Each pool spends part of every cycle underused.

Early rejection based on prediction removes the lag by deciding against the decode load expected at the moment the incoming request's prefill will finish. Predicting it per request would require each reply's length, which the authors call too costly or too inaccurate, especially under overload, and leave to future work. They predict at the system level instead: assume every request decodes for the same time $t_d$; for a future time $t$, add the requests the prefill pool will have finished by $t$ to the decode instances, remove the ones that will have run longer than $t_d$, and compare the resulting average TBT with the TBT limit.

The test cluster had 8 prefill and 8 decode instances replaying 23,000 real requests at twice their recorded speed. The baseline policy, which checks each pool only when the request reaches it, rejected 4,183 requests, early rejection 3,771, and early rejection based on prediction 3,589. The predictive policy rejects the fewest: by keeping both pools busy instead of alternating, the same cluster completes more requests.

Results

All experiments run on nodes of eight A800-SXM4-80GB GPUs with a model that has the LLaMA2-70B architecture but no trained weights, used to protect proprietary details. The results therefore measure serving speed and latency, never answer quality. The size of the gain depends on how long the prompts are:

• Public datasets. On ArXiv Summarization (8,088-token average input, 229 output) and L-Eval (19,019 input, 72 output), a cluster of 3 prefill and 1 decode instance beats four vLLM instances of the same total size by 20% and 40% in throughput while meeting the SLOs. L-Eval prompts share long prefixes (cache ratio above 80%), which is part of the larger gain. A 2-prefill, 2-decode split had worse TTFT than both, which the paper attributes to an imbalance between prefill and decode load: these workloads need more prefill capacity than decode.
• Simulated long context. With 16k to 128k prompts, Mooncake's throughput gain ranges from 50% to 525%. The abstract scopes the 525% to "certain simulated scenarios", and in this test vLLM ran requests one at a time rather than batched, because long prefills disrupted its decoding.
• Replayed Kimi traffic. On 10 prefill plus 10 decode instances against 20 vLLM instances, with a 30 s TTFT limit and 0.1 s per-token TBT limit, nearly all of Mooncake's requests met the TBT SLO against 57% for vLLM, while TTFT distributions were nearly identical. The paper summarizes this as about 75% more requests handled under the SLOs.

In the replay, first-token latency is the same for both systems and the difference is in TBT: vLLM's decoding stalls each time a long prefill lands on the same GPUs, and Mooncake's decode pool takes on a prefill only when it fits in one piece within the TBT SLO. The advantage is largest for Kimi's input-heavy traffic, long prompts with short replies, and shrinks for short, balanced requests, where the coupled baseline loses less to interference.