Example. When cheaper attempts compensate for lower success [ftip-00L3]
✍️source

Smaller caches can support more concurrent requests or avoid preemption and recomputation. These are actual serving mechanisms in vLLM's memory and scheduling guidance [⧉]. For post-training, the useful prediction is therefore about completed candidate attempts under a budget, including verification and optimization costs, rather than decoding speed alone.

Use the following cost model on the same hardware. One baseline attempt costs $c>0$; generation accounts for a fraction $f\in [0,1]$. Suppose generation becomes $s_{\rm gen}>0$ times as fast while all other cost per attempt stays fixed. The new cost and total attempt-throughput factor are

\[ c_K=c\left (1-f+\frac {f}{s_{\rm gen}}\right ),\qquad S=\frac {c}{c_K} =\frac {1}{1-f+f/s_{\rm gen}}. \]

For a fixed budget $B\geq 0$, this model permits $N_M=\lfloor B/c\rfloor $ baseline attempts and $N_K=\lfloor B/c_K\rfloor $ hybrid attempts. Assume $N_M,N_K\geq 1$ and verified-success events independent within each set of attempts, with respective probabilities $p_M,p_K\in [0,1]$. Then Theorem [ftip-0078] gives discovery probabilities $1-(1-p_M)^{N_M}$ and $1-(1-p_K)^{N_K}$. The hybrid matches or exceeds baseline discovery exactly when

\[ p_K\geq 1-(1-p_M)^{N_M/N_K}. \]

This follows by comparing the two failure probabilities and taking the nonnegative $N_K$-th root.

For an illustrative estimate, suppose generation consumes 80 percent of the baseline cost and use $s_{\rm gen}=2.2$, motivated by the batch-one decode figure in Example [ftip-00KW]. Applying that decode factor to the whole generation stage is a working approximation, not a measured rollout result. Then $S\approx 1.774$; a budget of $B=10c$ buys 10 baseline attempts or 17 hybrid attempts. If $p_M=0.10$ and $p_K=0.08$, the discovery probabilities are approximately $0.651$ and $0.758$. The break-even hybrid probability is about $0.0601$. Thus this model predicts that more attempts can outweigh a moderate per-attempt quality loss. The assumed probabilities are illustrative verified-success rates, not conversions of the benchmark scores in Example [ftip-00LC].

The prediction is strongest for long-context, generation-heavy work whose saved memory becomes usable rollout capacity. As verification or optimization dominates, $f$ shrinks and $S$ approaches one. Correlated attempts require the joint-law analysis of Convention [ftip-0077] instead of the independent product. Measure completed attempts, verified-success rates and total cost under a fixed prompt, decoding and verifier protocol to test the tradeoff. In the RLVR round of Example [ftip-004M], additional successful candidates can improve the available feedback; whether they produce useful accepted updates is a further optimizer-and-evaluation question, as Example [ftip-0085] demonstrates.

Context

Published hybrid-attention comparison [ftip-00KU]

Kimi Linear combines recurrent KDA layers with periodic full MLA attention. Its published comparison shows task-dependent score changes and lower long-context decoding times. The selected scores in Example [ftip-00LC], the memory estimate in Example [ftip-00KY], and the discovery calculation in Example [ftip-00L3] connect these observations to a post-training question: when can a cheaper attempt compensate for a possible lower per-attempt success probability?

Example. Published Kimi Linear operating point [ftip-00KV]

The Kimi Linear report compares experimental models pretrained on 1.4 trillion tokens, each with 48 billion total and 3 billion active parameters. Kimi Linear uses three Kimi Delta Attention layers for each full Multi-head Latent Attention layer, with no positional encoding (NoPE) in its MLA layers; the reference uses full MLA attention [kimi2025linear, Sections 4 and 5.4]. These are properties of the reported hybrid architecture.

These experimental models are distinct from the released Kimi Linear model, pretrained on 5.7 trillion tokens and supporting up to one million tokens of context. Appendix D compares that release with Moonlight, which has 16 billion total and 3 billion active parameters [kimi2025linear, Section 5.4.1 and Appendix D].

Example. Published efficiency observations [ftip-00KW]

For batch size one at one million tokens, Figure 7 labels a $2.9\times $ prefill speedup and a $2.2\times $ decoding speedup for Kimi Linear against MLA. Section 6.3 gives $2.3\times $ for the latter point, so the graphic and prose differ slightly [kimi2025linear, Figure 7 and Sections 5.6 and 6.3].

The larger-batch comparison in Figure 1(b) gives decoding time per output token of 1.84 milliseconds for Kimi Linear and 11.48 milliseconds for MLA, with a reported $6.3\times $ speedup. Section 6.3 describes this as a theoretical speedup from reallocating saved KV-cache memory to larger batches. It illustrates a different use of the memory saving from the batch-one latency result [kimi2025linear, Figure 1(b) and Section 6.3].

Figure 1(a) reports RULER scores of 84.3 for Kimi Linear and 81.3 for MLA at 128k context, with $3.98\times $ decoding acceleration [kimi2025linear, Figure 1(a)]. The report also gives a reduction of up to 75 percent in KV-cache use [kimi2025linear, Abstract]. Together these observations motivate studying context length and batch size as separate cost coordinates. The unequal RULER scores and different timing regimes are retained when interpreting those points.

Definition. Quality target and cost measurement [ftip-00KX]

For a reported benchmark score q, use the same task, prompt law, scoring rule, and post-training condition for both systems. The comparison records the smallest measured cost at which each system reaches q, when such a measurement exists. A throughput or cache ratio is not substituted for this iso-quality cost.

Example. A cache-memory estimate for hybrid attention [ftip-00KY]

The cited FLA KDA implementation at commit 6b6f546f stores one active recurrent state per sequence, with dimensions determined by the value heads and key/value dimensions rather than accumulated context length. Its newly allocated state uses float32 [⧉]. The optional short-convolution state has one fixed-width kernel buffer per channel and sequence [⧉]. This supplies a concrete implementation model for the recurrent part of the 3:1 mixture in Example [ftip-00KV].

Consider a stylized comparison with $4\ell $ layers, batch size $b$, and context length $T$, where $\ell ,b,T$ are positive integers. Assume each MLA layer stores $\kappa T$ bytes per sequence, with the same $\kappa >0$ in both designs, and each KDA layer stores one terminal recurrent-plus-convolution state of $\sigma >0$ bytes. The active attention-state memories are

\[ M_{\rm MLA}=4b\ell \kappa T,\qquad M_{\rm hybrid}=b\ell (\kappa T+3\sigma ),\qquad \frac {M_{\rm hybrid}}{M_{\rm MLA}} =\frac {1}{4}+\frac {3\sigma }{4\kappa T}. \]

The hybrid uses less active state exactly when $T>\frac {\sigma }{\kappa }$, and the ratio tends to $1/4$ as context grows. This explains why replacing three quarters of the growing caches can approach a 75-percent saving, while fixed recurrent-state overhead can erase the advantage at short contexts. The retained MLA layers keep the hybrid's state memory linear in $T$; the whole model is not a fixed-memory recurrent system.

Weights, training activations, workspaces, allocator padding and saved prefix states are outside this active-state calculation. At a fixed device memory budget, the estimate predicts more room for simultaneous or longer rollouts when attention state dominates. It does not predict a fourfold speedup. Training and decoding also use different execution paths: FLA selects chunk kernels for gradient-enabled computation and fused recurrence for short inference calls [⧉]. The fixed-state estimate is therefore a useful serving hypothesis, not a training-memory or training-speed formula.

Example. Common-recipe reading of the Kimi result [ftip-00KZ]

The report holds its stated pretraining and SFT recipes common across the experimental models [kimi2025linear, Sections 5.4--5.5 and Table 4]. The selected observations in Example [ftip-00LC] compare the resulting trained designs, including the hybrid's 3:1 KDA/MLA pattern and NoPE choice. The mixed score directions suggest a task-dependent tradeoff rather than one ordering of the two architectures.

The theoretical mechanism is a change in the cost of maintaining and using the prefix: recurrent layers compress it into state, while the retained MLA layers preserve growing attention caches. The model in Example [ftip-00KY] predicts the largest memory advantage at long contexts. If this advantage reduces the cost of generating useful candidates, Example [ftip-00L3] predicts when more attempts can offset a reduction in per-attempt quality. This is a conditional explanation to test through rollout costs and verified success rates; it does not require attributing the whole result to KDA alone.

Example. Measured iso-quality cost ratio [ftip-00L0]

Let $c_K(q)$ and $c_T(q)$ be measured inference costs for Kimi Linear and the Transformer reference at the same finite real score $q$ and context length, under one accounting rule. If $0\leq c_K(q)<+\infty $ and $0<c_T(q)<+\infty $, the measured ratio $\widehat \rho _{K/T}(q)=c_K(q)/c_T(q)$ is reported together with the full cost vectors. These measured costs need not be the inverse frontier infima of Definition [ftip-00JP]. If either threshold is unmeasured or the denominator is zero, the ratio is left undefined rather than inferred from the paper's speedup plot.

Remark. Hybrid attention is not pure linear attention [ftip-00L1]

Kimi Linear combines KDA with periodic full-attention layers. A measured advantage therefore identifies the reported hybrid intervention, not a claim that a purely linear recurrent architecture has the same quality or ceiling. The layer pattern belongs in the architecture record.

Remark. Demonstrated performance is a lower bound [ftip-00L2]

The published benchmark scores demonstrate attainable points for the reported checkpoints and protocol. They do not establish a representational ceiling, a universal scaling law, or a claim that additional cost cannot improve either architecture.

Example. When cheaper attempts compensate for lower success [ftip-00L3]
✍️source

\[ c_K=c\left (1-f+\frac {f}{s_{\rm gen}}\right ),\qquad S=\frac {c}{c_K} =\frac {1}{1-f+f/s_{\rm gen}}. \]

\[ p_K\geq 1-(1-p_M)^{N_M/N_K}. \]

This follows by comparing the two failure probabilities and taking the nonnegative $N_K$-th root.

Backlinks

Example. Common-recipe reading of the Kimi result [ftip-00KZ]

Example. How generation share changes the predicted gain [ftip-00LF]

Keep the illustrative generation factor $s_{\rm gen}=2.2$ from Example [ftip-00L3] and vary the baseline generation share $f$. Its cost model gives the following total attempt-throughput factors:

Generation share	Predicted throughput factor
20 percent	1.122
50 percent	1.375
80 percent	1.774
100 percent	2.200

These calculated values show why the same decoding improvement can matter much more for generation-heavy search than for a verifier- or optimizer-dominated workload. They suggest measuring the time spent in generation, verification and updates before selecting where to invest systems effort. A change in the bottleneck changes the useful intervention, even when the attention architecture stays fixed.

Remark. Measured and assumed costs [ftip-00LD]

A useful cost model can combine measurements with assumptions justified by the architecture and its implementation. In Example [ftip-00L3], the generation share $f$ and speed factor $s_{\rm gen}$ determine a predicted attempt-throughput factor. Their assumed values can be varied without pretending they were measured in the original experiment.

For fixed $f\in [0,1]$, write $S(s)=1/(1-f+f/s)$ for $s>0$. This function is nondecreasing. An interval $s_{\rm gen}\in [s_-,s_+]$, with $0<s_-\leq s_+$, therefore gives the conditional range $S\in [S(s_-),S(s_+)]$. This is sensitivity to a chosen assumption range; it is a statistical confidence interval only if a sampling argument supplies that interpretation.

Definition. Measured contrasts and conditional predictions [ftip-00LE]

A measured paired contrast evaluates the same declared quantity for two observed model instances under a specified task, protocol and cost accounting. The common-recipe scores in Example [ftip-00LC] are one example. A confirmatory comparison fixes its matching and selection rules before examining the scores.

A conditional paired prediction instead uses an explicit model to supply one or more cost or response quantities, then derives the comparison under those assumptions. The discovery probabilities in Example [ftip-00L3] are an example. Such a prediction can be motivated by existing observations and guide a later experiment; its unmeasured inputs remain assumptions, and agreement with new matched measurements tests the model.

Published hybrid-attention comparison [ftip-00KU]

Example. Published Kimi Linear operating point [ftip-00KV]

Example. Published efficiency observations [ftip-00KW]

Definition. Quality target and cost measurement [ftip-00KX]

Example. A cache-memory estimate for hybrid attention [ftip-00KY]

\[ M_{\rm MLA}=4b\ell \kappa T,\qquad M_{\rm hybrid}=b\ell (\kappa T+3\sigma ),\qquad \frac {M_{\rm hybrid}}{M_{\rm MLA}} =\frac {1}{4}+\frac {3\sigma }{4\kappa T}. \]

Example. Common-recipe reading of the Kimi result [ftip-00KZ]

Example. Measured iso-quality cost ratio [ftip-00L0]

Remark. Hybrid attention is not pure linear attention [ftip-00L1]

Remark. Demonstrated performance is a lower bound [ftip-00L2]

Example. When cheaper attempts compensate for lower success [ftip-00L3]
✍️source

\[ c_K=c\left (1-f+\frac {f}{s_{\rm gen}}\right ),\qquad S=\frac {c}{c_K} =\frac {1}{1-f+f/s_{\rm gen}}. \]

\[ p_K\geq 1-(1-p_M)^{N_M/N_K}. \]

This follows by comparing the two failure probabilities and taking the nonnegative $N_K$-th root.

Example. Counterexample: a Z-score gate is not an evaluation-safety certificate [ftip-0085]

Let two consecutive proposed reused updates have the same monitored energy, $g_{t-1}=g_t=1$. With trailing-window increment mean $\mu _t=0$, any positive scale $\sigma _t+\varepsilon $, and any positive threshold, DGG's score is \[ z_t=\frac {(g_t-g_{t-1})-\mu _t}{\sigma _t+\varepsilon }=0, \] so the gate accepts the proposal. Now choose a fixed evaluation interface for which the artifact before the accepted update has performance $J_{\rm ev}=1$ and the artifact after it has performance $J_{\rm ev}=0$.

This is consistent because the gate rule of Definition [ftip-005A] imposes no mathematical relation between its scalar monitor and $J_{\rm ev}$. A safety claim would need an additional assumption connecting proposed updates to the independent evaluation law; the Z-score calculation alone cannot provide it.

Theorem. Discovery under independent attempts [ftip-0078]

Assume the events $E_1,\ldots ,E_B$ of Convention [ftip-0077] are independent. Then

\[ \Pr (D_B)=1-\prod _{i=1}^{B}(1-p_i). \]

In particular, independent attempts with a common success probability $p$ satisfy

\[ \Pr (D_B)=1-(1-p)^B. \]

Proof.

The complement of discovery is $D_B^{\mathsf c}=\bigcap _{i=1}^{B}E_i^{\mathsf c}$. Independence gives $\Pr (D_B^{\mathsf c})=\prod _i\Pr (E_i^{\mathsf c}) =\prod _i(1-p_i)$. Taking complements proves the first identity; substituting $p_i=p$ proves the specialization.

This finite statement follows from the displayed hypotheses. The calculation in Example [ftip-0067] is its ten-attempt numerical preview.

Example. One response-level RLVR round [ftip-004M]

Three sampled responses pass through exact verification, group credit, and a proposed update in one finite RLVR round.

$\text{[math]}$

The exact-answer checker gives mean reward $\bar r=2/3$. Using the declared unnormalized centered credit $\widehat A_i=r_i-\bar r$ yields $(1/3,-2/3,1/3)$. An optimizer may use these numbers to propose a parameter change, but the tuple itself is not yet an accepted update.

The RLVR protocol is Definition [ftip-0044]. Its group-credit objects are Definition [ftip-004B], Definition [ftip-004C], and Definition [ftip-004D]. Together they specialize to this finite order of operations. Centered, unscaled credit and an exact-answer checker are local choices rather than requirements on other RLVR algorithms.

Example. Published Kimi Linear comparison [ftip-00LC]

Table 4 of the Kimi Linear report compares the experimental models pretrained on 1.4 trillion tokens in Example [ftip-00KV], after the same supervised fine-tuning (SFT) recipe. Both have 48 billion total and 3 billion active parameters. Three selected benchmark scores are [kimi2025linear, Sections 4 and 5.4 and Table 4]:

Arm	MMLU-Pro	LiveBench	EvalPlus
Kimi Linear	67.4	45.2	61.0
Full MLA	65.7	45.7	62.6

The entries use the paper's score units, with larger values better on each selected benchmark; LiveBench is reported as Pass@1. Kimi Linear is higher on MMLU-Pro and lower on LiveBench and EvalPlus in these observations. These are reported point estimates on different tasks, not a single capability score; the table supplies no uncertainty estimate.

The common recipe uses the K2 pretraining corpus, MuonClip and a shared training schedule. SFT proceeds from broad instruction data to targeted reasoning tasks; evaluation uses temperature 1.0 and an internal framework derived from LM-Harness [kimi2025linear, Section 5.4]. Thus the scores provide a concrete example of how the trained hybrid and full-MLA designs respond to the same recipe. The cost mechanism and its possible post-training consequences are developed in Example [ftip-00KZ].

Example. Published efficiency observations [ftip-00KW]

Convention. Repeated attempts and the discovery event [ftip-0077]

Fix a task instance, evaluator, success threshold, and a joint law for $B\geq 1$ attempts. Let $E_i$ be the event that attempt $i$ succeeds. The discovery event by attempt $B$ is

\[ D_B=\bigcup _{i=1}^{B}E_i. \]

Write $p_i=\Pr (E_i)$. Independence is an additional property of the declared joint law; it is not implied by repeated decoding, a shared model, or a common environment. In the independent and identically distributed case, write $p_i=p$. When the attempt law is the one fixed in Definition [ftip-005U], this $p$ is the corresponding successful-support probability.