Let \(M\) denote an existing model intervention. An architecture refinement is a record \(A=(\mathcal X,\mathcal Y,\operatorname {Map}_A)\) whose map realizes the same declared input and output interface. Forgetting \(A\) returns \(M\); no architecture claim is made when the field is omitted.

When needed, extend the record by \(O\) for optimizer and state-update rules, and \(S\) for kernels, precision, memory, cache, and scheduling. The triplet \((A,O,S)\) is descriptive; it is not a scalar intelligence score.

Fix an architecture \(A\), its base artifact, a declared intervention set \(\mathfrak I_A\), and the common evaluation interface of Convention [ftip-005D]. Each \(\eta \in \mathfrak I_A\) specifies a protocol \(P_{A,\eta }=\operatorname {PostTrain}(A,\eta )\) in that measurable and absolutely integrable evaluation domain, with real performance \(J_{\rm ev}(P_{A,\eta })\). Let \(c_A(\eta )\in [0,+\infty ]\) be its scalar cost under the study's declared accounting rule: a fixed measured cost or an expected nonnegative cost, as specified. For a finite budget \(C\in \mathbb R_{\geq 0}\), define \[ V_A(C)=\sup \left \{ J_{\rm ev}(P_{A,\eta }): \eta \in \mathfrak I_A,\ c_A(\eta )\leq C \right \}\in \overline {\mathbb R}. \] Infinite-cost interventions are infeasible at every such budget.

As in Definition [ftip-005M], the empty feasible set has value \(-\infty \); an unbounded-above feasible score set has value \(+\infty \). The value is real exactly when that set is nonempty and bounded above. Increasing \(C\) enlarges the feasible set, so \(V_A\) is nondecreasing. A finite supremum need not be attained by any intervention.

Frontier differences and derivatives are ordinary real operations only where the relevant values are finite, with differentiability additionally required for a derivative. Cost ratios use the domain in Definition 3.3.2. The allowed intervention set, evaluation law, and accounting rule are part of the definition, so \(V_A\) is not a universal intelligence function.

A statement that depends only on the model/interface map is invariant under changes to architecture \(A\), optimizer \(O\), or systems \(S\) that leave that map fixed. A conclusion about one of those fields instead requires assumptions that distinguish its possible values.

For fixed task and evaluation laws, distinguish the representational ceiling of \(A\), the optimizer-reachable ceiling of \((A,O)\), and the systems-feasible frontier of \((A,O,S,C)\). Each layer is conditional on the objects named; none implies that the next layer attains it.

For an architecture record \(A\), optimizer record \(O\), systems record \(S\), evaluation configuration \(\mathfrak e\) from Definition [ftip-00CE], and declared intervention \(\eta \) at cost cap \(C\), define the capability vector

Its coordinates respectively record state access, adaptive computation, representation and abstraction, plasticity, and portability under the named task, intervention, evaluation law, and cost accounting. A coordinate may be a descriptive field, an estimand, or \(\bot \) when the record is missing, incompatible, or not measured. The tuple is therefore not a common scalar and does not order architectures without a declared comparison rule.

The five coordinates attach to the existing ceiling layers conditionally: representation claims first require a map from \(A\); optimizer-reachable plasticity requires \((A,O)\); state, adaptive computation, and portability claims require the relevant \((A,O,S)\) interface, task family, and deployment regime. A systems-feasible claim also carries \(C\), the cost vector and its declared scalarization. The records in Definition 3.1.1, Definition 3.1.2, Definition 3.1.5, and Definition [ftip-00CE] remain authoritative.

The scalar frontier \(V_A(C)\) of Definition 3.1.3 is a projection onto the declared utility \(J_{\rm ev}\) for one architecture and intervention envelope; it is not a capability coordinate. For a fixed evaluation configuration, a vector-valued study may instead retain the conditional set

where Pareto dominance is declared for the reported coordinates. The set is conditional on the task, intervention class, evaluation law, cost vector, and missingness rule; it is not a universal intelligence frontier.

Matched-compute arms in § 3.4 compare entries only after stating which coordinates are held fixed, which are allowed to vary, and how the cost vector is charged. Cost, numerical stability, and monitorability remain evaluation coordinates alongside the vector, so a lower scalar cost or a stable implementation is not silently counted as a capability increase.

For an architecture record \(A\), distinguish token steps, feed-forward depth passes, recurrent passes, and latent-state updates. Let \(d(A,r)\) be the declared number of depth passes in run \(r\); it is a computation coordinate, not a synonym for parameter count or reasoning quality.

Giannou et al. show that a fixed shallow Transformer placed in a loop can execute programmed iterative computations, including conditional branches and in-context algorithms, in Looped Transformers as Programmable Computers[giannou2023looped]. This is a source result under its program, state, and precision assumptions; it is not a universal claim about language-model capability.

Xue et al. report that depth sharing can reduce trainable parameters while limiting modeling capacity, and study width and mixture-of-experts remedies in Go Wider Instead of Deeper[xue2021wider]. The comparison is empirical and task-specific; it does not identify a depth-independent ceiling.

Zhu et al. vary virtual logical depth by reusing weights and report reasoning gains at nearly fixed parameter count in Beyond Parameters: Exploring Virtual Logic Depth for Scaling Laws[zhu2025virtualdepth]. Their results motivate a depth coordinate in a scaling study, while leaving knowledge capacity, optimization, and transfer dependent on the declared training and evaluation protocol.

Mozer et al. introduce inference-time recirculation that feeds latent states back through an off-the-shelf model and report task improvements in Recirculation[mozer2026recirculation]. The added serial prefill and adaptive tuning belong in the systems and inference records; the observation is not a proof of a new representational ceiling.

Wang et al. widen the feedback channel between decoding steps with latent feedback while retaining a Transformer and language-modeling interface in Full-bandwidth transformer[wang2026fullbandwidth]. Any reported token or accuracy savings are conditional on the scheduled multi-pass training and measured decoding cost.

Teoh et al. add next-latent prediction to next-token training and report compact predictive states without changing the Transformer interface in Next-Latent Prediction Transformers Learn Compact World Models[teoh2025nextlatent]. This belongs to the training intervention record, not to an architecture-only comparison.

Two runs can share parameter count while using different numbers of recurrent or latent-feedback passes. A matched study must therefore report the computation-axis record and cannot infer an architecture ceiling from parameters alone.

For a depth-reuse comparison, record shared weights, pass count, stopping rule, recurrent state, training objective, token and pass FLOPs, prefill and decode latency, memory, and evaluation seeds. A pass-count change is an intervention even when the parameter tensor is unchanged.

In a fixed-recipe arm, hold data, optimizer, feedback, stopping rule, and evaluation law fixed while varying only the declared depth-reuse intervention. The resulting difference is an estimand for that recipe, not a best-achievable comparison.

A depth-reuse study may fit a surface over parameters, tokens, passes, and cost. The fit is an empirical summary over its measured range; it does not establish an asymptotic law or a universal saturation point.

Weight sharing, latent objectives, and recirculation can alter optimization without changing the declared external interface. The comparison must retain architecture, training intervention, and systems coordinates separately.

The cited looped, virtual-depth, latent-feedback, and recirculation results do not show that more passes always improve capability, that parameter sharing dominates added parameters, or that inference-time gains transfer to training or to another task family. Each transfer requires a matched model-instance record.

The fixed-recipe frontier evaluates one common \(\eta \). The equal-tuning frontier permits architecture-specific choices from a predeclared trial set with the same tuning data and budget. The restricted envelope permits a predeclared class of interventions under \(C\). These are distinct estimands.

For a finite real target \(q\in \mathbb R\) and the frontier of Definition 3.1.3, define the inverse cost \[ C_A(q)=\inf \{C\in \mathbb R_{\geq 0}:V_A(C)\geq q\} \in [0,+\infty ], \qquad \inf \varnothing =+\infty . \] The iso-quality ratio is defined only on the domain \[ \rho _{A/B}(q)=\frac {C_A(q)}{C_B(q)}, \qquad 0\leq C_A(q)<+\infty , \quad 0<C_B(q)<+\infty . \] The architectures must use the same target, evaluation interface, cost units, and comparison arm. The ratio compares efficiency under these choices.

An inverse cost is a threshold infimum, not an executable minimum. The infimum over budgets may be unattained; even if a budget satisfies \(V_A(C)\geq q\), its performance supremum may be unattained at \(q\). An actual target-achieving intervention requires a separate witness.

Positive individual costs do not guarantee a positive inverse cost. For interventions \(\eta _n\), \(n\geq 1\), with score \(1\) and cost \(1/n\), the target \(q=1\) has \(C_A(1)=0\), although every intervention costs more than zero. If both architectures have this family, the putative ratio is \(0/0\) and is excluded by the displayed domain.

Record training FLOPs, inference FLOPs, wall time, memory, energy, and hardware separately as \(\mathbf c\). A scalar cost \(C=w\cdot \mathbf c\) is a declared study choice with nonnegative units \(w\); changing \(w\) changes the frontier and must not be hidden as architecture quality.

Consider a causal system whose state after each prefix has at most \(K\) distinct values, and a task with \(K+1\) prefixes requiring pairwise distinct continuation labels. By the pigeonhole principle two prefixes share a state, so at least one continuation label is wrong. This is a finite toy obstruction only; it does not bound a concrete Transformer or KDA instance without a proved reduction to this state model.

Let \(V_A\) be the nondecreasing frontier of Definition 3.1.3, bounded above by a finite \(U\in \mathbb R\). If an intervention reaches a real target \(q<U\) at a finite cost \(C_q\geq 0\), then at every finite \(C\geq C_q\), \(q\leq V_A(C)\leq U\). In particular \(V_A(C)\) is real and \(0\leq U-V_A(C)\leq U-q\). This elementary certificate is conditional on the bound \(U\); it does not assert that real intelligence saturates.

Two systems can have the same scalar \(C\) while differing in memory, latency, and training allocation. A cost-weight change can reverse their ordering without changing either system. Thus a one-factor comparison needs a fixed scalarization and a reported cost vector; no architecture effect follows from equal \(C\) alone.

A common-recipe arm fixes training data, optimizer family, schedule, post-training procedure and feedback, training and post-training budgets, inference budget, and evaluation protocol and draws. Interface-compatible parameter choices are declared in advance as part of each permitted architecture package. The comparison varies that package under the common recipe; any further deviation is recorded as a separate comparison coordinate. Its score difference measures performance under this recipe, not the best attainable result for either architecture.

An equal-tuning-budget arm permits architecture-specific tuning under a predeclared protocol that fixes the tuning data and its volume, trial count, selection rule, stopping rule, and tuning cost for both architectures. The cost uses the same declared accounting rule. The resulting comparison measures practical performance under this equal optimization effort; it does not determine best possible training or a representation-only limit.

A restricted-envelope arm declares an allowed intervention class \(\mathcal E_A\subseteq \mathfrak I_A\) for each architecture, specifying the permitted training, post-training, and inference procedures. The classes expose exclusions, search budgets, seeds, and stopping rules. Restrict the evaluation functional of Definition 3.1.3 to \(\mathcal E_A\) and compare the resulting suprema under the same finite cost cap \(C\geq 0\), common evaluation law, and common declared cost accounting.

The measurable, integrable evaluation domain and extended-real conventions of Definition 3.1.3 apply: an empty feasible class has value \(-\infty \), an unbounded-above feasible score set has value \(+\infty \), and a finite supremum need not be attained. The allowed class is part of the comparison, so its envelope is a restriction-dependent quantity, not an unrestricted architectural ceiling.

A model-instance record contains architecture/checkpoint identity, parameter count, data and post-training recipe, optimizer, precision, hardware, cache policy, training and inference cost vectors, task suite, and independent evaluation seed. A missing field makes an architecture attribution unknown.

FlashAttention, KV-cache layouts, quantization, batching, and kernels can change feasible cost. If they approximate, truncate, or alter precision of the mathematical computation, the record must mark that as a systems intervention rather than as a pure architecture comparison.

Given \(n\geq 1\) evaluation draws, let \(\widehat V_A(C)\) be the sample mean of finite real evaluation scores for architecture \(A\) at a finite cost cap \(C\geq 0\). Under a fixed evaluation law and common-recipe arm, the finite difference \(\widehat V_A(C)-\widehat V_B(C)\) is a well-defined empirical estimand.

It does not identify a causal architecture effect when data, tuning, or systems coordinates differ. The conclusion follows directly from the declared record.

A toy theorem applies to a concrete model only if an explicit map from its state and interface to the model preserves the theorem's assumptions, including its cost accounting. Without such a map, the conclusion concerns only the abstract model; its validity for the concrete system is unknown.

For a Kimi Delta Attention (KDA) versus full-attention baseline study, record the exact Kimi Linear checkpoint, layer mix, context length, hardware, kernel, precision, batch, and decoding workload. The paper describes KDA as a fine-grained gated delta-rule module in a hybrid architecture; those are source observations from Kimi Linear: An Expressive, Efficient Attention Architecture[kimi2025linear], not universal theorems.

If the baseline is called a Transformer, record whether it is the paper's MLA baseline or another full-attention implementation; the label alone does not identify a common architecture or cost.

A matched comparison uses the same task prompts, post-training data and feedback, optimizer family, evaluation protocol, and quality target. The corresponding measurements include parameter count, training FLOPs, inference FLOPs, wall time, memory, KV-cache bytes, and context length for each architecture. A reported speed or quality difference is conditional on these experimental conditions and measurements.

Kimi Linear reports a hybrid KDA/MLA model and fair-comparison experiments in Sections 3--5, with setup in Section 5.4 and efficiency comparisons in Sections 5.5--5.6 of Kimi Linear: An Expressive, Efficient Attention Architecture[kimi2025linear]. These include long-context efficiency.

The reported throughput and quality are empirical under its training recipe and hardware; they are lower bounds on demonstrated performance, not an architecture-independent ceiling.

Two models with equal parameter count can have different attention FLOPs, KV-cache memory, kernel utilization, and attainable context. Conversely, equal wall time can hide different hardware and precision. Therefore a parameter-matched result cannot by itself identify a compute frontier.

The KDA example does not prove that every linear-attention model beats every Transformer, that cache savings imply equal quality, or that a measured throughput gain is a capability gain. Any such statement requires a new matched study with the evidence record of Definition 3.4.4.

Call architecture A more cost-efficient than B at quality \(q\) only when the compared threshold costs use the same declared protocol, the confidence procedure and stopping rule are fixed, and all cost-vector coordinates needed by the claim are present. A ratio additionally requires finite costs and a strictly positive denominator as in Definition 3.3.2. State whether the costs are measured target-achieving witnesses or justified inverse-frontier values \(C_A(q)\) and \(C_B(q)\); observing one successful run does not identify those infima. Otherwise the comparison is descriptive or unknown.

A JEPA-like architecture is comparable to an autoregressive model only after fixing the predictor, target representation, decoder or downstream interface, training objective, inference procedure, and evaluation law. A parameter-count match alone is not a typed comparison.

A latent predictor can be excellent under one decoder and unusable under another. The evaluation record must therefore include the interface that maps the representation to the declared task output.

If two architecture records induce the same conditional output law on a fixed task and evaluation interface, every evaluation functional \(J\) gives the same value. This is an immediate pushforward identity under the stated output-law and interface assumptions. It does not imply that distinct architectures are generally equivalent.

Two architectures may tie on a finite benchmark while differing on an unmeasured task slice or longer context. Thus a single score cannot establish equal representational ceilings; the task family and evaluation coverage must be declared.

A result stated solely in terms of a model/interface map applies to every architecture realizing that map and satisfying its hypotheses. An architecture-specific conclusion can require further assumptions, as in Remark 3.1.4.

The relative capability and cost of KDA, full attention, and JEPA candidates remain empirical questions. A reproducible comparison requires identified checkpoints or reproducible training, fixed evaluation seeds, and measured cost vectors. Architecture-specific ceilings remain estimands and hypotheses rather than established constants.

The Kimi Linear report compares experimental models pretrained on 1.4 trillion tokens, each with 48 billion total and 3 billion active parameters. Kimi Linear uses three Kimi Delta Attention layers for each full Multi-head Latent Attention layer, with no positional encoding (NoPE) in its MLA layers; the reference uses full MLA attention [kimi2025linear, Sections 4 and 5.4]. These are properties of the reported hybrid architecture.

These experimental models are distinct from the released Kimi Linear model, pretrained on 5.7 trillion tokens and supporting up to one million tokens of context. Appendix D compares that release with Moonlight, which has 16 billion total and 3 billion active parameters [kimi2025linear, Section 5.4.1 and Appendix D].

For batch size one at one million tokens, Figure 7 labels a \(2.9\times \) prefill speedup and a \(2.2\times \) decoding speedup for Kimi Linear against MLA. Section 6.3 gives \(2.3\times \) for the latter point, so the graphic and prose differ slightly [kimi2025linear, Figure 7 and Sections 5.6 and 6.3].

The larger-batch comparison in Figure 1(b) gives decoding time per output token of 1.84 milliseconds for Kimi Linear and 11.48 milliseconds for MLA, with a reported \(6.3\times \) speedup. Section 6.3 describes this as a theoretical speedup from reallocating saved KV-cache memory to larger batches. It illustrates a different use of the memory saving from the batch-one latency result [kimi2025linear, Figure 1(b) and Section 6.3].

Figure 1(a) reports RULER scores of 84.3 for Kimi Linear and 81.3 for MLA at 128k context, with \(3.98\times \) decoding acceleration [kimi2025linear, Figure 1(a)]. The report also gives a reduction of up to 75 percent in KV-cache use [kimi2025linear, Abstract]. Together these observations motivate studying context length and batch size as separate cost coordinates. The unequal RULER scores and different timing regimes are retained when interpreting those points.

For a reported benchmark score q, use the same task, prompt law, scoring rule, and post-training condition for both systems. The comparison records the smallest measured cost at which each system reaches q, when such a measurement exists. A throughput or cache ratio is not substituted for this iso-quality cost.

The cited FLA KDA implementation at commit 6b6f546f stores one active recurrent state per sequence, with dimensions determined by the value heads and key/value dimensions rather than accumulated context length. Its newly allocated state uses float32 [⧉]. The optional short-convolution state has one fixed-width kernel buffer per channel and sequence [⧉]. This supplies a concrete implementation model for the recurrent part of the 3:1 mixture in Example 3.7.1.

Consider a stylized comparison with \(4\ell \) layers, batch size \(b\), and context length \(T\), where \(\ell ,b,T\) are positive integers. Assume each MLA layer stores \(\kappa T\) bytes per sequence, with the same \(\kappa >0\) in both designs, and each KDA layer stores one terminal recurrent-plus-convolution state of \(\sigma >0\) bytes. The active attention-state memories are

The hybrid uses less active state exactly when \(T>\frac {\sigma }{\kappa }\), and the ratio tends to \(1/4\) as context grows. This explains why replacing three quarters of the growing caches can approach a 75-percent saving, while fixed recurrent-state overhead can erase the advantage at short contexts. The retained MLA layers keep the hybrid's state memory linear in \(T\); the whole model is not a fixed-memory recurrent system.

Weights, training activations, workspaces, allocator padding and saved prefix states are outside this active-state calculation. At a fixed device memory budget, the estimate predicts more room for simultaneous or longer rollouts when attention state dominates. It does not predict a fourfold speedup. Training and decoding also use different execution paths: FLA selects chunk kernels for gradient-enabled computation and fused recurrence for short inference calls [⧉]. The fixed-state estimate is therefore a useful serving hypothesis, not a training-memory or training-speed formula.

The report holds its stated pretraining and SFT recipes common across the experimental models [kimi2025linear, Sections 5.4--5.5 and Table 4]. The selected observations in Example 3.8.8 compare the resulting trained designs, including the hybrid's 3:1 KDA/MLA pattern and NoPE choice. The mixed score directions suggest a task-dependent tradeoff rather than one ordering of the two architectures.

The theoretical mechanism is a change in the cost of maintaining and using the prefix: recurrent layers compress it into state, while the retained MLA layers preserve growing attention caches. The model in Example 3.7.4 predicts the largest memory advantage at long contexts. If this advantage reduces the cost of generating useful candidates, Example 3.7.9 predicts when more attempts can offset a reduction in per-attempt quality. This is a conditional explanation to test through rollout costs and verified success rates; it does not require attributing the whole result to KDA alone.

Let \(c_K(q)\) and \(c_T(q)\) be measured inference costs for Kimi Linear and the Transformer reference at the same finite real score \(q\) and context length, under one accounting rule. If \(0\leq c_K(q)<+\infty \) and \(0<c_T(q)<+\infty \), the measured ratio \(\widehat \rho _{K/T}(q)=c_K(q)/c_T(q)\) is reported together with the full cost vectors. These measured costs need not be the inverse frontier infima of Definition 3.3.2. If either threshold is unmeasured or the denominator is zero, the ratio is left undefined rather than inferred from the paper's speedup plot.

Kimi Linear combines KDA with periodic full-attention layers. A measured advantage therefore identifies the reported hybrid intervention, not a claim that a purely linear recurrent architecture has the same quality or ceiling. The layer pattern belongs in the architecture record.

The published benchmark scores demonstrate attainable points for the reported checkpoints and protocol. They do not establish a representational ceiling, a universal scaling law, or a claim that additional cost cannot improve either architecture.

Smaller caches can support more concurrent requests or avoid preemption and recomputation. These are actual serving mechanisms in vLLM's memory and scheduling guidance [⧉]. For post-training, the useful prediction is therefore about completed candidate attempts under a budget, including verification and optimization costs, rather than decoding speed alone.

Use the following cost model on the same hardware. One baseline attempt costs \(c>0\); generation accounts for a fraction \(f\in [0,1]\). Suppose generation becomes \(s_{\rm gen}>0\) times as fast while all other cost per attempt stays fixed. The new cost and total attempt-throughput factor are

For a fixed budget \(B\geq 0\), this model permits \(N_M=\lfloor B/c\rfloor \) baseline attempts and \(N_K=\lfloor B/c_K\rfloor \) hybrid attempts. Assume \(N_M,N_K\geq 1\) and verified-success events independent within each set of attempts, with respective probabilities \(p_M,p_K\in [0,1]\). Then Theorem [ftip-0078] gives discovery probabilities \(1-(1-p_M)^{N_M}\) and \(1-(1-p_K)^{N_K}\). The hybrid matches or exceeds baseline discovery exactly when

This follows by comparing the two failure probabilities and taking the nonnegative \(N_K\)-th root.

For an illustrative estimate, suppose generation consumes 80 percent of the baseline cost and use \(s_{\rm gen}=2.2\), motivated by the batch-one decode figure in Example 3.7.2. Applying that decode factor to the whole generation stage is a working approximation, not a measured rollout result. Then \(S\approx 1.774\); a budget of \(B=10c\) buys 10 baseline attempts or 17 hybrid attempts. If \(p_M=0.10\) and \(p_K=0.08\), the discovery probabilities are approximately \(0.651\) and \(0.758\). The break-even hybrid probability is about \(0.0601\). Thus this model predicts that more attempts can outweigh a moderate per-attempt quality loss. The assumed probabilities are illustrative verified-success rates, not conversions of the benchmark scores in Example 3.8.8.

The prediction is strongest for long-context, generation-heavy work whose saved memory becomes usable rollout capacity. As verification or optimization dominates, \(f\) shrinks and \(S\) approaches one. Correlated attempts require the joint-law analysis of Convention [ftip-0077] instead of the independent product. Measure completed attempts, verified-success rates and total cost under a fixed prompt, decoding and verifier protocol to test the tradeoff. In the RLVR round of Example [ftip-004M], additional successful candidates can improve the available feedback; whether they produce useful accepted updates is a further optimizer-and-evaluation question, as Example [ftip-0085] demonstrates.

A paired comparison identifies the two systems, exact checkpoint or training commit, architecture variant, parameter counts, tokenizer, and context limit. A label such as ``Transformer'' is insufficient when attention, cache, or layer patterns differ.

Both systems use the same task family, prompt and data law, scoring rule, sampling seeds, stopping rule, and evaluator version. Any exception is a separate comparison arm, not an unrecorded architecture effect.

Record data volume, token order policy, optimizer and schedule, supervised tuning, feedback or reinforcement procedure, update count, and selection rule. The common-recipe arm fixes these coordinates; the equal-tuning arm allows predeclared alternatives with equal search resources.

Record hardware, software and kernel versions, precision, batch, cache policy, sequence lengths, warm-up procedure, concurrency, and decoding settings. A kernel or cache change that alters numerical computation is marked as an intervention rather than hidden as an implementation detail.

For each system, the cost vector \(\mathbf c\) contains training FLOPs, inference FLOPs, wall time, memory, energy when available, and context length. The primary scalar \(C\) and its weights are declared beside the vector; omitted coordinates are marked unknown.

For each fixed workload and system, record warm-up runs, repetition count, random seeds, and the aggregation rule. Report mean and dispersion for time, memory, and score; a single fastest run cannot define a cost frontier.

When the same evaluation items are used, report paired score differences and an uncertainty procedure fixed before inspecting the result. Confidence intervals quantify sampling variation; they do not repair unmatched training, systems, or task coordinates.

Table 4 of the Kimi Linear report compares the experimental models pretrained on 1.4 trillion tokens in Example 3.7.1, after the same supervised fine-tuning (SFT) recipe. Both have 48 billion total and 3 billion active parameters. Three selected benchmark scores are [kimi2025linear, Sections 4 and 5.4 and Table 4]:

The entries use the paper's score units, with larger values better on each selected benchmark; LiveBench is reported as Pass@1. Kimi Linear is higher on MMLU-Pro and lower on LiveBench and EvalPlus in these observations. These are reported point estimates on different tasks, not a single capability score; the table supplies no uncertainty estimate.

The common recipe uses the K2 pretraining corpus, MuonClip and a shared training schedule. SFT proceeds from broad instruction data to targeted reasoning tasks; evaluation uses temperature 1.0 and an internal framework derived from LM-Harness [kimi2025linear, Section 5.4]. Thus the scores provide a concrete example of how the trained hybrid and full-MLA designs respond to the same recipe. The cost mechanism and its possible post-training consequences are developed in Example 3.7.5.

A useful cost model can combine measurements with assumptions justified by the architecture and its implementation. In Example 3.7.9, the generation share \(f\) and speed factor \(s_{\rm gen}\) determine a predicted attempt-throughput factor. Their assumed values can be varied without pretending they were measured in the original experiment.

For fixed \(f\in [0,1]\), write \(S(s)=1/(1-f+f/s)\) for \(s>0\). This function is nondecreasing. An interval \(s_{\rm gen}\in [s_-,s_+]\), with \(0<s_-\leq s_+\), therefore gives the conditional range \(S\in [S(s_-),S(s_+)]\). This is sensitivity to a chosen assumption range; it is a statistical confidence interval only if a sampling argument supplies that interpretation.

A measured paired contrast evaluates the same declared quantity for two observed model instances under a specified task, protocol and cost accounting. The common-recipe scores in Example 3.8.8 are one example. A confirmatory comparison fixes its matching and selection rules before examining the scores.

A conditional paired prediction instead uses an explicit model to supply one or more cost or response quantities, then derives the comparison under those assumptions. The discovery probabilities in Example 3.7.9 are an example. Such a prediction can be motivated by existing observations and guide a later experiment; its unmeasured inputs remain assumptions, and agreement with new matched measurements tests the model.

Keep the illustrative generation factor \(s_{\rm gen}=2.2\) from Example 3.7.9 and vary the baseline generation share \(f\). Its cost model gives the following total attempt-throughput factors:

These calculated values show why the same decoding improvement can matter much more for generation-heavy search than for a verifier- or optimizer-dominated workload. They suggest measuring the time spent in generation, verification and updates before selecting where to invest systems effort. A change in the bottleneck changes the useful intervention, even when the attention architecture stays fixed.

The memory and discovery models predict the strongest benefit when long prefixes make attention state expensive, saved capacity improves useful rollout throughput, and per-attempt success remains above the break-even threshold. Short contexts, expensive verification, or strongly correlated attempts can weaken that benefit. These are distinct mechanisms to test, rather than a universal ranking of KDA and full attention.

A comparison across implementations can retain this reasoning while re-estimating state sizes, generation share and success probabilities for the new system. Report both the prediction and the matched measurements: their agreement or disagreement identifies which mechanism or assumption needs revision.

Choose a finite ordered grid \(0<C_1<\cdots <C_m\) and evaluate each architecture at every declared grid point under the same task and protocol. The resulting values \(\widehat V_A(C_i)\) are observations of the restricted frontier, not its values between grid points.

For an observed score \(s_A(C_i)\) define the discrete upper envelope \(U_A(C_i)=\max _{j\leq i}s_A(C_j)\). It is a descriptive monotone summary of the measured points; it is not evidence that unmeasured costs attain the envelope or that extra cost cannot reduce score.

Attach a predeclared uncertainty interval \(I_A(C_i)\) to each score and carry those intervals through score differences and threshold crossings. A finite band describes sampling and measurement variation; it does not cover unmeasured training procedures or architectures.

Fix a quality target \(q\in \mathbb R\), tolerance and cost increment \(\varepsilon ,\delta \in (0,+\infty )\), and a budget \(C\in \mathbb R_{\geq 0}\). Require both \(V_A(C)\) and \(V_A(C+\delta )\) to be finite real values. A study calls an architecture \(\varepsilon \)-saturated at \(C\) only relative to its allowed intervention class when the restricted frontier has certified gain \(V_A(C+\delta )-V_A(C)\leq \varepsilon \).

Take finite real \(U\), \(C\geq 0\), \(\delta \geq 0\), and \(\varepsilon \geq 0\), with the nondecreasing frontier of Definition 3.1.3. If a justified upper bound gives \(V_A(C+\delta )\leq U\) and a justified lower bound gives \(V_A(C)\geq U-\varepsilon \), then \(U-\varepsilon \leq V_A(C)\leq V_A(C+\delta )\leq U\). Both endpoint values are therefore finite real, and \(0\leq V_A(C+\delta )-V_A(C)\leq \varepsilon \). This is a conditional bound for the named frontier; it proves no universal saturation of intelligence.

For the finite controller class with hard resource admission, Corollary [ftip-00MC] obtains the required bounds from a feasible controller and a uniform Bellman certificate.

Let \(S\subset \mathbb R_{\geq 0}\) be a finite set of measured costs, and choose \(C_0>0\) larger than every element of \(S\). Suppose the exact frontier value observed at each cost in \(S\) is zero. For finite budgets \(C\geq 0\), suppose the declared class of possible frontiers permits both \[ V_0(C)=0, \qquad V_1(C)= \begin {cases} 0,&0\leq C<C_0,\\ 1,&C\geq C_0. \end {cases} \] These nondecreasing frontiers with scores in \([0,1]\) agree on every observed cost and differ at \(C_0\). The observations alone do not distinguish them.

Both possibilities have finite realizations in the framework of Definition 3.1.3: allow two interventions with costs \(0\) and \(C_0\). Give the first score zero and the second score \(\theta \in \{0,1\}\). For example, on a single deterministic evaluation task with utility equal to the output bit, let the two resulting protocols return \(0\) and \(\theta \). The two possible choices of \(\theta \) give \(V_0\) and \(V_1\), respectively, under the same intervention and cost specification. Every score is finite, and the feasible score maximum is attained at each budget.

This example does not supply a strictly better agreeing frontier for every possible data set or every admissible class. If a proved global score bound is \(1\) and an intervention attains it at a finite cost \(C_*\geq 0\), monotonicity forces \(V_A(C)=1\) for every \(C\geq C_*\); no higher value is admissible. A singleton class of possible frontiers can also identify the frontier without such an alternative. A conditional certificate such as Theorem 3.9.5 therefore requires its stated upper-bound evidence; that evidence does not follow merely from the absence of an observed improvement.

The primary cost \(C\) is a declared scalarization of the recorded cost vector. A frontier statement is conditional on its units and weights; the same paired measurements may produce a different ordering under a different scalarization.

Two architectures may alternate in the observed ordering across costs: one can score higher at small \(C\) while the other catches up at larger \(C\). Thus a single comparison point cannot establish a global ordering or a common ceiling.

A Kimi Delta Attention versus Transformer study should publish the cost grid, paired scores, uncertainty intervals, scalarization weights, and the exact identities and configurations of the compared models. The report may then state which measured points are Pareto or iso-quality comparisons under that protocol.

The conditional frontier is indexed by architecture, intervention class, task family, evaluation law, and cost scalarization. Changing any of these coordinates creates a new estimand; no ordering transfers automatically to a new checkpoint, optimizer, hardware stack, or task family.

A model-instance record names the architecture, parameterization or checkpoint, training and post-training recipe, optimizer, execution stack, inference budget, task family, evaluation law, and recorded cost vector. Two records are comparable only after the fields held fixed and the fields allowed to vary are declared.

A comparison unit is a pair of model-instance records evaluated on the same task family and evaluation law, together with a declared cost scalarization. A Transformer reference and a Kimi Delta Attention instance may form such a pair only when the remaining coordinates are documented.

For a pair of matched model-instance records, the common-recipe arm measures how the permitted architecture packages respond to one recipe. A score advantage in this comparison need not persist after either package is retuned.

Under the equal-tuning-budget arm, the two selected model instances may use different recipes. That difference is compatible with equal predeclared tuning effort; forcing the selected recipes to match would answer a different comparison question.

In the restricted-envelope arm, enlarging an allowed intervention class at fixed evaluation law, cost accounting, and budget can raise its envelope and cannot lower it. A finite search over feasible interventions supplies a lower bound from the scores it attains. It certifies the supremum only if it covers every feasible intervention or is accompanied by a matching upper-bound argument.

Changing the optimizer, post-training method, hardware or kernel, precision, context policy, or evaluation law changes a comparison coordinate. A score difference after such a change belongs to the new intervention, unless the study explicitly estimates the interaction.

A transfer matrix records which interfaces, intervention arms, and budgets permit a comparison. Each row is a study record: its source, estimand, matched controls, evaluation coordinates, and declared scope boundary are explicit. A blank or incompatible cell is an incomparability finding, not a missing score to be imputed. The vector and evaluation coordinates are reported together under the conditional frontier of Definition 3.1.6.

A completed row must report its paired observations through § 3.8 and the common-recipe, equal-tuning, and restricted-envelope arms of § 3.4. Until those records are supplied, its status remains open or incompatible.

A toy representation or recurrence result may motivate a hypothesis, but it is not transferred to a model instance until its interface, task, cost accounting, numerical regime, and evaluation protocol are instantiated and checked. If any required assumption is unverified, applicability to the model instance remains unestablished.

A completed comparison reports the tested cost grid, excluded configurations, stopping rule, uncertainty, and negative results. Stopping without a detected difference is evidence about the declared study, not proof that the architectures have equal potential.

The matrix in Definition 3.10.7 is a protocol map, not a result table. The following probes keep the five capability coordinates separate and leave an unknown cell visible until the required records exist. Each probe names its source, estimand, controls, and evaluation coordinates so that a missing record cannot be mistaken for a zero score.

1. State access. Sources: Mamba, Sections 3.1--3.3 and linear attention, Sections 3.2--3.3. Estimand: the paired difference in retrieval utility and \(K_{\rm state}\). Controls: Transformer, Mamba, and linear-attention systems on a predeclared retrieval and continuation suite at matched parameter, context, recipe, and cost budgets. Evaluation: report cost, recurrence or summary stability, and state observability. Scope boundary: a long-context score does not transfer to another task family without its task and transport record.
2. Adaptive computation. Source: Continuous Thought Machine, Sections 3.1--3.5 and the looped-model contract in § 3.2. Estimand: the paired change in utility and \(K_{\rm adapt}\) per charged internal tick or loop. Controls: common input, base model, recipe, stopping rule, and charged internal compute. Evaluation: report tick/loop cost, training and inference stability, and stopping-signal monitorability. Scope boundary: an observed adaptive gain remains conditional on that stopping law.
3. Representation and abstraction. Sources: I-JEPA, Section 3 and SplitJEPA, Sections 3.1--3.4. Estimand: subspace identifiability and the conditional change in \(K_{\rm repr}\). Controls: fixed correspondences, full-rank variant differences, normalization, encoder budget, and matched downstream probes. Evaluation: report probe cost, optimization stability, and subspace monitorability. Scope boundary: the conditional subspace result is separate from downstream task transfer; failure of a premise is an incompatible cell, not a zero score.
4. Plasticity. Source: the objective and optimizer interfaces in Definition 3.1.6 and the I-JEPA intervention description in § 2. Estimand: the paired change in \(K_{\rm plast}\) after a declared post-training intervention. Controls: common recipe and equal-tuning arms with data, optimizer, trial count, stopping, and feedback fixed or explicitly varied. Evaluation: report intervention cost, training stability, and update-signal monitorability. Scope boundary: an intervention effect is limited to the declared base artifact, task, and objective; it does not transfer across architectures without a new paired record.
5. Portability. Sources: the Mamba, BDH, and Neural Operator method descriptions in § 2: Mamba, BDH, and Neural Operators. Estimand: survival of the declared utility and \(K_{\rm port}\) across one transfer edge. Controls: source model, task family, model scale, deployment regime, evaluation law, and cost vector retained as paired fields. Evaluation: report migration cost, stability after the shift, and failure monitorability. Scope boundary: evidence for one transfer edge does not order unmeasured edges; blank edges remain unmeasured.
6. Adjunct interactions. Sources: liquid time-constant networks, Sections 1--2, Neural Operators, Section 3, and I-JEPA, Section 3. Estimand: the joint change in the declared capability coordinates after an explicit intervention. Controls: declared core model, objective, temporal law, function-space bias, matched controls, and separate single-intervention arms. Evaluation: report joint and marginal costs, numerical/training stability, and interaction monitorability. Scope boundary: a joint effect is limited to the declared core, task, temporal law, and function-space regime; it does not transfer across regimes without separate intervention records.

These probes do not establish a common ordering of architecture families. They specify what would count as evidence for a declared bottleneck change, while cost, numerical stability, monitorability, and transfer remain separate evaluation coordinates. Paired observations should use § 3.8, the study arms of § 3.10, and the capability vector of Definition 3.1.6.

Fix integers \(d_1,d_2,k\geq 1\). Let \(U\in \mathbb R^{d_1\times k}\) and \(V\in \mathbb R^{d_2\times k}\) be factors, let

and let \(f:\mathbb R^{d_1\times d_2}\to \mathbb R\) be differentiable. The corresponding factored objective is \(L(U,V)=f(W(U,V))\). The pair \((U,V)\) is a parameterization; \(W(U,V)\) is the represented object on which \(f\) depends. This is the rectangular form noted in Section 3 of The loss does not see the basis, but Adam does[singh2026lossbasis].

For \(A\in \mathrm {GL}(k)\), define the general linear gauge action by

The inverse transpose on the second factor is essential for a general invertible \(A\). When \(A=Q\in \mathrm O(k)\), one has \(Q^{-\mathsf T}=Q\), and the action becomes \((UQ,VQ)\).

For every \(A\in \mathrm {GL}(k)\),

This is the factored-model calculation in Section 3 of The loss does not see the basis, but Adam does[singh2026lossbasis].

The orthogonal gauge orbit of \((U,V)\) is

Every pair in the orbit represents the same \(W\). This orbit is generally smaller than the complete fibre of the map \((U,V)\mapsto UV^{\mathsf T}\). The restriction to \(\mathrm O(k)\) is geometric: it preserves the Euclidean metric on factor space.

The action \(\gamma _A\) preserves \(\lVert U\rVert _F^2+\lVert V\rVert _F^2\) for every pair \((U,V)\) if and only if \(A\in \mathrm O(k)\).

See Lemma 3.2 and its proof in Appendix B.1 of The loss does not see the basis, but Adam does[singh2026lossbasis].

For \(Q\in \mathrm O(k)\),

See Lemma 4.1 and its proof in Appendix B.2 of The loss does not see the basis, but Adam does[singh2026lossbasis].

Use the optimizer-state convention of Definition [ftip-001H]. For every \(Q\in \mathrm O(k)\), an optimizer state action is a map \(\sigma _Q:\mathcal S_{\rm opt}\to \mathcal S_{\rm opt}\) compatible with the shapes of the factor states and satisfying \(\sigma _Q(s_0)=s_0\) for the declared initial state.

For example, a momentum buffer shaped like \(U\) transforms by right multiplication by \(Q\); a shared scalar second moment is fixed. An entrywise second-moment array need not admit a state action compatible with every orthogonal \(Q\).

Let \(\mathcal A\) be a deterministic instance of the parameter-update rule of Definition [ftip-001I], acting on \((U,V,s)\). It is gauge-equivariant when for every \(Q\in \mathrm O(k)\) there is a state action \(\sigma _Q\) such that gauge-related initial states produce

This is Definition 3.1 of The loss does not see the basis, but Adam does[singh2026lossbasis]. Any randomness, schedules, stopping rule, and mixed update blocks must also be coupled as required by Remark [ftip-001J]; the definition does not hide them.

One update of a gauge-equivariant optimizer makes the following square commute. The vertical arrows change factor basis and the horizontal arrows apply the same declared update.

Commutation is stronger than equality of the scalar losses before the step. It states equality of the represented product trajectory after aligning the parameter bases.

The full \(\mathrm {GL}(k)\) action in Definition 4.1.2 preserves \(W\), but ordinary Euclidean gradients are covariant under the orthogonal specialization used in Lemma 4.1.6. A theorem for \((UQ,VQ)\) must therefore not be silently promoted to every product-preserving reparameterization.

The source calls the orthogonal subgroup the gauge in its optimizer classification while also noting the larger function-preserving group. These are compatible statements only when their different geometric scopes remain explicit.

A memoryless factor-update map is a function \(\Phi :\mathbb R^{n\times k}\to \mathbb R^{n\times k}\) applied to a current gradient \(G\) without a persistent optimizer state. It is right-equivariant when

This local definition is the input of Theorem 4.5 in The loss does not see the basis, but Adam does[singh2026lossbasis]. It does not classify stateful optimizers such as Adam.

Gradient descent and Polyak or Nesterov momentum are gauge-equivariant when their factor gradients, momentum buffers, schedules, and look-ahead points are transformed consistently.

Proposition 4.2(1) and its proof are in Appendix B.3 of The loss does not see the basis, but Adam does[singh2026lossbasis].

Replace Adam's entrywise second-moment denominator by one scalar whose updates depend only on gauge-invariant quantities such as the pooled mean of the squared entries of both factor gradients. With a covariant first-moment buffer, the resulting stateful update is gauge-equivariant.

Proposition 4.2(2) and its proof are in Appendix B.3 of The loss does not see the basis, but Adam does[singh2026lossbasis]. The result concerns this declared shared-scalar variant, not stock Adam.

Fix one factor, write \(n\in \{d_1,d_2\}\) for its row dimension, and let \(G_t,M_t\in \mathbb R^{n\times k}\) be its gradient and Muon momentum buffer. For \(0\leq \beta <1\), \(M_{-1}=0\), and the source's linear momentum update \(M_t=\beta M_{t-1}+G_t\), Muon's direction is \(\Delta _t=\operatorname {msign}(M_t)\). Under real arithmetic this stateful update and the source's damped Shampoo update are right-equivariant. The Muon claim includes the stated finite Newton--Schulz approximations; the Shampoo claim uses its complete left and right accumulators.

Proposition 4.2(3)--(4) and its proof are in Appendix B.3 of The loss does not see the basis, but Adam does[singh2026lossbasis]. Variable splitting, coordinatewise clipping, mixed update rules, finite precision, and different damping conventions lie outside this statement. The Muon part is stateful; it does not replace the momentum buffer by the current gradient or invoke the memoryless classification of Definition 4.2.1.

Let \(k\geq 2\) and let a memoryless update have the fixed entrywise form \(\Phi (G)_{ij}=\phi (G_{ij})\). If \(\Phi (GQ)=\Phi (G)Q\) for every \(G\) and \(Q\in \mathrm O(k)\), then \(\phi (x)=cx\) for some \(c\in \mathbb R\).

Proposition 4.3 and its proof are in Appendix B.4 of The loss does not see the basis, but Adam does[singh2026lossbasis]. No continuity assumption is needed.

A nonlinear fixed entrywise update applied at a zero optimizer state is not gauge-equivariant when \(k\geq 2\). In particular, the source applies Proposition 4.2.5 to the first-step maps of Adam, RMSProp, signSGD, and Lion; it checks Adafactor separately because its factored statistics are not an entrywise map.

This is a first-step obstruction. It suffices to disprove equivariance of the full run, but it does not classify every later state or say that such an optimizer cannot reach a particular solution.

Fix \(\epsilon >0\). For zero-state, bias-corrected Adam define the entrywise first-step direction

and, for \(Q\in \mathrm O(k)\), define its gauge-aligned defect by

Here \(|G|\), addition, and division are entrywise. Bias correction makes the first direction independent of Adam's two decay coefficients under the declared convention [singh2026lossbasis, Proposition 4.4].

Let \(G_U=\nabla _UL(U_0,V_0)\), \(G_V=\nabla _VL(U_0,V_0)\), \(D_U=D_\epsilon (G_U)\), and \(D_V=D_\epsilon (G_V)\). Let \(W_1\) be the product after one Adam step from \((U_0,V_0)\) and \(\widetilde W_1\) the product after one step from \((U_0Q,V_0Q)\). Then

Proposition 4.4 and its proof are in Appendix B.5 of The loss does not see the basis, but Adam does[singh2026lossbasis].

Take \(d_1=d_2=1\), \(k=2\), \(f(w)=w^2/2\), \(U_0=V_0=(1,0)\), and

The original and rotated products after one zero-state Adam step are

They differ when \(0<\eta <2^{-1/2}+\epsilon \). This is the explicit witness in Proposition 4.4 of The loss does not see the basis, but Adam does[singh2026lossbasis]; it is a statement about one factored quadratic, not about language-model performance.

For \(G=(1,1)\) and the 45-degree rotation matrix \(Q\) of Example 4.2.9, one has \(GQ=(0,\sqrt 2)\). With \(\operatorname {sign}(0)=0\),

Thus even this two-coordinate update violates the commuting equation in Definition 4.2.1. The calculation is the explicit Appendix B.4 witness of The loss does not see the basis, but Adam does[singh2026lossbasis].

Let \(k\leq n\), and let \(\Phi \) be defined on full-column-rank matrices \(G\in \mathbb R^{n\times k}\). Then \(\Phi (GQ)=\Phi (G)Q\) for every \(Q\in \mathrm O(k)\) if and only if there is a matrix-valued function \(H\) of \(GG^{\mathsf T}\) such that

See Theorem 4.5 and its proof in Appendix B.6 of The loss does not see the basis, but Adam does[singh2026lossbasis]. It classifies the full-rank stratum only.

In the specialization \(k=n\) with \(G\) invertible, let \(P^{1/2}\) denote the unique positive-definite square root of \(P\). The function in Theorem 4.2.11 has the unique canonical representative

For rectangular \(G\), the action of \(H(P)\) away from the range needed to multiply \(G\) is not determined by \(\Phi \). The proof above establishes uniqueness in the square invertible specialization.

In the ambient matrix space, full-column-rank matrices form an open dense subset whose complement has Lebesgue measure zero. A probability-one claim requires the random gradient law to be absolutely continuous and not confined to a lower-rank set; random initialization alone does not supply that premise.

Theorem 4.2.11 therefore makes no classification claim on a general rank-deficient stratum. Equivariance still constrains values within each orthogonal orbit, and it forces \(\Phi (0)=0\), but it does not determine the rule there by continuity unless continuity is separately assumed. The source's generic-rank discussion therefore does not establish a classification at rank deficiency.

Let \(\theta =(U,V)\) and let \(a(t)>0\) be measurable with \(1/a\) locally integrable. A common-scalar preconditioned flow is a locally absolutely continuous trajectory satisfying

for almost every \(t\). The same scalar multiplies every coordinate of both factor gradients. For gauge-uniform conclusions, the source additionally requires \(a(t)\) to be determined by gauge-invariant statistics of the trajectory up to time \(t\).

For the flow of Definition 4.3.1, define the effective optimizer clock

The clock is strictly increasing on its domain. If \(\tau _{\max }=\infty \), it reaches every gradient-flow time; otherwise it traverses only the prefix with effective time below \(\tau _{\max }\).

Let \(\theta (t)\) satisfy Definition 4.3.1, let \(t(\tau )\) be the inverse of the clock in Definition 4.3.2, and put \(\widetilde \theta (\tau )=\theta (t(\tau ))\). Then, for almost every \(\tau \),

See Theorem 4.6 and its proof in Appendix B.9 of The loss does not see the basis, but Adam does[singh2026lossbasis]. Gauge invariance of \(a\) is not needed for this single-trajectory clock identity; it makes the clock common across a gauge orbit.

Every property depending only on the portion of the gradient-flow path traversed before \(\tau _{\max }\) transfers to the common-scalar flow. If \(\tau _{\max }=\infty \) and gradient flow converges, both flows have the same limit point. A sufficient condition for clock divergence is an eventual finite upper bound on \(a(t)\).

For shallow factorization, the source cites Implicit regularization in matrix factorization[gunasekar2017implicit]. For deep factorization, it cites Implicit regularization in deep matrix factorization[arora2019implicit]. For greedy low-rank dynamics, it cites Towards resolving the implicit bias of gradient descent for matrix factorization: Greedy low-rank learning[li2021greedy].

Those conclusions transfer only when both this clock theorem and every hypothesis of the original result hold. This corollary supplies no missing matrix-sensing assumption.

Two flows can traverse the same curve with arbitrarily different physical clocks. A convergence rate, finite-step budget, or hitting time stated in \(t\) therefore does not transfer through Theorem 4.3.3 without quantitative bounds on \(a\) and its integral. If \(\tau _{\max }<\infty \), even the tail and limit of the gradient-flow path are not reached.

The shared-scalar Adam variant in Proposition 4.2.3 retains a first-moment exponential moving average. It is stateful and is not the memoryless flow in Definition 4.3.1. Remark 4.7 of The loss does not see the basis, but Adam does[singh2026lossbasis] treats its agreement with gradient flow as empirical rather than as a consequence of Theorem 4.6.

The theorem likewise does not apply to discrete Adam, PPO, RLVR update loops, clipping, weight decay, and mixed optimizers. Gauge equivariance and time reparameterization are separate properties.

The source records controls on both sides of the proposed implication. Its ScaledGD-inspired control is equivariant but equalizes the spectral schedule and does not recover the planted low-rank target at the declared budget. A long-anneal sign update is non-equivariant but eventually reaches a low-recovery-error regime.

The controls are reported in Remark 4.7 and Appendix D.10 of The loss does not see the basis, but Adam does[singh2026lossbasis]. The source reports no tuned reproducibility baseline for the ScaledGD control. The diagram is not a convergence theorem.

Two factor pairs \((U,V)\) and \((\widetilde U,\widetilde V)\) are objective-equivalent for \(L=f\circ W\) when

They then have the same scalar loss for every objective depending only on that represented matrix. Orthogonal gauge-related pairs are objective-equivalent by Lemma 4.1.3; objective equivalence need not imply membership in the same orthogonal orbit.

Fix a factorized objective, an update algorithm, hyperparameter schedule, randomness coupling, stopping rule, and initial optimizer state. The resulting training intervention is parameterization-stable on an orthogonal orbit when every two gauge-related initial states have identical represented trajectories:

This is a local stability property of a fully declared intervention. It does not say that arbitrary objective-equivalent pairs outside the orbit must agree.

Every gauge-equivariant optimizer is parameterization-stable on each orthogonal orbit, provided the remaining protocol coordinates are coupled as in Definition 4.4.2.

This is a sufficient condition on a declared orbit, not a claim that equivariance is necessary for every possible equality of represented trajectories.

Suppose two orthogonally gauge-related initializations have equal represented object and scalar objective, but a declared update produces different represented objects after one step. Then the training intervention is not determined by the represented initialization and scalar objective alone.

Proposition Proposition 4.2.8 and Example 4.2.9 instantiate its premise for one factored quadratic and zero-state Adam. It proves underspecification, not a capability difference.

For the row-vector attention weights of Definition 1.5, let \(W^Q,W^K\in \mathbb R^{d\times d_h}\) and \(Q_h=XW^Q\), \(K_h=XW^K\). Attention logits depend on these weights through

For \(A\in \mathrm O(d_h)\), the right action \((W^Q,W^K)\mapsto (W^QA,W^KA)\) preserves the middle product.

Section 6 of The loss does not see the basis, but Adam does[singh2026lossbasis] uses column-vector weights and the invariant \(W_Q^{\mathsf T}W_K\). Transposition gives the row-vector invariant \(W^Q(W^K)^{\mathsf T}\), with the action on the right as displayed.

Section 6 and Table 3 of The loss does not see the basis, but Adam does[singh2026lossbasis] compare two function-equivalent initializations of a two-layer, four-head transformer on modular addition. The reported relative logit distance after one Adam step is \(3.6\times 10^{-3}\), versus \(2.9\times 10^{-7}\) for a same-basis noise twin; the final per-head invariant distance is reported as 56 percent.

The exact first-step defect has an algebraic explanation, but later drift, scale robustness, and downstream behavior are empirical. The tested small transformers, precision, optimizer allocation, seeds, and task do not establish a frontier-wide law.

A loss-only description collapses a gauge orbit to one represented starting point. A basis-sensitive optimizer can split that quotient back into different represented trajectories.

The right-hand split is possible only because the centre box omits the factor basis and optimizer geometry. An FTIP admissible protocol in Definition [ftip-005J] must retain those intervention coordinates when they affect the executable result.

The relevant optimizer results appear in version 1 of [⧉], submitted 5 August 2026: Sections 3--4 and 6, Appendices B.1--B.6 and B.9, with limitations in Section 11.

The full product symmetry acts on the factors by

It does not use \((UA,VA)\). For row-vector attention weights, the corresponding action uses right multiplication, as in Example 4.4.5.

The general rectangular preconditioner representation need not have the square-case uniqueness. A probability-one full-rank claim additionally needs a law for the gradient; a generic full-rank observation does not specify one.

The coordinatewise theorem is a zero-state, memoryless obstruction. The clock theorem is continuous-time and excludes scalar-Adam's momentum. Experimental optimizer and attention results remain observations reported by the source. Equivariance is not identified with low-rank recovery.

The factored-model results prove that update geometry can be a genuine intervention coordinate even when objective and represented initialization are fixed. They do not prove transformer or RLVR convergence, practical optimizer superiority, a scaling law, or any change in reliable capability. Different represented trajectories are not by themselves an acquisition witness of Definition [ftip-005X].

The memoryless theorem does not describe Adam's full state, and the continuous-flow result does not supply discrete rates or finite-step bounds. The source also notes that fixed-rank LoRA changes the learned-rank mechanism; no LoRA transfer is made here.

A dynamic-compute comparison also depends on active research state, persistent harness state, retained history, evaluation configuration, and descendant and retry costs. One realized chain and an archive-best envelope are different outcomes, even at the same declared budget.