Definition. Architecture-family transfer matrix [ftip-00LZ]
✍️source

A transfer matrix records which interfaces, intervention arms, and budgets permit a comparison. Each row is a study record: its source, estimand, matched controls, evaluation coordinates, and declared scope boundary are explicit. A blank or incompatible cell is an incomparability finding, not a missing score to be imputed. The vector and evaluation coordinates are reported together under the conditional frontier of Definition [ftip-00JM].

Comparison cell	Source	Estimand	Matched controls	Cost / stability / monitorability	Status and scope boundary
Transformer--KDA/full attention	Remark [ftip-00KF], Definition [ftip-00LU], Example [ftip-00LQ]	Difference in utility, cost, and \(K_{\rm state}\) between KDA and full attention	Same task, model scale, recipe, context, seeds, and evaluation law	Report measured cost vector, training/inference stability, and state/failure observability	Open; a finite KDA--Transformer result does not establish an architecture-wide ceiling
Transformer--Mamba	Mamba, Sections 3.1--3.3; § [ftip-00OQ]	Difference in retrieval utility and \(K_{\rm state}\) at the declared budget	Task, scale, recipe, context, and inference budget	Report cost vector, recurrence stability, and state/failure observability	Open; selective compression does not transfer a retrieval result across tasks
Transformer--linear attention	Linear attention, Sections 3.2--3.3; § [ftip-00OQ]	Difference in retrieval utility and \(K_{\rm state}\) for associative summaries	Task family, feature map, context budget, and evaluation law	Report prefix cost, numerical stability, and summary-state observability	Open; associative summaries do not imply full-attention equivalence
Transformer--CTM or looped model	Continuous Thought Machine, Sections 3.1--3.5; § [ftip-00KG]; § [ftip-00OQ]	Difference in utility and \(K_{\rm adapt}\) per charged tick or loop	Input, base model, stopping rule, recipe, and charged internal compute	Report tick/loop cost, training and inference stability, and stopping-signal monitorability	Open; extra passes are charged and do not imply continuous-time behavior
Transformer--BDH	BDH, Section 3.2; § [ftip-00OQ]	Difference in task utility and \(K_{\rm state}\) under the reduced state interface	Task, state parameterization, credit horizon, systems stack, and cost	Report memory and compute cost, optimization stability, and state observability	Open; a reduced state-space proposal is not a production transfer result
Transformer-Recurrent Looped Transformer	§ [ftip-00OQ]; [zhang2026rlt, Sections 2-3 and 5-8]	Paired change in finite-task utility and \(K_{\rm state}\) under recurrent decoder feedback	Same task, model family, parameter count, training tokens, seeds, and feedback interface; causal no-feedback and chunking arms	Report encoder/decoder FLOPs, recurrent state, chunk latency, memory, stability, and state monitorability	Open and finite-only until a declared deployment arm and transfer record exist
Residual baseline: Deep Delta rewrite	§ [ftip-00OQ]; [zhang2026deepdelta, Sections 2-4 and 6]	Paired change in utility and rewrite-sensitive state behavior after the rank-one read/compare/target/write intervention	Matched residual capacity, optimizer, tokens, seeds, and model family; no-write, write-only, and capacity controls	Report inference FLOPs, memory, residual/cache state, numerical stability, and rewrite monitorability	Open; finite GPT-2 evidence does not establish a general capability or frontier claim
Core architecture--JEPA adjunct	I-JEPA, Section 3; § [ftip-00OQ]	Conditional change in \(K_{\rm repr}\) and \(K_{\rm plast}\) after the objective intervention	Same base artifact, target task, predictor budget, data, and optimizer arm	Report intervention cost, training stability, and representation/probe monitorability	Conditional; an objective intervention is not a replacement architecture
I-JEPA--SplitJEPA	SplitJEPA, Sections 3.1--3.4; § [ftip-00OQ]	Subspace identifiability and conditional change in \(K_{\rm repr}\)	Matched invariant correspondences, variant-rank premise, normalization, and encoder budget	Report probe cost, numerical and optimization stability, and subspace monitorability	Conditional; identifiability does not become a general I-JEPA guarantee
Core architecture--liquid dynamics	Liquid time-constant networks, Sections 1--2; § [ftip-00OQ]	Difference in utility and \(K_{\rm adapt}\) under the solver intervention	Input law, solver, step budget, tolerance, task, and cost vector	Report solver cost, numerical stability, and trajectory/failure monitorability	Open; continuous dynamics and solver choices remain intervention coordinates
Core architecture--Neural Operator	Neural Operators, Section 3; § [ftip-00OQ]	Difference in utility and \(K_{\rm repr}\) across discretizations	Function-space task, discretization family, operator layer, and evaluation law	Report training/inference cost, discretization stability, and field-observation monitorability	Open; discretization invariance under one source formulation is not task transfer

A completed row must report its paired observations through § [ftip-00L4] and the common-recipe, equal-tuning, and restricted-envelope arms of § [ftip-00JU]. Until those records are supplied, its status remains open or incompatible.

References

Reference. Deep Delta Learning [zhang2026deepdelta]

2026
Yifan Zhang, Yifeng Liu, Mengdi Wang, Quanquan Gu
https://arxiv.org/abs/2601.00417

@misc{zhang2026deepdelta,
  title = {Deep Delta Learning},
  author = {Zhang, Yifan and Liu, Yifeng and Wang, Mengdi and Gu, Quanquan},
  year = {2026},
  eprint = {2601.00417},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2601.00417}
}

Reference. Recurrent Looped Transformer [zhang2026rlt]

2026
Yifan Zhang, Jichen Feng, Shihan Qin
https://arxiv.org/abs/2610.07591

@misc{zhang2026rlt,
  title = {Recurrent Looped Transformer},
  author = {Zhang, Yifan and Feng, Jichen and Qin, Shihan},
  year = {2026},
  eprint = {2610.07591},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.07591}
}

Context

Study design for comparisons across model instances [ftip-00LS]

This subsection specifies how a conditional architecture comparison may be instantiated without changing the architecture-neutral formulation. It is a study design, not a universal ranking of architectures.

Definition. Matched model-instance record [ftip-00LT]

A model-instance record names the architecture, parameterization or checkpoint, training and post-training recipe, optimizer, execution stack, inference budget, task family, evaluation law, and recorded cost vector. Two records are comparable only after the fields held fixed and the fields allowed to vary are declared.

Definition. Pair identity and comparison unit [ftip-00LU]

A comparison unit is a pair of model-instance records evaluated on the same task family and evaluation law, together with a declared cost scalarization. A Transformer reference and a Kimi Delta Attention instance may form such a pair only when the remaining coordinates are documented.

Remark. Paired responses to a common recipe [ftip-00LV]

For a pair of matched model-instance records, the common-recipe arm measures how the permitted architecture packages respond to one recipe. A score advantage in this comparison need not persist after either package is retuned.

Remark. Different recipes under equal tuning effort [ftip-00LW]

Under the equal-tuning-budget arm, the two selected model instances may use different recipes. That difference is compatible with equal predeclared tuning effort; forcing the selected recipes to match would answer a different comparison question.

Remark. How an allowed class changes its envelope [ftip-00LX]

In the restricted-envelope arm, enlarging an allowed intervention class at fixed evaluation law, cost accounting, and budget can raise its envelope and cannot lower it. A finite search over feasible interventions supplies a lower bound from the scores it attains. It certifies the supremum only if it covers every feasible intervention or is accompanied by a matching upper-bound argument.

Remark. Coordinate-change rule [ftip-00LY]

Changing the optimizer, post-training method, hardware or kernel, precision, context policy, or evaluation law changes a comparison coordinate. A score difference after such a change belongs to the new intervention, unless the study explicitly estimates the interaction.

Definition. Architecture-family transfer matrix [ftip-00LZ]
✍️source

Comparison cell	Source	Estimand	Matched controls	Cost / stability / monitorability	Status and scope boundary
Transformer--KDA/full attention	Remark [ftip-00KF], Definition [ftip-00LU], Example [ftip-00LQ]	Difference in utility, cost, and \(K_{\rm state}\) between KDA and full attention	Same task, model scale, recipe, context, seeds, and evaluation law	Report measured cost vector, training/inference stability, and state/failure observability	Open; a finite KDA--Transformer result does not establish an architecture-wide ceiling
Transformer--Mamba	Mamba, Sections 3.1--3.3; § [ftip-00OQ]	Difference in retrieval utility and \(K_{\rm state}\) at the declared budget	Task, scale, recipe, context, and inference budget	Report cost vector, recurrence stability, and state/failure observability	Open; selective compression does not transfer a retrieval result across tasks
Transformer--linear attention	Linear attention, Sections 3.2--3.3; § [ftip-00OQ]	Difference in retrieval utility and \(K_{\rm state}\) for associative summaries	Task family, feature map, context budget, and evaluation law	Report prefix cost, numerical stability, and summary-state observability	Open; associative summaries do not imply full-attention equivalence
Transformer--CTM or looped model	Continuous Thought Machine, Sections 3.1--3.5; § [ftip-00KG]; § [ftip-00OQ]	Difference in utility and \(K_{\rm adapt}\) per charged tick or loop	Input, base model, stopping rule, recipe, and charged internal compute	Report tick/loop cost, training and inference stability, and stopping-signal monitorability	Open; extra passes are charged and do not imply continuous-time behavior
Transformer--BDH	BDH, Section 3.2; § [ftip-00OQ]	Difference in task utility and \(K_{\rm state}\) under the reduced state interface	Task, state parameterization, credit horizon, systems stack, and cost	Report memory and compute cost, optimization stability, and state observability	Open; a reduced state-space proposal is not a production transfer result
Transformer-Recurrent Looped Transformer	§ [ftip-00OQ]; [zhang2026rlt, Sections 2-3 and 5-8]	Paired change in finite-task utility and \(K_{\rm state}\) under recurrent decoder feedback	Same task, model family, parameter count, training tokens, seeds, and feedback interface; causal no-feedback and chunking arms	Report encoder/decoder FLOPs, recurrent state, chunk latency, memory, stability, and state monitorability	Open and finite-only until a declared deployment arm and transfer record exist
Residual baseline: Deep Delta rewrite	§ [ftip-00OQ]; [zhang2026deepdelta, Sections 2-4 and 6]	Paired change in utility and rewrite-sensitive state behavior after the rank-one read/compare/target/write intervention	Matched residual capacity, optimizer, tokens, seeds, and model family; no-write, write-only, and capacity controls	Report inference FLOPs, memory, residual/cache state, numerical stability, and rewrite monitorability	Open; finite GPT-2 evidence does not establish a general capability or frontier claim
Core architecture--JEPA adjunct	I-JEPA, Section 3; § [ftip-00OQ]	Conditional change in \(K_{\rm repr}\) and \(K_{\rm plast}\) after the objective intervention	Same base artifact, target task, predictor budget, data, and optimizer arm	Report intervention cost, training stability, and representation/probe monitorability	Conditional; an objective intervention is not a replacement architecture
I-JEPA--SplitJEPA	SplitJEPA, Sections 3.1--3.4; § [ftip-00OQ]	Subspace identifiability and conditional change in \(K_{\rm repr}\)	Matched invariant correspondences, variant-rank premise, normalization, and encoder budget	Report probe cost, numerical and optimization stability, and subspace monitorability	Conditional; identifiability does not become a general I-JEPA guarantee
Core architecture--liquid dynamics	Liquid time-constant networks, Sections 1--2; § [ftip-00OQ]	Difference in utility and \(K_{\rm adapt}\) under the solver intervention	Input law, solver, step budget, tolerance, task, and cost vector	Report solver cost, numerical stability, and trajectory/failure monitorability	Open; continuous dynamics and solver choices remain intervention coordinates
Core architecture--Neural Operator	Neural Operators, Section 3; § [ftip-00OQ]	Difference in utility and \(K_{\rm repr}\) across discretizations	Function-space task, discretization family, operator layer, and evaluation law	Report training/inference cost, discretization stability, and field-observation monitorability	Open; discretization invariance under one source formulation is not task transfer

Remark. Applying a finite model to an architecture [ftip-00M0]

A toy representation or recurrence result may motivate a hypothesis, but it is not transferred to a model instance until its interface, task, cost accounting, numerical regime, and evaluation protocol are instantiated and checked. If any required assumption is unverified, applicability to the model instance remains unestablished.

Definition. Negative-result and stopping report [ftip-00M1]

A completed comparison reports the tested cost grid, excluded configurations, stopping rule, uncertainty, and negative results. Stopping without a detected difference is evidence about the declared study, not proof that the architectures have equal potential.

Remark. Open comparison questions [ftip-00M2]

The matrix in Definition [ftip-00LZ] is a protocol map, not a result table. The following probes keep the five capability coordinates separate and leave an unknown cell visible until the required records exist. Each probe names its source, estimand, controls, and evaluation coordinates so that a missing record cannot be mistaken for a zero score.

State access. Sources: Mamba, Sections 3.1--3.3 and linear attention, Sections 3.2--3.3. Estimand: the paired difference in retrieval utility and \(K_{\rm state}\). Controls: Transformer, Mamba, and linear-attention systems on a predeclared retrieval and continuation suite at matched parameter, context, recipe, and cost budgets. Evaluation: report cost, recurrence or summary stability, and state observability. Scope boundary: a long-context score does not transfer to another task family without its task and transport record.
Recurrent feedback. Source: [zhang2026rlt, Sections 2-3 and 5-8]. Estimand: the paired change in finite-task utility and \(K_{\rm state}\) from decoder feedback. Controls: a common causal encoder, model size, training tokens, seeds, and matched per-token budget, with no-feedback and declared chunk-size arms. Evaluation: report recurrent-state bytes, encoder/decoder FLOPs, chunk latency, numerical/training stability, and feedback monitorability. Scope boundary: parity, permutation tracking, and modular arithmetic remain finite evidence; without a deployment and transfer record, the cell is finite-only.
Residual rewrite. Source: [zhang2026deepdelta, Sections 2-4 and 6]. Estimand: the paired change in utility and rewrite-sensitive state behavior from a gated rank-one read/compare/target/write edit. Controls: residual capacity, optimizer, tokens, seeds, and model family, with no-write, write-only, and capacity-matched arms. Evaluation: report rewrite and residual-state monitorability, inference FLOPs, memory, numerical stability, and a held-out transfer task. Scope boundary: finite GPT-2 observations do not identify a general capability gain or architecture-wide law.
Adaptive computation. Source: Continuous Thought Machine, Sections 3.1--3.5 and the looped-model contract in § [ftip-00KG]. Estimand: the paired change in utility and \(K_{\rm adapt}\) per charged internal tick or loop. Controls: common input, base model, recipe, stopping rule, and charged internal compute. Evaluation: report tick/loop cost, training and inference stability, and stopping-signal monitorability. Scope boundary: an observed adaptive gain remains conditional on that stopping law.
Representation and abstraction. Sources: I-JEPA, Section 3 and SplitJEPA, Sections 3.1--3.4. Estimand: subspace identifiability and the conditional change in \(K_{\rm repr}\). Controls: fixed correspondences, full-rank variant differences, normalization, encoder budget, and matched downstream probes. Evaluation: report probe cost, optimization stability, and subspace monitorability. Scope boundary: the conditional subspace result is separate from downstream task transfer; failure of a premise is an incompatible cell, not a zero score.
Plasticity. Source: the objective and optimizer interfaces in Definition [ftip-00JM] and the I-JEPA intervention description in § [ftip-00OQ]. Estimand: the paired change in \(K_{\rm plast}\) after a declared post-training intervention. Controls: common recipe and equal-tuning arms with data, optimizer, trial count, stopping, and feedback fixed or explicitly varied. Evaluation: report intervention cost, training stability, and update-signal monitorability. Scope boundary: an intervention effect is limited to the declared base artifact, task, and objective; it does not transfer across architectures without a new paired record.
Portability. Sources: the Mamba, BDH, and Neural Operator method descriptions in § [ftip-00OQ]: Mamba, BDH, and Neural Operators. Estimand: survival of the declared utility and \(K_{\rm port}\) across one transfer edge. Controls: source model, task family, model scale, deployment regime, evaluation law, and cost vector retained as paired fields. Evaluation: report migration cost, stability after the shift, and failure monitorability. Scope boundary: evidence for one transfer edge does not order unmeasured edges; blank edges remain unmeasured.
Adjunct interactions. Sources: liquid time-constant networks, Sections 1--2, Neural Operators, Section 3, and I-JEPA, Section 3. Estimand: the joint change in the declared capability coordinates after an explicit intervention. Controls: declared core model, objective, temporal law, function-space bias, matched controls, and separate single-intervention arms. Evaluation: report joint and marginal costs, numerical/training stability, and interaction monitorability. Scope boundary: a joint effect is limited to the declared core, task, temporal law, and function-space regime; it does not transfer across regimes without separate intervention records.

Semantic representation diagnostics along pretraining checkpoints [ftip-00OZ]

This bounded diagnostic records when a declared semantic representation becomes recoverable along a pretraining trajectory. It uses the spatial and temporal probing design of [gurnee2024space, Sections 2-4 and 7] as a measurement template. The result is a representation diagnostic, not a claim about a developmental mechanism, a deployment capability, or a post-training law.

The task family is fixed before inspecting checkpoints: the six coordinate task datasets in the source family (world, United States, and New York City locations; historical figures, cultural works, and news headlines). Each score uses held-out entities. An eligible model family supplies at least three ordered pretraining checkpoints, identical train, validation, and test entity splits and filtering at every checkpoint, and the same layer-wise linear ridge-probe protocol, including a predeclared ridge-regularization grid and validation selection rule. A family or checkpoint missing one of those fields is reported as missing rather than silently pooled.

For model family \(m\), checkpoint \(c\), task \(q\), and layer \(\ell \), let \(s^{\mathrm {val}}_{m,c,q,\ell }\) and \(s^{\mathrm {test}}_{m,c,q,\ell }\) be validation and held-out-test coefficients of determination for the fixed probe. Choose the layer on validation entities, then evaluate once on the test entities:

\[ \ell ^*_{m,c,q}=\arg \max _{\ell \in L_m}s^{\mathrm {val}}_{m,c,q,\ell },\qquad s_{m,c,q}=s^{\mathrm {test}}_{m,c,q,\ell ^*_{m,c,q}}. \]

A predeclared layer may replace the validation choice, but a test score is never used to select \(\ell ^*\); this keeps the emergence score from being a layer-selection envelope.

The primary absolute emergence threshold is \(\tau =0.50\) in this held-out score, chosen before the trajectory is inspected. The first crossing is

\[ e_{m,q}(\tau )=\min \{c:s_{m,c,q}\geq \tau \}, \]

with \(e_{m,q}(\tau )=\bot \) when no checkpoint crosses. A crossing is reported with the checkpoint token count, layer, a bootstrap uncertainty interval over held-out entities, and the probe seed; it is not smoothed into a claim about an unobserved training interval. Sensitivity must include \(\tau \in \{0.25,0.50,0.75\}\), model family, parameter scale, and checkpoint spacing.

The structural record contains held-out coordinate prediction, block and cross-entity generalization, prompt variants, and the layer at which the score peaks. These are content and representation probes. A function or deployment record is separate: it must specify an intervention or user task, an outcome metric, matched controls, and its own cost and failure accounting. A probe crossing alone cannot populate that record.

The source reports linear spatial and temporal representations across Llama-2 and Pythia scales, prompt sensitivity, cross-entity robustness, and individual neurons; see [gurnee2024space, Sections 2.1-2.3, 3.1-3.3, 4.1-4.2, and 5]. Those observations motivate the fields above, but every trajectory association reported by this protocol is noncausal. In particular, it makes no claim about which training environment caused the score, whether a representation is used by a deployed function, or what post-training will preserve.

Each trajectory record therefore retains the model family and size, checkpoint token count, task and entity split, probe class and seed, all layer scores, threshold grid, missingness, and the separately reported function or deployment assay. Comparisons are within the declared task family and model family before any cross-family summary. A change in checkpoint spacing or probe protocol is a new arm, not evidence of an earlier or later emergence.

Convention. Identifiability regimes for semantic representation diagnostics [ftip-00P1]

Zhou and Klindt study labels-to-latents and latents-to-labels data-generating directions. Their general assumptions include a fixed continuous invertible map \(g:Z\to X\), a continuous learned encoder with \(h=f\mathbin \circ g\), a row-centered log-linear posterior \(p(y|z)={\rm softmax}(W^*z)\), effective dimension \(n'={\rm rank}(W^*)\), and a global population cross-entropy optimum with linear readout \(U\). The source boundary for this note is the DGP construction and assumptions in Sections 3.1 to 3.2, Theorems 4.1 to 4.4, Sections 5.1 to 5.4, and Section 6, with Supplementary Assumptions 8.1 to 8.3, of [zhou2026complete, Theorems 4.1 to 4.4, Sections 3 to 5, and Section 6].

Source observation. When \(m\geq n'\), Theorem 4.1 makes the classification-relevant part of every optimum linear, \(h(z)=Az+\nu (z)\) with \(UA=W^*\); the null-readout term satisfies \(\nu (z)\in N(U)\) and may remain unconstrained. When \(m<n'\), Theorem 4.2 gives generic nonlinear, non-identifiable optima when \(\operatorname {supp}(p_Z)\) has nonempty interior \(\Omega \), \(p_Z\) is continuous and strictly positive on \(\Omega \), and every candidate \(A\) has \(N(A)\cap \Omega \ne \varnothing \). These are different capacity regimes, not two scores for one representation.

The stronger conclusions require additional structure. In the generative case, Assumption 8.2 takes \(Z=\mathbb R_+^n\) and adds a factorized exponential conditional, constant conditional entropy, a uniform class prior, exact capacity \(m=n=n'\), non-negative activations, fixed within-class variance, and an energy-minimizing global optimum. Theorem 4.3 then gives the scaled permutation \(h(z)=DP_\sigma z\). In the discriminative case, Assumption 8.3 uses \(A^*\in \mathbb R^{n\prime \times n}\) with full row rank and \(U^*\in \mathbb R^{|Y|\times n\prime }\), adds \(W^*=U^*A^*\), \(k\)-sparse latents, and \(\operatorname {spark}(A^*)>2k\). With \(m\geq n\prime \), let \(P_U\) be the orthogonal projector onto \(\operatorname {Row}(U)\); the classifier-visible encoder \(P_Uh(z)=Az\) has \(N(A)=N(A^*)\), so the unique \(k\)-sparse solution can recover \(z\). Axis alignment and sparse recovery are distinct routes through distinct assumptions.

Section 5 reports finite experiments rather than a universal law. Linear alignment uses held-out coefficient of determination for a linear decoder; axis alignment uses the mean correlation coefficient (MCC) after Hungarian matching recovered axes to true latents. Sparse recovery compares supervised Lasso, dense OLS, and unsupervised sparse dictionary learning. Cross-system correspondence uses the Maximum Matching Unit (MMU): for each source unit, the maximum absolute Pearson correlation with any unit in a declared comparison model and layer on a declared stimulus set. The synthetic experiments use ten random seeds; the real-data cross-system analysis does not inherit that seed count. Section 6 records limits from constant conditional entropy, non-negativity, class priors, and the assumed sparse data-generating process; signed latents do not inherit Theorem 4.3 directly.

Finite identifiability record. For a fixed model family, checkpoint, task split, and probe seed, record

\[ \mathcal D=\bigl (r,m,n,n',\mathcal A,[h]_{\sim _r},R^2_{\rm lin}, R^2_{\rm sparse},\operatorname {MCC}_{\rm axis}, \operatorname {MCC}_{\rm sparse},\operatorname {MMU}_{\rm cross},\mathsf E\bigr ). \]

Here \(r\) is one declared regime: sufficient-capacity linear, undercomplete nonlinear, generative axis-aligned, or discriminative sparse recovery. \(\mathcal A\) lists the causal direction, DGP, capacity, sparsity, and assumptions actually held. \(R^2_{\rm lin}\) is the held-out coefficient for linear latent alignment; \(R^2_{\rm sparse}\) names the decoder and target, with supervised Lasso and dense OLS kept as separate arms. \(\operatorname {MCC}_{\rm axis}\) records the matching and correlation convention for axis recovery, while \(\operatorname {MCC}_{\rm sparse}\) records the matching convention, sparsity level, and whether the decoder is supervised or an unsupervised dictionary. \(\operatorname {MMU}_{\rm cross}\) records the directed source and comparison model, layers, unit sets, stimulus set, and correlation rule. An unavailable or inapplicable field is \(\bot \).

The equivalence class identifies only the ambiguity allowed by the declared regime: \((A,\nu )\) with \(UA=W^*\), \(\operatorname {rank}(A)=n'\), and \(\nu (z)\in N(U)\) in the linear regime; no universal quotient, recorded as non-identifiable under the stated assumptions, in the undercomplete regime; positive diagonal scaling and permutation \((D,P_\sigma )\) in the axis-aligned regime; and the classifier-visible null-space class \(N(A)=N(A^*)\) in the sparse regime. A sparse decoder has permutation-and-scaling ambiguity only when that convention is declared and measured. No scalar summary is defined, and none of these fields alone establishes monosemanticity.

Source-grounded finite instance. In the compressed-sensing arm, the paper fixes \(n=16\), \(n'=m=8\), \(|Y|=64\), draws a Gaussian measurement matrix with normalized columns, and tests \(k\in \{1,2,3\}\) over ten seeds. For the \(k=1\) arm, the record declares discriminative sparse recovery and the \(W^*=U^*A^*\) and spark conditions; it reports \(R^2_{\rm sparse}\approx 1\) for supervised Lasso, \(R^2_{\rm sparse}\approx n'/n=0.5\) for OLS, and \(\operatorname {MCC}_{\rm sparse}\approx 0.86\) for unsupervised Lasso, with \(\operatorname {MCC}_{\rm axis}=\bot \) and \(\operatorname {MMU}_{\rm cross}=\bot \). The ten-seed estimate, decoder arm, and missing fields are part of \(\mathsf E\); the values are finite synthetic observations, not a deployment or capability claim.

FTIP deduction. This record makes the representation question conditional on capacity, causal direction, and an explicit equivalence class. It lets the existing checkpoint route report linear recoverability, axis alignment, sparse recovery, and cross-system matching without treating any one readout as a capability or deployment result. A finite observation can support a regime-specific comparison; it cannot identify a developmental mechanism or transfer beyond the declared task and assumptions.

Proposed protocol. Before inspecting a checkpoint, freeze the task family, latent or proxy labels, split, width ratio, causal regime, assumptions, decoder class, comparison model and layer for MMU, stimulus set, and held-out evaluator. Select layers or decoder settings on validation data only. Report the full record, seed repetitions, missingness, and assumption-violation arms; keep each \(R^2\), MCC, and MMU in its native units. For real model activations, treat latent recovery as a declared proxy assay and report the proxy construction with the same boundary.

A scoped local formalization search of Forest, formal-math, TauCetiReview, and TauCetiWorker found no declaration that packages this four-regime record with its separately typed readouts. Existing representation and evaluation declarations cover neighboring fields only, so this finite identifiability record remains a protocol proposal.

This protocol links the conceptual-discovery question in § [ftip-00MH] to the capability coordinates in Definition [ftip-00JM]; the panel of open comparison questions in Remark [ftip-00M2] transcludes this child.

These probes do not establish a common ordering of architecture families. They specify what would count as evidence for a declared bottleneck change, while cost, numerical stability, monitorability, and transfer remain separate evaluation coordinates. Paired observations should use § [ftip-00L4], the study arms of § [ftip-00LS], and the capability vector of Definition [ftip-00JM]. Recurrent feedback and residual rewrite remain separate arms in every record; a finite result without a deployment arm stays finite-only.

Backlinks

Remark. Open comparison questions [ftip-00M2]

State access. Sources: Mamba, Sections 3.1--3.3 and linear attention, Sections 3.2--3.3. Estimand: the paired difference in retrieval utility and \(K_{\rm state}\). Controls: Transformer, Mamba, and linear-attention systems on a predeclared retrieval and continuation suite at matched parameter, context, recipe, and cost budgets. Evaluation: report cost, recurrence or summary stability, and state observability. Scope boundary: a long-context score does not transfer to another task family without its task and transport record.
Recurrent feedback. Source: [zhang2026rlt, Sections 2-3 and 5-8]. Estimand: the paired change in finite-task utility and \(K_{\rm state}\) from decoder feedback. Controls: a common causal encoder, model size, training tokens, seeds, and matched per-token budget, with no-feedback and declared chunk-size arms. Evaluation: report recurrent-state bytes, encoder/decoder FLOPs, chunk latency, numerical/training stability, and feedback monitorability. Scope boundary: parity, permutation tracking, and modular arithmetic remain finite evidence; without a deployment and transfer record, the cell is finite-only.
Residual rewrite. Source: [zhang2026deepdelta, Sections 2-4 and 6]. Estimand: the paired change in utility and rewrite-sensitive state behavior from a gated rank-one read/compare/target/write edit. Controls: residual capacity, optimizer, tokens, seeds, and model family, with no-write, write-only, and capacity-matched arms. Evaluation: report rewrite and residual-state monitorability, inference FLOPs, memory, numerical stability, and a held-out transfer task. Scope boundary: finite GPT-2 observations do not identify a general capability gain or architecture-wide law.
Adaptive computation. Source: Continuous Thought Machine, Sections 3.1--3.5 and the looped-model contract in § [ftip-00KG]. Estimand: the paired change in utility and \(K_{\rm adapt}\) per charged internal tick or loop. Controls: common input, base model, recipe, stopping rule, and charged internal compute. Evaluation: report tick/loop cost, training and inference stability, and stopping-signal monitorability. Scope boundary: an observed adaptive gain remains conditional on that stopping law.
Representation and abstraction. Sources: I-JEPA, Section 3 and SplitJEPA, Sections 3.1--3.4. Estimand: subspace identifiability and the conditional change in \(K_{\rm repr}\). Controls: fixed correspondences, full-rank variant differences, normalization, encoder budget, and matched downstream probes. Evaluation: report probe cost, optimization stability, and subspace monitorability. Scope boundary: the conditional subspace result is separate from downstream task transfer; failure of a premise is an incompatible cell, not a zero score.
Plasticity. Source: the objective and optimizer interfaces in Definition [ftip-00JM] and the I-JEPA intervention description in § [ftip-00OQ]. Estimand: the paired change in \(K_{\rm plast}\) after a declared post-training intervention. Controls: common recipe and equal-tuning arms with data, optimizer, trial count, stopping, and feedback fixed or explicitly varied. Evaluation: report intervention cost, training stability, and update-signal monitorability. Scope boundary: an intervention effect is limited to the declared base artifact, task, and objective; it does not transfer across architectures without a new paired record.
Portability. Sources: the Mamba, BDH, and Neural Operator method descriptions in § [ftip-00OQ]: Mamba, BDH, and Neural Operators. Estimand: survival of the declared utility and \(K_{\rm port}\) across one transfer edge. Controls: source model, task family, model scale, deployment regime, evaluation law, and cost vector retained as paired fields. Evaluation: report migration cost, stability after the shift, and failure monitorability. Scope boundary: evidence for one transfer edge does not order unmeasured edges; blank edges remain unmeasured.
Adjunct interactions. Sources: liquid time-constant networks, Sections 1--2, Neural Operators, Section 3, and I-JEPA, Section 3. Estimand: the joint change in the declared capability coordinates after an explicit intervention. Controls: declared core model, objective, temporal law, function-space bias, matched controls, and separate single-intervention arms. Evaluation: report joint and marginal costs, numerical/training stability, and interaction monitorability. Scope boundary: a joint effect is limited to the declared core, task, temporal law, and function-space regime; it does not transfer across regimes without separate intervention records.

Semantic representation diagnostics along pretraining checkpoints [ftip-00OZ]

\[ \ell ^*_{m,c,q}=\arg \max _{\ell \in L_m}s^{\mathrm {val}}_{m,c,q,\ell },\qquad s_{m,c,q}=s^{\mathrm {test}}_{m,c,q,\ell ^*_{m,c,q}}. \]

A predeclared layer may replace the validation choice, but a test score is never used to select \(\ell ^*\); this keeps the emergence score from being a layer-selection envelope.

The primary absolute emergence threshold is \(\tau =0.50\) in this held-out score, chosen before the trajectory is inspected. The first crossing is

\[ e_{m,q}(\tau )=\min \{c:s_{m,c,q}\geq \tau \}, \]

Convention. Identifiability regimes for semantic representation diagnostics [ftip-00P1]

Finite identifiability record. For a fixed model family, checkpoint, task split, and probe seed, record

Definition. Capability vector and conditional frontier [ftip-00JM]

For an architecture record \(A\), optimizer record \(O\), systems record \(S\), evaluation configuration \(\mathfrak e\) from Definition [ftip-00CE], and declared intervention \(\eta \) at cost cap \(C\), define the capability vector

\[ \mathbf K(A,O,S,\mathfrak e,\eta ,C) =\bigl (K_{\rm state},K_{\rm adapt},K_{\rm repr}, K_{\rm plast},K_{\rm port}\bigr ). \]

Its coordinates respectively record state access, adaptive computation, representation and abstraction, plasticity, and portability under the named task, intervention, evaluation law, and cost accounting. A coordinate may be a descriptive field, an estimand, or \(\bot \) when the record is missing, incompatible, or not measured. The tuple is therefore not a common scalar and does not order architectures without a declared comparison rule.

The five coordinates attach to the existing ceiling layers conditionally: representation claims first require a map from \(A\); optimizer-reachable plasticity requires \((A,O)\); state, adaptive computation, and portability claims require the relevant \((A,O,S)\) interface, task family, and deployment regime. A systems-feasible claim also carries \(C\), the cost vector and its declared scalarization. The records in Definition [ftip-00JH], Definition [ftip-00JI], Definition [ftip-00JL], and Definition [ftip-00CE] remain authoritative.

The scalar frontier \(V_A(C)\) of Definition [ftip-00JJ] is a projection onto the declared utility \(J_{\rm ev}\) for one architecture and intervention envelope; it is not a capability coordinate. For a fixed evaluation configuration, a vector-valued study may instead retain the conditional set

\[ \mathcal F_{\mathfrak e}(C) =\operatorname {Pareto}\left \{ \bigl (\mathbf K(A,O,S,\mathfrak e,\eta ,C), J_{\rm ev}(P_{A,\eta }),\mathbf c(P_{A,\eta })\bigr ): c_A(\eta )\leq C \right \}, \]

where Pareto dominance is declared for the reported coordinates. The set is conditional on the task, intervention class, evaluation law, cost vector, and missingness rule; it is not a universal intelligence frontier.

The semantic trajectory diagnostic in semantic representation diagnostics supplies a conditional representation record for this vector; its held-out probe score does not fill a function or deployment coordinate.

Matched-compute arms in § [ftip-00JU] compare entries only after stating which coordinates are held fixed, which are allowed to vary, and how the cost vector is charged. Cost, numerical stability, and monitorability remain evaluation coordinates alongside the vector, so a lower scalar cost or a stable implementation is not silently counted as a capability increase.

Depth reuse and latent recurrence [ftip-00KG]

Repeated depth, latent feedback, and recirculation change the computation performed at inference. Relative to the fixed reference autoregressive Transformer, their costs include any extra passes, state, and prefill work.

Definition. Computation-axis record [ftip-00KH]

For an architecture record \(A\), distinguish token steps, feed-forward depth passes, recurrent passes, and latent-state updates. Let \(d(A,r)\) be the declared number of depth passes in run \(r\); it is a computation coordinate, not a synonym for parameter count or reasoning quality.

Remark. Looped Transformers as a source result [ftip-00KI]

Giannou et al. show that a fixed shallow Transformer placed in a loop can execute programmed iterative computations, including conditional branches and in-context algorithms, in Looped Transformers as Programmable Computers[giannou2023looped]. This is a source result under its program, state, and precision assumptions; it is not a universal claim about language-model capability.

Remark. Parameter sharing and width tradeoffs [ftip-00KJ]

Xue et al. report that depth sharing can reduce trainable parameters while limiting modeling capacity, and study width and mixture-of-experts remedies in Go Wider Instead of Deeper[xue2021wider]. The comparison is empirical and task-specific; it does not identify a depth-independent ceiling.

Remark. Virtual logical depth and scaling [ftip-00KK]

Zhu et al. vary virtual logical depth by reusing weights and report reasoning gains at nearly fixed parameter count in Beyond Parameters: Exploring Virtual Logic Depth for Scaling Laws[zhu2025virtualdepth]. Their results motivate a depth coordinate in a scaling study, while leaving knowledge capacity, optimization, and transfer dependent on the declared training and evaluation protocol.

Remark. Recirculation as an inference intervention [ftip-00KL]

Mozer et al. introduce inference-time recirculation that feeds latent states back through an off-the-shelf model and report task improvements in Recirculation[mozer2026recirculation]. The added serial prefill and adaptive tuning belong in the systems and inference records; the observation is not a proof of a new representational ceiling.

Remark. Latent feedback with a preserved Transformer interface [ftip-00KM]

Wang et al. widen the feedback channel between decoding steps with latent feedback while retaining a Transformer and language-modeling interface in Full-bandwidth transformer[wang2026fullbandwidth]. Any reported token or accuracy savings are conditional on the scheduled multi-pass training and measured decoding cost.

Remark. Latent prediction as a training intervention [ftip-00KN]

Teoh et al. add next-latent prediction to next-token training and report compact predictive states without changing the Transformer interface in Next-Latent Prediction Transformers Learn Compact World Models[teoh2025nextlatent]. This belongs to the training intervention record, not to an architecture-only comparison.

Example. Equal parameters do not fix effective depth [ftip-00KO]

Two runs can share parameter count while using different numbers of recurrent or latent-feedback passes. A matched study must therefore report the computation-axis record and cannot infer an architecture ceiling from parameters alone.

Definition. Depth-reuse model-instance record [ftip-00KP]

For a depth-reuse comparison, record shared weights, pass count, stopping rule, recurrent state, training objective, token and pass FLOPs, prefill and decode latency, memory, and evaluation seeds. A pass-count change is an intervention even when the parameter tensor is unchanged.

Example. Fixed-recipe depth comparison [ftip-00KQ]

In a fixed-recipe arm, hold data, optimizer, feedback, stopping rule, and evaluation law fixed while varying only the declared depth-reuse intervention. The resulting difference is an estimand for that recipe, not a best-achievable comparison.

Remark. Scaling surfaces rather than a single law [ftip-00KR]

A depth-reuse study may fit a surface over parameters, tokens, passes, and cost. The fit is an empirical summary over its measured range; it does not establish an asymptotic law or a universal saturation point.

Remark. Architecture and training are separable records [ftip-00KS]

Weight sharing, latent objectives, and recirculation can alter optimization without changing the declared external interface. The comparison must retain architecture, training intervention, and systems coordinates separately.

Remark. Depth reuse across passes, parameters, and tasks [ftip-00KT]

The cited looped, virtual-depth, latent-feedback, and recirculation results do not show that more passes always improve capability, that parameter sharing dominates added parameters, or that inference-time gains transfer to training or to another task family. Each transfer requires a matched model-instance record.

Example. Frontier report for the KDA study [ftip-00LQ]

A Kimi Delta Attention versus Transformer study should publish the cost grid, paired scores, uncertainty intervals, scalarization weights, and the exact identities and configurations of the compared models. The report may then state which measured points are Pareto or iso-quality comparisons under that protocol.

Matched-compute comparison protocol [ftip-00JU]

Definition. Common-recipe comparison arm [ftip-00JV]

A common-recipe arm fixes training data, optimizer family, schedule, post-training procedure and feedback, training and post-training budgets, inference budget, and evaluation protocol and draws. Interface-compatible parameter choices are declared in advance as part of each permitted architecture package. The comparison varies that package under the common recipe; any further deviation is recorded as a separate comparison coordinate. Its score difference measures performance under this recipe, not the best attainable result for either architecture.

Definition. Equal-tuning-budget comparison arm [ftip-00JW]

An equal-tuning-budget arm permits architecture-specific tuning under a predeclared protocol that fixes the tuning data and its volume, trial count, selection rule, stopping rule, and tuning cost for both architectures. The cost uses the same declared accounting rule. The resulting comparison measures practical performance under this equal optimization effort; it does not determine best possible training or a representation-only limit.

Definition. Restricted-envelope comparison arm [ftip-00JX]

A restricted-envelope arm declares an allowed intervention class \(\mathcal E_A\subseteq \mathfrak I_A\) for each architecture, specifying the permitted training, post-training, and inference procedures. The classes expose exclusions, search budgets, seeds, and stopping rules. Restrict the evaluation functional of Definition [ftip-00JJ] to \(\mathcal E_A\) and compare the resulting suprema under the same finite cost cap \(C\geq 0\), common evaluation law, and common declared cost accounting.

The measurable, integrable evaluation domain and extended-real conventions of Definition [ftip-00JJ] apply: an empty feasible class has value \(-\infty \), an unbounded-above feasible score set has value \(+\infty \), and a finite supremum need not be attained. The allowed class is part of the comparison, so its envelope is a restriction-dependent quantity, not an unrestricted architectural ceiling.

Definition. Model-instance comparison evidence record [ftip-00JY]

A model-instance record contains architecture/checkpoint identity, parameter count, data and post-training recipe, optimizer, precision, hardware, cache policy, training and inference cost vectors, task suite, and independent evaluation seed. A missing field makes an architecture attribution unknown.

Remark. Systems optimizations are measured interventions [ftip-00JZ]

FlashAttention, KV-cache layouts, quantization, batching, and kernels can change feasible cost. If they approximate, truncate, or alter precision of the mathematical computation, the record must mark that as a systems intervention rather than as a pure architecture comparison.

Theorem. Matched-protocol difference is an estimand [ftip-00K0]

Given \(n\geq 1\) evaluation draws, let \(\widehat V_A(C)\) be the sample mean of finite real evaluation scores for architecture \(A\) at a finite cost cap \(C\geq 0\). Under a fixed evaluation law and common-recipe arm, the finite difference \(\widehat V_A(C)-\widehat V_B(C)\) is a well-defined empirical estimand.

It does not identify a causal architecture effect when data, tuning, or systems coordinates differ. The conclusion follows directly from the declared record.

Remark. Conditions for applying a toy model [ftip-00K1]

A toy theorem applies to a concrete model only if an explicit map from its state and interface to the model preserves the theorem's assumptions, including its cost accounting. Without such a map, the conclusion concerns only the abstract model; its validity for the concrete system is unknown.

Mechanism families and capability bottlenecks [ftip-00OQ]

This section compares mechanism families by the capability bottlenecks they change and by the evidence needed to detect those changes. The shared model, task, intervention, cost and evaluation interfaces remain fixed conditions for every claim, while source descriptions, measured results and transfer claims remain separate.

The comparison uses five capability dimensions: state access, the information that can remain available across tokens, steps or external memory operations; adaptive computation, conditional allocation of depth, recurrence, search or stopping together with its charged cost; representation and abstraction, the available representation, composition or predictive state; plasticity, the changes reachable by a declared objective, optimizer, parameter tying and post-training recipe; and portability, survival of a result when the model family, task, scale or deployment regime changes. This is distinct from transfer, which evaluates a frozen artifact on new problems under a common deployment budget. They are coordinates for the capability vector, not a single intelligence scalar.

A bottleneck is the declared resource or interface condition that limits one of these dimensions in a comparison. Cost, numerical stability and monitorability remain evaluation coordinates: the ability to observe the relevant state, compute, update or failure signals under the study protocol. A matched-compute comparison fixes the cost accounting before comparing outcomes; its protocol and the architecture frontier remain authoritative elsewhere in this chapter.

The default core panel contains mechanism families that alter the sequence computation or its state interface. Existing Transformer notation and the decoder-only reference are in § [ftip-000K]; the entries below identify the question each family makes testable.

Transformer reference. Attention exposes token-indexed content access through the declared context and serves as the baseline for the matched-compute comparison and interface comparisons.
Selective state-space models. The Mamba construction, Sections 3.1--3.3 makes the state-space parameters input-dependent and computes the resulting time-varying recurrence with a hardware-aware scan. The testable bottleneck is how selective compression trades long-context access and throughput against retrieval and portability boundaries; a source mechanism does not settle that tradeoff.
Linear attention. The linear-attention formulation, Sections 3.2--3.3 uses a feature-map factorization and associative prefix summaries rather than the full softmax attention matrix. The comparison tests which retrieval and memory tasks preserve their evaluation under the changed state interface; it does not identify every hybrid or recurrent model with pure linear attention.
Internal-tick and looped models. The Continuous Thought Machine, Sections 3.1--3.5 uses a discrete internal tick dimension, private neuron-level models that process activation histories, synchronization as a latent representation, and standard cross-attention to the input features. Its loss selects minimum-loss and maximum-certainty ticks, making variable internal compute a testable mechanism. The looped computations in § [ftip-00KG] are a separate family; neither is a continuous-time claim, and every extra tick or loop must be charged in the comparison.
Recurrent or compressed-state mechanisms. The Dragon Hatchling / BDH proposal, Section 3.2 defines a tensor-friendly BDH-GPU state-space system whose attention state admits a reduced representation, alongside local graph dynamics. Its open comparison is whether this state interface supports the target tasks and stable credit assignment at a declared budget; the proposal is not a production result.
Recurrent feedback. The [zhang2026rlt, Sections 2-3 and 5-8] splits a causal parallel encoder from a recurrent decoder whose previous-token state is fed back into the next token. On finite parity, permutation-tracking, and modular-arithmetic tasks, its reported length extrapolation depends on the feedback ablation and on chunk size. This is evidence for a state-access intervention in a bounded synthetic setting; it does not establish a frontier reasoning gain or a general recurrence law.
Residual rewrite. [zhang2026deepdelta, Sections 2-4 and 6] replaces an additive residual shortcut with a gated rank-one read/compare/target/write edit. Reported GPT-2 experiments are a finite mechanism observation whose effect remains confounded with residual capacity, compression, and rewrite behavior. A write-only control, capacity-matched control, multiple seeds, and a charged transfer task are needed before attributing a capability change to the rewrite interface.

The recurrent-feedback and residual-rewrite entries are separate intervention arms. Their paired records must expose the changed state or write interface, causal ablations, parameter and token budgets, inference FLOPs, memory, KV/cache or recurrent-state costs, chunking overhead, and monitorability. If the available evidence remains finite-only, the result stays finite-only and does not support ceiling or frontier language.

The adjunct panel changes an objective, temporal law or function-space bias without being interchangeable with the core architecture families. I-JEPA, Section 3 is an encoder/predictor training architecture as well as an embedding-space prediction objective: its context, target and predictor use ViT backbones, the target encoder is updated by exponential moving average, and masked target blocks are predicted from a context block. That predictive objective leaves a specific representation limit: retaining or recovering the latent state does not by itself identify which directions are invariant and which vary across matched observations. SplitJEPA, Sections 3.1--3.4 (the project overview) makes the boundary explicit—predictive recovery can leave a global orthogonal mixing—and resolves it under stationary Gaussian predictive dynamics, a full-rank second moment of variant differences across invariant correspondences, and representation normalization, by identifying invariant and variant subspaces up to within-block isometries without an observation decoder. Here SplitJEPA is a structured JEPA adjunct and a conditional identifiability result, not a general guarantee for I-JEPA. Liquid time-constant networks, Sections 1--2 supply input-dependent continuous dynamics and a numerical ODE-solver interface; the Neural Operator formulation, Sections 1 and 3 supplies a function-space lens with discretization-invariant operator layers. The Zoology study, Sections 3--5 provides controlled recall analyses of sequence mixers. These sources motivate hypotheses and controls; they do not establish a common capability ordering or make an adjunct a replacement architecture. The JEPA comparability interface specifies fields that must be fixed for a JEPA-like comparison; it is not the source of I-JEPA's architecture. Combining CTM's internal dynamics with I-JEPA's encoder/predictor package is a proposed joint intervention, not an established result, and would require separate attribution of both components.

For each panel row, a study should name the changed state or compute interface, the bottleneck dimension, the task and intervention, the evidence type, and the transport boundary. Existing architecture-indexed ceilings and matched compute, published paired measurements (§ [ftip-00KU] and § [ftip-00L4]), conditional frontiers (§ [ftip-00LH]) and study design (§ [ftip-00LS]) provide those controls. Blank, incompatible and unmeasured cells remain findings rather than scores to be filled by intuition.

Remark. Open model-instance questions [ftip-00KF]

The relative capability and cost of KDA, full attention, and JEPA candidates remain empirical questions. A reproducible comparison requires identified checkpoints or reproducible training, fixed evaluation seeds, and measured cost vectors. Architecture-specific ceilings remain estimands and hypotheses rather than established constants.

Definition. Pair identity and comparison unit [ftip-00LU]

Paired measurements for model comparison [ftip-00L4]

A paired measurement specifies both model instances and their task, intervention, evaluation, and cost coordinates. These conditions determine the meaning of each point in the measured frontier.

Definition. Comparison pair identity [ftip-00L5]

A paired comparison identifies the two systems, exact checkpoint or training commit, architecture variant, parameter counts, tokenizer, and context limit. A label such as ``Transformer'' is insufficient when attention, cache, or layer patterns differ.

Definition. Common task and evaluation conditions [ftip-00L6]

Both systems use the same task family, prompt and data law, scoring rule, sampling seeds, stopping rule, and evaluator version. Any exception is a separate comparison arm, not an unrecorded architecture effect.

Definition. Training and post-training coordinates [ftip-00L7]

Record data volume, token order policy, optimizer and schedule, supervised tuning, feedback or reinforcement procedure, update count, and selection rule. The common-recipe arm fixes these coordinates; the equal-tuning arm allows predeclared alternatives with equal search resources.

Definition. Inference and systems coordinates [ftip-00L8]

Record hardware, software and kernel versions, precision, batch, cache policy, sequence lengths, warm-up procedure, concurrency, and decoding settings. A kernel or cache change that alters numerical computation is marked as an intervention rather than hidden as an implementation detail.

Definition. Per-system cost vector [ftip-00L9]

For each system, the cost vector \(\mathbf c\) contains training FLOPs, inference FLOPs, wall time, memory, energy when available, and context length. The primary scalar \(C\) and its weights are declared beside the vector; omitted coordinates are marked unknown.

Definition. Repeated measurement rule [ftip-00LA]

For each fixed workload and system, record warm-up runs, repetition count, random seeds, and the aggregation rule. Report mean and dispersion for time, memory, and score; a single fastest run cannot define a cost frontier.

Definition. Paired uncertainty estimates [ftip-00LB]

When the same evaluation items are used, report paired score differences and an uncertainty procedure fixed before inspecting the result. Confidence intervals quantify sampling variation; they do not repair unmatched training, systems, or task coordinates.

Example. Published Kimi Linear comparison [ftip-00LC]

Table 4 of the Kimi Linear report compares the experimental models pretrained on 1.4 trillion tokens in Example [ftip-00KV], after the same supervised fine-tuning (SFT) recipe. Both have 48 billion total and 3 billion active parameters. Three selected benchmark scores are [kimi2025linear, Sections 4 and 5.4 and Table 4]:

Arm	MMLU-Pro	LiveBench	EvalPlus
Kimi Linear	67.4	45.2	61.0
Full MLA	65.7	45.7	62.6

The entries use the paper's score units, with larger values better on each selected benchmark; LiveBench is reported as Pass@1. Kimi Linear is higher on MMLU-Pro and lower on LiveBench and EvalPlus in these observations. These are reported point estimates on different tasks, not a single capability score; the table supplies no uncertainty estimate.

The common recipe uses the K2 pretraining corpus, MuonClip and a shared training schedule. SFT proceeds from broad instruction data to targeted reasoning tasks; evaluation uses temperature 1.0 and an internal framework derived from LM-Harness [kimi2025linear, Section 5.4]. Thus the scores provide a concrete example of how the trained hybrid and full-MLA designs respond to the same recipe. The cost mechanism and its possible post-training consequences are developed in Example [ftip-00KZ].

Remark. Measured and assumed costs [ftip-00LD]

A useful cost model can combine measurements with assumptions justified by the architecture and its implementation. In Example [ftip-00L3], the generation share \(f\) and speed factor \(s_{\rm gen}\) determine a predicted attempt-throughput factor. Their assumed values can be varied without pretending they were measured in the original experiment.

For fixed \(f\in [0,1]\), write \(S(s)=1/(1-f+f/s)\) for \(s>0\). This function is nondecreasing. An interval \(s_{\rm gen}\in [s_-,s_+]\), with \(0<s_-\leq s_+\), therefore gives the conditional range \(S\in [S(s_-),S(s_+)]\). This is sensitivity to a chosen assumption range; it is a statistical confidence interval only if a sampling argument supplies that interpretation.

Definition. Measured contrasts and conditional predictions [ftip-00LE]

A measured paired contrast evaluates the same declared quantity for two observed model instances under a specified task, protocol and cost accounting. The common-recipe scores in Example [ftip-00LC] are one example. A confirmatory comparison fixes its matching and selection rules before examining the scores.

A conditional paired prediction instead uses an explicit model to supply one or more cost or response quantities, then derives the comparison under those assumptions. The discovery probabilities in Example [ftip-00L3] are an example. Such a prediction can be motivated by existing observations and guide a later experiment; its unmeasured inputs remain assumptions, and agreement with new matched measurements tests the model.

Example. How generation share changes the predicted gain [ftip-00LF]

Keep the illustrative generation factor \(s_{\rm gen}=2.2\) from Example [ftip-00L3] and vary the baseline generation share \(f\). Its cost model gives the following total attempt-throughput factors:

Generation share	Predicted throughput factor
20 percent	1.122
50 percent	1.375
80 percent	1.774
100 percent	2.200

These calculated values show why the same decoding improvement can matter much more for generation-heavy search than for a verifier- or optimizer-dominated workload. They suggest measuring the time spent in generation, verification and updates before selecting where to invest systems effort. A change in the bottleneck changes the useful intervention, even when the attention architecture stays fixed.

Remark. Testing the predicted architecture tradeoff [ftip-00LG]

The memory and discovery models predict the strongest benefit when long prefixes make attention state expensive, saved capacity improves useful rollout throughput, and per-attempt success remains above the break-even threshold. Short contexts, expensive verification, or strongly correlated attempts can weaken that benefit. These are distinct mechanisms to test, rather than a universal ranking of KDA and full attention.

A comparison across implementations can retain this reasoning while re-estimating state sizes, generation share and success probabilities for the new system. Report both the prediction and the matched measurements: their agreement or disagreement identifies which mechanism or assumption needs revision.