Dynamic compute, retained context, and rollout approximation [ftip-00DA]
✍️source

In a joint inference and training intervention, the retained-context rule, rollout horizon, runtime budget, and gradient estimator may all change together. Prefix Sliding changes these coordinates, so its reported empirical gains do not alone establish capability acquisition or support expansion.

Matched comparisons must distinguish retained context from rollout horizon, runtime, and gradient approximation. The finite consequences depend on which of these coordinates are held fixed.

1. Inference intervention coordinates [ftip-00DB]

A context intervention is a change to what a fixed model can attend to while it generates a rollout. The intervention is meaningful only after the prompt law, decoding rule, token budget, and retained-state rule are named.

Definition 1.1. Retained-context policy [ftip-00DC]

Fix a token history $h_t=(x_0,\ldots ,x_{t-1})$. A retained-context policy is a deterministic map $\kappa _t(h_t)$ that selects an ordered subsequence of the history, together with a decoder that conditions its next token on that subsequence. The policy, rather than the model weights, names which prior tokens remain available at step $t$.

A comparison of two policies is conditional on the same model, prompt law, decoding law, stopping rule, and budget unless the comparison includes a second declared intervention coordinate.

Definition 1.2. Prefix-window retained state [ftip-00DD]

For prefix and window lengths $p,w\in \mathbb N_0$, define the prefix-window policy by retaining every token in positions $0,\ldots ,p-1$ and the most recent $w$ tokens after that prefix. If the history is shorter than either region, the available positions are retained without padding.

The policy is a context rule only. It does not assert that omitted tokens are irrelevant, nor that the resulting continuation distribution equals the full-history distribution.

Remark 1.3. Prefix Sliding is a source-reported intervention [ftip-00DE]

Muennighoff et al. describe Prefix Sliding in Sections 2--5, Figures 6--9, and the Limitations section of [muennighoff2026prefixsliding, Sections 2--5, Figures 6--9, and Limitations]. Their method preserves a prefix while sliding a recent attention window during test-time scaling. The reported experiments change retained context and runtime, and may also change rollout horizon and the training gradient approximation. These are source observations, not a theorem of universal speedup or capability.

The source studies named model and task configurations. It does not by itself establish support expansion, latent capability acquisition, or a deployment-shift guarantee.

Example 1.4. One trace under full and prefix-window attention [ftip-00DF]

This schematic keeps the model and token history fixed while changing only the retained-context policy. It is a local illustration, not a source figure.

$\text{[math]}$

The two outputs can differ even though the represented weights and visible prompt are identical. Any claim about improvement must therefore name the retained-context intervention and its evaluation law.

Remark 1.5. Retained context is not weight learning [ftip-00DG]

Changing $\kappa _t$ changes the information supplied to a fixed decoder; it does not update represented weights. A successful continuation can therefore be an elicitation effect, a context effect, or both. Weights, optimizer and feedback, inference state, and evaluation remain distinct intervention coordinates, as in § [ftip-0001].

2. Finite horizon and retained-context cost [ftip-00DH]

Counting exposure to retained tokens makes the finite-horizon cost comparison precise. A context cap alone does not guarantee information preservation.

Definition 2.1. Finite rollout token budget [ftip-00DI]

A rollout has a finite token budget $T\in \mathbb N_{\geq 1}$ when its history contains at most $T$ generated positions after the prompt. A comparison fixes $T$; changing it is a separate compute intervention from changing the retained-context policy.

Definition 2.2. Context exposure count [ftip-00DJ]

For a retained-context policy $\kappa $ and generated history $h_t$, define its exposure count through budget $T$ by

\[ E_T(\kappa )=\sum _{t=0}^{T-1}\left |\kappa _t(h_t)\right |. \]

This is a finite attention-input proxy. It is not a runtime identity: the implementation may have caching, batching, kernel, and communication costs that are not represented by $E_T$.

Lemma 2.3. Prefix-window exposure is uniformly capped [ftip-00DK]

Let $p,w\in \mathbb N_0$ and let $\kappa ^{p,w}$ be the prefix-window policy of Definition 1.2. For every history and every $t$,

\[ \left |\kappa ^{p,w}_t(h_t)\right |\leq p+w. \]

Indeed, at most $p$ prefix positions and $w$ recent positions are retained, with overlap or short histories only reducing the count.

Theorem 2.4. Capped retained context bounds finite exposure work [ftip-00DL]

Under the hypotheses of Lemma 2.3, every rollout with budget $T$ satisfies

\[ E_T(\kappa ^{p,w})\leq T(p+w). \]

Proof. Apply the pointwise bound in Lemma 2.3 to each of the $T$ nonnegative summands in Definition 2.2, then sum. This bounds token exposure, not wall-clock runtime or evaluation quality.

Example 2.5. Warm-up and sliding-window arithmetic [ftip-00DM]

For $p=3$, $w=2$, and $T=6$, the first steps grow the retained set until the cap $p+w=5$ is reached. The exposure bound is therefore $E_T\leq 6\cdot 5=30$; the exact count depends on the prompt and stopping convention.

$\text{[math]}$

The arithmetic is an FTIP finite consequence, not a runtime measurement from the Prefix Sliding experiments.

Remark 2.6. A memory cap does not preserve information [ftip-00DN]

The bound in Theorem 2.4 controls the number of retained token positions. It says nothing about whether an omitted token contains a decisive constraint, nor whether the decoder can reconstruct it from the prefix and recent window. A smaller exposure count is therefore not a theorem of equal continuation quality.

3. Training and evaluation confounds [ftip-00DO]

A retained-context comparison can alter both inference and training. Attribution to either mechanism depends on the joint intervention and the coordinates held fixed by a matched evaluation.

Definition 3.1. Joint intervention cell [ftip-00DP]

A joint intervention cell is a tuple $I=(M,\mu ,\pi ,T,\kappa ,\widehat g)$ consisting of fixed model weights $M$, prompt law $\mu $, decoding law $\pi $, token budget $T$, retained-context policy $\kappa $, and training or evaluation estimator $\widehat g$. Two cells differ in a declared coordinate only when all other coordinates are held fixed.

Definition 3.2. Estimator-changing training coordinate [ftip-00DQ]

An estimator-changing coordinate is a change in $\widehat g$, the map used to turn sampled histories and rewards into an update. A context mask can change this coordinate when it changes which tokens contribute to the sampled loss or gradient. The notation does not assume a particular optimizer.

Remark 3.3. Training masks change more than attention cost [ftip-00DR]

If a retained-context rule is used during training, it can change the attention inputs, sampled continuation, loss support, and gradient estimator at once. A lower value of Definition 2.2 is therefore not an isolated compute intervention unless the training estimator and all other coordinates are matched explicitly.

Example 3.4. Equal terminal score, different trajectory exposure [ftip-00DS]

Two cells can reach the same terminal score while exposing different numbers of context positions. This finite counterexample blocks an inference from equal endpoint score to equal trajectory cost.

$\text{[math]}$

The score equality is compatible with unequal intermediate histories and unequal exposure counts.

Remark 3.5. What a retained-context comparison can identify [ftip-00DT]

With matched weights, prompt law, decoding, budget, estimator, and evaluation interface, a comparison can identify a conditional contrast between retained-context policies. Without those controls it identifies only the joint cell, not a causal effect of context alone.

Definition 3.6. Retained-context evaluation protocol [ftip-00DU]

A retained-context evaluation protocol fixes a model $M$, prompt law $\mu $, decoder $\pi $, budget $T$, evaluator $\mathsf E$, and two policies $\kappa _1,\kappa _2$. It reports the paired outcomes under the same sampled prompts and declared randomization coupling, together with exposure counts from Definition 2.2.

Lemma 3.7. Matched protocols isolate a conditional policy contrast [ftip-00DV]

Let $m\in \mathbb N_{\geq 1}$ and let $Y_{1j},Y_{2j}\in \mathbb R$ for $j=1,\ldots ,m$ be the paired scalar evaluator outcomes generated by the protocol of Definition 3.6. Then the finite paired contrast

\[ \widehat \Delta =\frac 1m\sum _{j=1}^{m}(Y_{1j}-Y_{2j}) \]

is a statistic of the declared policy contrast under that common protocol. It is not an unconditional capability effect and does not identify what would happen after changing any held-fixed coordinate.

Remark 3.8. Separating horizon, cache, and estimator effects [ftip-00DW]

Prefix Sliding is useful as a named intervention for retained context and test-time scaling, but its reported configurations do not establish a universal runtime law, gradient theorem, support expansion, or capability acquisition. The next questions are to measure horizon, cache, estimator, and evaluation effects separately, while preserving the controls in Definition 3.6.

The finite conclusions depend on the declared protocols. They do not describe a full optimizer or establish a result for all language models.

4. Typed harness adaptation and total lifecycle accounting [ftip-00OY]

A harness adaptation claim must keep the executable solver, the proposer that edits its harness, the retained harness state Definition [ftip-00C8], and the evaluator Definition [ftip-001T] as different objects. A change in one object is not evidence that the others changed. The stateful-search convention already records durable research state and replay; the comparison therefore fixes an adaptation and accounting boundary for interpreting evidence.

Typed roles and declared endowment. Let $M$ be a fixed solver artifact, $\theta $ the parameters of a proposer, $h_t\in \mathcal H$ the versioned harness state after inner round $t$, and $q\in \mathcal Q$ a task. The proposer is a revision policy

\[ \pi _\theta :\mathcal Q\times \mathcal H\times \mathcal R \longrightarrow \mathsf {Dist}(\mathcal Y), \]

where $\mathcal R$ is a typed execution-report space and $\mathcal Y$ contains candidate edits. Applying an edit to a parent state produces a candidate $\widetilde h_t$; a validation and selection rule either commits $h_t=\widetilde h_t$ or retains $h_t=h_{t-1}$. The solver $M$ and final proposer parameters $\theta _*$ are fixed during this inner loop.

For the declared outer training, define

\[ O_{\rm outer}=(\theta _0,\ldots ,\theta _K,D_{\rm train},\xi _{<K}, L_{\rm outer},C_{\rm outer}), \]

The final proposer is $\theta _*:=\theta _K$, and $L_{\rm outer}$ records prompts, generated and failed revisions, and teacher filtering. The declared endowment for an inner-loop comparison is

\[ E_0=(M,\theta _*,h_0,\kappa _0,\mathcal Q_0,\mathcal P_0,\rho _0,O_{\rm outer}), \]

where $\kappa _0$ contains implementation and harness versions, $\mathcal Q_0$ the task and split laws, $\mathcal P_0$ permissions and available tools, and $\rho _0$ seed, API, tokenizer, and evaluator state. Existing solver code, seed harnesses, accessible examples, and checker interfaces are part of $E_0$; if an arm constructs any of them, that preparation is a charged outer operation recorded in $C_{\rm outer}$. An adaptation result is therefore conditional on the same declared endowment, including its outer-training trace, event reports, and cost, even when the retained state is later changed.

Outer training and inner adaptation. Outer training updates only the proposer, for example

\[ \theta _{k+1}=U(\theta _k;D_{\rm train},\xi _k), \]

using proposer or teacher data and execution feedback. Its outer-training trace is included in $E_0$; at test time the inner loop starts from the declared seed $h_0$. For a finite nonempty candidate index set $J_t$, it repeatedly records

\[ R_t=\operatorname {report}(M,h_{t-1},q_{\rm fb,t},\xi _t),\qquad \{y_{t,j}\}_{j\in J_t}\sim \pi _{\theta _*}^{\otimes J_t}(q,h_{t-1},R_t),\qquad h_t=\operatorname {select}\bigl (h_{t-1}, \{\operatorname {apply}(h_{t-1},y_{t,j})\}_{j\in J_t};q_{\rm score,t}\bigr ). \]

An outer teacher, when used for supervised initialization, is a separate role from the proposer and solver. Its prompts, generated revisions, failed revisions, and filtering work remain in $L_{\rm outer}$. Neither an outer weight update nor an inner harness edit may be described as a model-weight capability change.

Leakage controls. A reported comparison declares pairwise-disjoint sets $Q_{\rm train}$, $Q_{\rm fb}$, $Q_{\rm score}$, $Q_{\rm dev}$, and $Q_{\rm eval}$ (or records the exact exception). Feedback and score questions may select or reject a candidate during adaptation; development questions may select a frozen proposer or protocol. Final evaluation questions are revealed only after the adapted state and its configuration are frozen, and their outcomes cannot revise that state in the same run. The task family, solver, proposer, seed harness, checker, and split law are reported with every score.

Provenance and rollback. Each committed state carries a record

\[ V_t=(\operatorname {digest}(E_0,M,\theta _*,h_t,\kappa _t), E_0, L_t,\{y_i,R_i,d_i\}_{i<t},C_{\leq t}), \]

where the digest also covers the declared endowment $E_0$, including its task and split laws, permissions, tools, and seed and evaluator state. $L_t$ is the event prefix, $d_i$ records validation and commit decisions, and $C_{\leq t}$ is the cumulative cost record. A rejected edit leaves the parent state in place but remains in the event log. A proposed rollback, as specified in Definition [ftip-00D7], to $V_j$ restores the complete named snapshot and its configuration; it cannot erase the proposal, execution, storage, or evaluator costs already incurred. Rollback is an FTIP protocol proposal, not a result established by the source papers.

Total accounting. Refine the lifecycle vector $C(P)=(C_{\rm data},C_{\rm feedback},C_{\rm rollout},C_{\rm env}, C_{\rm update},C_{\rm storage},C_{\rm eval})$ from Definition [ftip-005H] by recording a role-indexed vector for

\[ \mathbf C_{\rm total} =\sum _{r\in \{\rm teacher,prop_{outer},prop_{inner},solver,tool,evaluator\}} \mathbf C^{r}. \]

Every component keeps its declared unit. The account includes initial endowment preparation when it is arm-specific, proposer and teacher calls, solver and tool executions, evaluator calls, context and artifact storage, retries, invalid edits, rejected candidates, failed experiments, and any development work. Cached and uncached tokens, wall time, FLOPs, bytes, and calls are not silently converted into one scalar. A price or scalar utility may be reported only after this typed vector and the split of each role are available.

Finite typed example. Fix $M$, $\theta _*$, and $h_0$ in the endowment. In round one, the proposer emits an invalid edit $y_1^a$ and a valid edit $y_1^b$; the first execution and rejection are charged, while selection commits $h_1=\operatorname {apply}(h_0,y_1^b)$. In round two, noisy feedback produces $y_2$; an adversarial score check rejects it, so $h_2=h_1$. A later audit may restore $V_1$, but the costs of $y_1^a$, $y_1^b$, $y_2$, their executions, and the checks remain in $\mathbf C_{\rm total}$. Evaluating $h_1$ on fresh $Q_{\rm eval}$ tests the frozen state; it does not prove that the proposer learned a general solver.

Source observations. Harness Learning defines a harness as the executable program that organizes solver calls, tools, and information flow, trains a proposer in an outer loop, and uses execution feedback to revise a fixed solver harness in an inner loop. Its test-time proposer and solver stay fixed while the harness changes; the paper reports Reasoning Gym and multi-hop question-answering experiments, including held-out and unseen-task comparisons ([zhang2026harnesslearning, Sections 3-4.3 and 5-6]).

ReHarness reports a different intervention: under a strictly online stream, forward read and write are coupled with backward revision of reflections using the current trajectory. Its three continual-learning benchmarks and three backbones report source-grounded revisions and failures under distribution shift, but the result remains a benchmarked online protocol ([xiao2026reharness, Sections 3-5]). Stateless Language Agents likewise place durable research state in the harness and reconstruct role-specific contexts for stateless calls; their protected evaluation and token records motivate the provenance fields, while their mostly single-run settings and omitted experiment/evaluation compute limit the cost conclusion ([zhang2026statelessagents, Sections 3-5, Appendix B.1, and limitations]).

FTIP deduction and boundary. These observations support separating outer training, inner adaptation, state provenance, feedback, and lifecycle costs. The conceptual-discovery boundary in § [ftip-00MH] remains separate: these results do not establish a universal self-improvement law, transfer to frontier reasoning, or a monotone relation between more revisions and capability. The source results are finite empirical observations; the typed state record, leakage controls, complete cost decomposition, and rollback test are FTIP protocol proposals. The checked Forest, formal-math, TauCetiReview, and TauCetiWorker sources contain no declaration corresponding to these empirical harness mechanisms or their total-cost record; existing Forest state, replay, version, and lifecycle conventions are reused without adding a formalization claim.

4.1. Controlled reset of search state [ftip-00P0]

A lifecycle comparison must distinguish the state that proposes a candidate from the state that stores a candidate, its workspace, and its evaluation history. Fix an immutable verifier law/configuration $\mathcal V\in \mathbf V$. Write $s_t\in \mathcal S$, $h_t\in \mathcal H$, $w_t\in \mathcal W$, $b_t\in \mathcal B$, $q_t\in \mathcal Q$, and $\rho _t\in \mathcal G$. Let $\mathcal J$ be a finite set of lifecycle boundaries, with a recorded state $Z_r$ at every $r\in \mathcal J$ at which an arm may act. Let $\mathcal A$ be a finite arm-label set with maps $r:\mathcal A\to \mathcal J$ and $M:\mathcal A\to \mathcal P(\{s,h,w\})$.

Fix baseline coordinates $(s_0,h_0,w_0,b_0,q_0,\rho _0)$ in these spaces. The declared endowment $E_0$ from § 4 fixes the baseline implementation, split, permission, and initial-source state. The comparison also declares its verifier configuration, task law, and budget below; an arm is admissible only when its retained-best record $b_0$ and initial source $\rho _0$ are included in the declared endowment. A search state at round $t$ is

\[ Z_t=(s_t,h_t,w_t,b_t,q_t,\rho _t;\mathcal V), \]

where $s_t$ is the session context visible to the proposer, $h_t$ is the candidate-history record, $w_t$ is the workspace and filesystem snapshot, $b_t$ is the retained best candidate together with its score, artifact digest, and required verifier status, $q_t$ is mutable reducer state, and $\rho _t$ is the random-source state. The reset recomputes $q_0$ rather than carrying mutable reducer pointers. Candidate visibility and reducer behavior are part of the record: a candidate that is scored but not shown to a later proposer is not the same state as a candidate that remains in its context.

Let $\mathcal R$ be the declared joint paired randomization law. For every $r\in \mathcal J$ and nonempty mask $M\subseteq \{s,h,w\}$, $\mathcal R$ determines a measurable Markov kernel $K_{r,M}(\mathord \cdot \mid \rho _r)$ on $\mathcal G$; write $\rho '_r\sim K_{r,M}(\mathord \cdot \mid \rho _r)$ under that joint law and record the draw in the run receipt. The continue arm uses $\rho _r$ itself.

For each arm, the protocol declares a workspace materialization map $\operatorname {mat}:\mathcal B\to \mathcal W$ for the verified retained-best artifact. For a reset mask $M\subseteq \{s,h,w\}$, define the typed transition at boundary $r$ by

\[ \mathsf {Reset}_{r,M}(Z_r)= (s^M_r,h^M_r,w^M_r,b_r,q^M_r,\rho ^M_r;\mathcal V), \]

For $x\in \{s,h\}$, set $x^M_r=x_0$ when $x\in M$ and $x^M_r=x_r$ otherwise. Set $w^M_r=\operatorname {mat}(b_r)$ when $w\in M$ and $w^M_r=w_r$ otherwise. Set $q^M_r=q_0$ and $\rho ^M_r=\rho '_r$ when $M\ne \varnothing $, and $q^M_r=q_r$, $\rho ^M_r=\rho _r$ for the continue mask $M=\varnothing $. Thus session-only and history-only arms preserve $w_r$. For a workspace reset, $w^M_r$ is the declared materialization of $b_r$. It also requires the precondition $\operatorname {NoOverlap}(h_0,b_r)$ for each arm, meaning that the history namespace contains neither discarded history nor retained-best metadata duplicated in $b_r$; an arm that fails this predicate is inadmissible. The transition retains the declared best candidate $b_r$ and its content-addressed artifact. For a training-script task, runtime checkpoints and other generated outputs remain in $w_t$ and are cleared unless the protocol explicitly declares them part of $b_r$. The retained artifact and its materialization are counted explicitly as retained storage. The transition records the parent and child hashes, the discarded bytes and tokens, the reducer and visibility rules, and the random-source change. It does not erase the rejected proposals or their execution and evaluation costs from the lifecycle vector in Definition [ftip-005H]. A reset is therefore an intervention on state lifecycle, not evidence that the solver or proposer learned a new general algorithm.

The source observation in [tan2026forgetting, Abstract; Sections 3-5 and Tables 1-3] uses the Pi agent with DeepSeek V4.1 Flash on four bounded tasks spanning heuristic programming, kernel optimization, and model training, with the same budget in a single-agent run, four parallel agents, and a four-agent team that forgets three times; it reports three runs per task and configuration. Its session-and-history intervention carries forward the best candidate. That observation motivates, but does not test, the following paired FTIP decomposition arms:

continue all declared state;
clear session context only (new FTIP arm);
clear candidate history only (new FTIP arm); and
clear session, candidate history, and workspace state while retaining the best candidate, score, status, and artifact digest (new FTIP arm).

Use the scalar evaluation convention of Convention [ftip-005D]; write $Q:=Q_{\rm ev}$ and $\Lambda :=\Lambda _{\rm ev}$ for its fixed task and evaluation kernels. $U$ denotes the measurable scalar random variable obtained by evaluating the retained candidate with the declared inference protocol and evaluator draw. Higher $U$ is better; a minimization cost is negated before comparison, and incompatible task coordinates are kept as separate contrasts rather than silently pooled.

For each $a\in \mathcal A$, define $\mathsf {Reset}_a:=\mathsf {Reset}_{r(a),M(a)}$ and $\mathsf {Continue}_a:=\mathsf {Reset}_{r(a),\varnothing }$. Let $\mathbb P^{\rm pair}_a$ be the declared joint paired law induced by the fixed endowment $E_0$, verifier law $\mathcal V$, componentwise budget $\mathbf B\in \mathbb R_+^7$ from Definition [ftip-005H], task law and evaluation law $Q,\Lambda $, and $\mathcal R$. The declared measurable run/evaluation kernel $\mathsf K^{\rm run}_a$ maps each reset or continue state and its random-source draw to the final candidate, costs, and evaluation outcome; $\mathbb P^{\rm pair}_a$ is the joint law obtained by applying this kernel to the paired arms. Its reset and continue marginals are $\mathbb P^{\rm reset}_a$ and $\mathbb P^{\rm cont}_a$. Require $\mathbb E_{\mathbb P^{\rm reset}_a}|U|<\infty $ and $\mathbb E_{\mathbb P^{\rm cont}_a}|U|<\infty $; the primary finite estimand is

\[ \Delta _a=\mathbb E_{\mathbb P^{\rm reset}_a}[U] -\mathbb E_{\mathbb P^{\rm cont}_a}[U]. \]

The joint law $\mathbb P^{\rm pair}_a$ assigns paired runs, their random-source draws, and the declared coupling; it is not defined by the informal instruction to reuse a seed. The estimand is a marginal contrast, while the joint law supplies a paired estimator. A run reports the best-candidate utility, proposal and verifier outcomes, realized token/tool/storage cost, and the complete state and artifact hashes. A noisy or deceptive verifier is a future stress arm for this estimand; it is not an assumption that the reset is beneficial.

The source does not identify whether its gain comes from session length, candidate visibility, filesystem persistence, proposal diversity, or an interaction among them. The four-arm decomposition is an FTIP protocol proposal. Without the isolated arms, the source supports a bounded reset observation but not a theorem that memory is harmful, a monotone law for forgetting, or a general self-improvement claim.

Existing lifecycle, replay, and evaluation declarations cover subcomponents only; this typed reset transition remains an evaluation protocol proposal and does not claim a corresponding formal theorem.

4.2. Trace-quality intervention [ftip-00P4]

Long-horizon training needs two linked questions: which events must a trace preserve, and how can a selector be tested without confusing repeatability with coverage? The first question is answered by the trace schema and behavior vocabulary; the second is answered by a fixed-budget protocol.

The trace schema defines the record and its productive-persistence distinctions. The fixed-budget protocol defines the comparison arms, state interventions, metrics, and future-research gate.

4.2.1. Trace quality and productive persistence [ftip-00P2]

FTIP asks which parts of a long-horizon trace remain useful to a later decision. The source horizon and selective-refinement observations are finite conditional evidence, while the trace record, persistence predicates, and build-repair example below make the retained events and state boundary explicit [dang2026selecting, Sections 3-7].

A useful horizon preserves the declared decision, evidence, verification, and recovery events while separating repeatability from task and strategy coverage. The fixed-budget protocol in § 4.2.2 supplies the finite selection, receipt, and contrast definitions needed to test that distinction.

Remark 4.2.1.1. Source horizon and selection hypothesis [ftip-00PB]

The direct observation in [dang2026selecting, Abstract; Sections 3-5 and Tables 1-4] compares 4K, 8K, 12K, and 16K supervision horizons for terminal-agent SFT. In its reported Terminal-Bench v1.0 setting, 12K reaches more tasks than 16K with about thirty percent less training time. The shorter setting under-persisted, the longest setting over-persisted, and the intermediate setting produced the best reported balance. This is a finite terminal-agent observation under one model, teacher corpus, scaffold, and benchmark family; it does not establish a universal horizon, long-term memory, or persistent identity.

The paper's bias-complexity account gives the FTIP hypothesis [dang2026selecting, Sections 5.2 and 6-7; Tables 4-5, 10]. Truncation can increase temporal-supervision bias by removing later decisions, while a longer continuation increases the policy complexity and finite-sample burden. The useful horizon is therefore conditional on the task law, teacher policy, model capacity, verifier, and retained state.

Selective refinement makes the data intervention explicit [dang2026selecting, Sections 6-7; Tables 4-5, 10]. A warm-start model is trained on a shorter horizon, continuation segments are scored by teacher-forced likelihood, and a fixed fraction is retained for longer-horizon refinement. The score measures compatibility with the warm-start policy, not intrinsic truth. High-likelihood selection can improve repeatability; the FTIP protocol tests whether low-likelihood or random selection preserves breadth, rare strategies, and recovery behavior.

A run is premature when it terminates before the declared evidence or recovery condition, productive when it continues through bounded evidence gathering, verification, and recovery, and over-persistent when additional work repeats or explores after task-relevant value has saturated. Record coverage loss separately: a selector can improve repeatability while deleting hard tasks or unusual recovery paths. These terms describe observable behavior; they are not claims about an agent's memory or inner objective.

Definition 4.2.1.2. Trace record and state boundary [ftip-00P5]

A long-horizon training trace should preserve the behavior that a later agent must learn: its decisions, evidence, verification, recovery, and the state boundary visible to the continuation. Fix event domains $\mathcal O_{\rm evt}$, $\mathcal A_{\rm evt}$, $\mathcal Y_{\rm evt}$, $\mathcal E_{\rm evt}$, and $\mathcal V_{\rm evt}$, together with a termination domain $\mathcal T_{\rm evt}$ and a boundary domain $\mathcal S_{\rm boundary}$. The horizon comparison and selective-refinement intervention in [dang2026selecting, Sections 3-7] motivate this adaptation; the typed record below is an FTIP protocol proposal.

For $H\in \mathbb N$, a trace record is

\[ T=(o_{0:H},a_{0:H},y_{0:H},e_{0:H},v_{0:H},\tau ,\sigma ,\kappa ) \in \mathcal O_{\rm evt}^{H+1}\times \mathcal A_{\rm evt}^{H+1} \times \mathcal Y_{\rm evt}^{H+1}\times \mathcal E_{\rm evt}^{H+1} \times \mathcal V_{\rm evt}^{H+1}\times \mathcal T_{\rm evt} \times \mathcal S_{\rm boundary}\times \mathbb N. \]

The components $o$, $a$, and $y$ are observation, action, and outcome events; $e$ records evidence or an environment-state reference; $v$ records verification or recovery; $\tau $ records termination; $\sigma $ records the state boundary; and $\kappa $ is token length. The horizon $H$ and token length $\kappa $ are separate coordinates. A task may use a declared subtype of each event domain, but it must preserve the coordinate meaning.

The record is a training datum only together with its declared boundary $\sigma $: a continuation may access exactly the state coordinates allowed by that boundary. The record therefore supports a later selector or run kernel without treating token count as a proxy for the events that make continuation useful.

This is a protocol definition rather than a formal theorem; the event domains and transition semantics must be instantiated by the experiment that uses the record.

Definition 4.2.1.3. Persistence classes of a retained trace [ftip-00P6]

Fix a trace record $T$ from Definition 4.2.1.2 and a declared receipt interface with event indicators $e_{\rm term}(t)$, $e_{\rm ev}(t)$, $e_{\rm pass}(t)$, $e_{\rm fail}(t)$, $e_{\rm rec}(t)$ and a productive-novelty indicator $n_{\rm prod}(t)$ for $0\leq t\leq H$. Let $u(t)$ be the declared task-value score and $u^{\rm sat}$ its saturation threshold. The source's reported horizon tradeoff and selective refinement [dang2026selecting, Sections 5.2 and 6-7; Tables 4-5, 10] motivate these observable classes, which are an FTIP proposal. Write

\[ \tau _{\rm end}:=\inf \{t:e_{\rm term}(t)=1\},\quad \tau _{\rm ev}:=\inf \{t:e_{\rm ev}(t)=1\},\quad \tau _{\rm pass}:=\inf \{t:e_{\rm pass}(t)=1\}, \]\[ \tau _{\rm fail}:=\inf \{t:e_{\rm fail}(t)=1\},\quad \tau _{\rm rec}:=\inf \{t:e_{\rm rec}(t)=1\},\quad \tau _{\rm sat}:=\inf \{t:u(t)\geq u^{\rm sat}\}, \]

with $\inf \varnothing :=+\infty $. The receipt declares all indicators and the score as functions of the same run, so the following classes are replayable predicates rather than impressions about an agent's inner state.

\[ \operatorname {Prem}(T):=\mathbf 1[\tau _{\rm end}<\tau _{\rm ev}\lor (\tau _{\rm fail}<\tau _{\rm end}\land \tau _{\rm rec}\geq \tau _{\rm end})], \]\[ \operatorname {Prod}(T):=\mathbf 1[\tau _{\rm ev}<\tau _{\rm end}\land \tau _{\rm pass}<\tau _{\rm end}\land (\tau _{\rm fail}\geq \tau _{\rm end}\lor \tau _{\rm fail}<\tau _{\rm rec}<\tau _{\rm end})], \]\[ \operatorname {Over}(T):=\mathbf 1[\tau _{\rm sat}<\tau _{\rm end}\land \sum _{t=\tau _{\rm sat}}^{\tau _{\rm end}-1}n_{\rm prod}(t)=0]. \]

A productive trace reaches evidence and a final verifier pass, and any failure before termination is followed by recovery. An over-persistent trace continues after saturation without another declared productive event. The experiment may refine $n_{\rm prod}$ for a task family, but it must declare that predicate before comparing selectors.

Example 4.2.1.4. A retained build-repair trace [ftip-00P7]

The horizon and selective-refinement observations in [dang2026selecting, Sections 3-7] motivate the following finite classification. Consider a run whose task is to repair a failing build. The retained record contains the failing observation, the source edit, a test execution, and a final verifier pass. With the event interface of Definition 4.2.1.3, the test and verification steps contribute to $e_{\rm ev}$, $e_{\rm pass}$, and $n_{\rm prod}$, so the record can satisfy $\operatorname {Prod}(T)$.

If the test event is removed from the retained record, the same token budget can fail the productive predicate because the evidence chain is incomplete. If identical shell retries continue after $\tau _{\rm sat}$ with no new productive event, $\operatorname {Over}(T)$ can hold. This is a finite classification example; it does not claim that every short trace is better or that a model has an inner persistence objective.

4.2.2. Fixed-budget trace-selection protocol [ftip-00P3]

The trace-selection intervention is a finite evaluation proposal. It compares horizon and selector choices at a declared retained budget while keeping the model, teacher, scaffold, task split, verifier, randomization law, and cost coordinates visible. The selective-refinement study supplies the comparison pattern; the typed interface, receipts, and estimators below are FTIP adaptations rather than source results [dang2026selecting, Sections 6-7; Tables 4-5, 10].

The route separates the experiment interface (Definition 4.2.2.1), its receipt-derived quantities (Definition 4.2.2.2), and a finite paired calculation (Example 4.2.2.3). No result is claimed until an implementation supplies the declared receipts.

Definition 4.2.2.1. Finite trace-selection experiment [ftip-00P8]

The selective-refinement study compares fixed-horizon data interventions under a declared training and evaluation boundary [dang2026selecting, Sections 6-7; Tables 4-5, 10]. A trace-selection experiment is the following finite typed record.

Fix the P0 lifecycle arm set $\mathcal A$, its boundary map $r:\mathcal A\to \mathcal J$, mask map $M:\mathcal A\to \mathcal P(\{s,h,w\})$, and endowment $E_0$ from § 4.1 and § 4. Choose finite nonempty label sets

\[ \mathfrak H\subset \mathbb N,\qquad \mathfrak L,\qquad \mathfrak B\subset \mathbb N_{>0},\qquad \mathfrak D,\qquad \mathfrak R. \]

A split map returns a triple

\[ \operatorname {split}:\mathfrak D\longrightarrow \{(Q_{\rm sel},Q_{\rm hold},Q_{\rm xb})\mid Q_{\rm sel},Q_{\rm hold},Q_{\rm xb}\subseteq \mathcal Q\text { are finite}\}, \]

whose three outputs are pairwise disjoint and satisfy $|Q_{\rm sel}|>0$, $|Q_{\rm hold}|>0$; $Q_{\rm xb}$ may be empty only when the experiment records that no cross-benchmark comparison is made. A randomization map $\lambda _{\rm pair}:\mathfrak R\to \operatorname {Dist}(\mathcal G\times \mathcal G)$ supplies a joint source law for each paired comparison.

The experiment declares a finite, nonempty arm set $\mathfrak A\subseteq \mathcal A\times \mathfrak H\times \mathfrak L\times \mathfrak B\times \mathfrak D\times \mathfrak R$. For each $a\in \mathfrak A$, write its projections as

\[ (a_0(a),H_a,\ell _a,B_a,D_a,\eta _a),\qquad r_a:=r(a_0(a)),\quad M_a:=M(a_0(a)),\quad \mathcal R_a:=\lambda _{\rm pair}(\eta _a). \]

The arm record additionally supplies one model, teacher, scaffold, verifier, task law, inference protocol, and componentwise budget. These objects are maps of $a$, not implicit labels; paired arms must agree on every object except the factor named by their comparison. The split attached to $D_a$ is written $(Q_{\rm sel,a},Q_{\rm hold,a},Q_{\rm xb,a})$; it also supplies a nonempty finite strategy-label set $\mathfrak S_a$ and a receipt space $\mathcal R_a^{\rm rec}$ containing a typed held-out outcome $Y_a\in \mathcal Y_a\cup \{\bot \}$, a declared scalarization $g_a:\mathcal Y_a\to \mathbb R$, and an admission field. For every paired comparison $(a,a')$, the record additionally declares the finite pairing space $\mathcal P_{a,a'}:=(Q_{\rm hold,a}\cap Q_{\rm hold,a'}) \times \mathfrak S_a\times \mathfrak S_{a'}$ and a law $\Lambda _{a,a'}$ on $\mathcal G\times \mathcal G\times \mathcal P_{a,a'}$. The shared task coordinate in this law is the pairing key; the two strategy coordinates may differ.

For each arm, let $\mathcal X^{\rm sel}_a$ be the finite candidate-pool space with $1\leq B_a\leq |\mathcal X^{\rm sel}_a|$, and let $\mathcal F^{\rm sel}_a$ be the sigma-field generated by the selection-time prefix, teacher-forced score, data-only trace metadata, and $Q_{\rm sel,a}$. The metadata map excludes held-out outcomes, verifier outcomes, and post-selection continuation events. The selector is a measurable kernel with respect to this sigma-field

\[ \mathsf S_{a,\ell _a}:\mathcal X^{\rm sel}_a \longrightarrow \operatorname {Dist}\bigl (\{I\subseteq \mathcal X^{\rm sel}_a: |I|=B_a\}\bigr ). \]

It therefore returns a retained index set of the declared size without access to held-out outcomes. Write $\mathcal I_a:=\{I\subseteq \mathcal X^{\rm sel}_a:|I|=B_a\}$. A run kernel

\[ \mathsf K^{\rm run}_{a,\ell _a}: \mathcal X^{\rm sel}_a\times \mathcal I_a\times \mathcal G \times \mathcal Q\times \mathfrak S_a \longrightarrow \operatorname {Dist}(\mathcal R_a^{\rm rec}) \]

maps a candidate pool, its retained index set, one component of the paired source draw, a task, and its arm-specific strategy coordinate to a receipt with an event sequence, task and strategy labels, final verifier outcome, typed held-out outcome, admission field, state and artifact hashes, and the charged token, tool, storage, restart, discarded trace, and verification costs, with exactly $H_a+1$ event slots and a timeout receipt at the final slot when needed. For a paired comparison, $\Lambda _{a,a'}$ supplies a common held-out task and the two strategy coordinates, while $\mathcal R_a$ supplies the two source components before the kernels are applied. Thus an arm defines maps and kernels that use the selector output to produce replayable receipts; it does not merely list factors whose later meaning is left implicit.

The reset transition is $\mathsf {Reset}_{r_a,M_a}$ from § 4.1. Horizon, selector, budget, split, and randomization are therefore attached to a concrete P0 state boundary and a concrete run kernel. This is an empirical protocol definition, not a theorem or a claim that any selector improves behavior.

Definition 4.2.2.2. Receipt metrics and paired contrasts [ftip-00P9]

The source compares retained-horizon choices under declared task and evaluation boundaries [dang2026selecting, Sections 6-7; Tables 4-5, 10]. Fix an arm $a\in \mathfrak A$ from Definition 4.2.2.1 and a positive run count $N_a$. Each receipt $R_{a,i}$ exposes a finite event sequence $(e^{\rm term},e^{\rm ev},e^{\rm pass},e^{\rm fail},e^{\rm rec}, n^{\rm prod})_{t=0}^{H_a}$, a task label $q_{a,i}$, a strategy label $s_{a,i}$, a measurable task-value score $u_{a,i}(t)$, a declared saturation threshold $u_a^{\rm sat}$, and a typed held-out outcome $Y_{a,i}\in \mathcal Y_a\cup \{\bot \}$, final verifier outcome $v_{a,i}$, and complete-admission indicator $A_{a,i}$. Define $A_{a,i}:=\mathbf 1[q_{a,i}\in Q_{\rm hold,a}\land v_{a,i}=1 \land Y_{a,i}\in \mathcal Y_a]$; when $A_{a,i}=0$, set $Y_{a,i}:=\bot $. The receipt contract emits a terminal event by $H_a$; an execution that would not terminate naturally receives a timeout terminal event at $H_a$ and $A_{a,i}=0$. Put $\overline {\mathbb N}:=\mathbb N\cup \{+\infty \}$ and $\inf \varnothing :=+\infty $.

\[ \tau ^{\rm end}_{a,i}:=\inf \{t:e^{\rm term}_{a,i}(t)=1\},\quad \tau ^{\rm ev}_{a,i}:=\inf \{t:e^{\rm ev}_{a,i}(t)=1\},\quad \tau ^{\rm pass}_{a,i}:=\inf \{t:e^{\rm pass}_{a,i}(t)=1\}, \]\[ \tau ^{\rm fail}_{a,i}:=\inf \{t:e^{\rm fail}_{a,i}(t)=1\},\quad \tau ^{\rm rec}_{a,i}:=\inf \{t:e^{\rm rec}_{a,i}(t)=1\},\quad \tau ^{\rm sat}_{a,i}:=\inf \{t:u_{a,i}(t)\geq u_a^{\rm sat}\}. \]

Define the receipt predicates

\[ \begin {aligned} I^{\rm pre}_{a,i}&:=\mathbf 1[\tau ^{\rm end}_{a,i}<\tau ^{\rm ev}_{a,i}],\\ I^{\rm rec}_{a,i}&:=\mathbf 1[\tau ^{\rm fail}_{a,i}<\tau ^{\rm rec}_{a,i}<\tau ^{\rm end}_{a,i}],\\ I^{\rm prod}_{a,i}&:=\mathbf 1[\tau ^{\rm ev}_{a,i}<\tau ^{\rm end}_{a,i} \land \tau ^{\rm pass}_{a,i}<\tau ^{\rm end}_{a,i} \land (\tau ^{\rm fail}_{a,i}\geq \tau ^{\rm end}_{a,i} \lor I^{\rm rec}_{a,i}=1)],\\ I^{\rm over}_{a,i}&:=\mathbf 1[\tau ^{\rm sat}_{a,i}<\tau ^{\rm end}_{a,i} \land \sum _{t=\tau ^{\rm sat}_{a,i}}^{\tau ^{\rm end}_{a,i}-1} n^{\rm prod}_{a,i}(t)=0]. \end {aligned} \]

Here $n^{\rm prod}_{a,i}(t)$ is a declared productive-novelty indicator in the receipt, and $I^{\rm rec}$ is zero when no failure precedes recovery. The five run rates are functions of the receipts, with the declared denominator $N_a$:

\[ (\widehat p_a^{\rm pre},\widehat p_a^{\rm prod},\widehat p_a^{\rm over}, \widehat p_a^{\rm rec},\widehat p_a^{\rm pass}) :=\frac 1{N_a}\sum _{i=1}^{N_a} (I^{\rm pre}_{a,i},I^{\rm prod}_{a,i},I^{\rm over}_{a,i}, I^{\rm rec}_{a,i},\mathbf 1[\tau ^{\rm pass}_{a,i}<\tau ^{\rm end}_{a,i}]). \]

For the finite held-out set, task and strategy coverage are

\[ \widehat C^{\rm task}_a:= \frac {|\{q\in Q_{\rm hold,a}:\exists i,\ q_{a,i}=q,\ A_{a,i}=1\}|} {|Q_{\rm hold,a}|},\qquad \widehat C^{\rm strat}_a:= \frac {|\{s\in \mathfrak S_a:\exists i,\ s_{a,i}=s,\ A_{a,i}=1\}|} {|\mathfrak S_a|}, \]

where $\mathfrak S_a$ is a declared nonempty strategy-label set. Every receipt reports both positive denominators and the admission rule.

For two paired arms $a$ and $a'$ that differ in one declared factor, let $N:=\min (N_a,N_{a'})$. Define the common admitted index set $J_{a,a'}:=\{i\leq N:A_{a,i}=A_{a',i}=1\}$ and $N_{\rm pair}:=|J_{a,a'}|$. For $i\in J_{a,a'}$, let $Y_{a,i}$ and $Y_{a',i}$ be their admitted held-out outcomes under the shared pairing law $\Lambda _{a,a'}$, so $q_{a,i}=q_{a',i}$. Compare their declared scalarizations $g_a(Y_{a,i})$ and $g_{a'}(Y_{a',i})$. When $N_{\rm pair}\geq 2$, the primary finite contrast and its paired sample variance are

\[ \widehat \Delta _{a,a'}:= \frac 1{N_{\rm pair}}\sum _{i\in J_{a,a'}} \bigl (g_a(Y_{a,i})-g_{a'}(Y_{a',i})\bigr ),\qquad \widehat V_{a,a'}:= \frac 1{N_{\rm pair}-1}\sum _{i\in J_{a,a'}} \bigl ((g_a(Y_{a,i})-g_{a'}(Y_{a',i}))-\widehat \Delta _{a,a'}\bigr )^2. \]

When $N_{\rm pair}=1$, report the single paired contrast but set $\widehat V_{a,a'}:=\bot $; when $N_{\rm pair}=0$, the paired contrast is undefined and the missingness is reported. These are finite estimators under the declared arm, split, admission rule, and coupling; they are not a universal selector law.

Example 4.2.2.3. A paired trace-selection calculation [ftip-00PA]

The selective-refinement comparison motivates contrasting repeatability and coverage under a fixed retained budget [dang2026selecting, Sections 6-7; Tables 4-5, 10]. Consider two arms from Definition 4.2.2.1 with the same P0 boundary, model, task split, verifier, budget, and paired assignment law. They differ only in the selector: $a$ retains the highest teacher-forced scores and $a'$ retains a coverage-stratified subset. Both have $Q_{\rm hold}=\{q_1,q_2\}$ and $\mathfrak S=\{s_1,s_2\}$. The paired assignment law gives the same held-out task at indices 1 and 2; arm $a'$ has one additional admitted coverage run at index 3. Take $g_a=g_{a'}=\operatorname {id}_{\{0,1\}}$. The receipts are

\[ \begin {array}{c|cc|ccc} &q_{a,i}&s_{a,i}&A_{a,i}&I^{\rm prod}_{a,i}&Y_{a,i}\\\hline i=1&q_1&s_1&1&1&1\\ i=2&q_1&s_1&1&1&0 \end {array} \qquad \begin {array}{c|cc|ccc} &q_{a',i}&s_{a',i}&A_{a',i}&I^{\rm prod}_{a',i}&Y_{a',i}\\\hline i=1&q_1&s_1&1&1&1\\ i=2&q_1&s_1&1&1&1\\ i=3&q_2&s_2&1&1&1 \end {array} \]

The first two rows are the common paired indices, so $N_a=2$, $N_{a'}=3$, and $N_{\rm pair}=2$. Both arms have $\widehat p^{\rm prod}=1$; the third row is included in arm $a'$'s coverage and run-rate denominators but not in the paired contrast. Thus $\widehat C^{\rm task}_a=\widehat C^{\rm strat}_a=\tfrac 12$ and $\widehat C^{\rm task}_{a'}=\widehat C^{\rm strat}_{a'}=1$, while $\widehat \Delta _{a,a'}=-\tfrac 12$. The calculation shows why a repeatability rate alone cannot identify coverage: the two selectors have the same productive rate but different admitted task and strategy support. This is a finite FTIP construction using the typed receipt and pairing definitions, not a source result.

With the trace schema and the fixed-budget protocol now in scope, the FTIP boundary is operational: an admitted trace must preserve the declared decision, evidence, verification, and recovery events, and every selector comparison must report the finite run rates, held-out task and strategy coverage, and charged selection/storage/restart costs defined in § 4.2.2. A result may claim a selection benefit only when its immutable arm, split, seed, and randomization receipt supports that comparison.

References

Reference. Harness Learning Enables Generalizable Test-Time Adaptation [zhang2026harnesslearning]

2026
Alvin Zhang, Xuecheng Liu, Zixuan Wang, Fahim Tajwar, Daman Arora, Ruslan Salakhutdinov, Daniel Khashabi, Yuda Song, Andrea Zanette
https://arxiv.org/abs/2609.35738

@misc{zhang2026harnesslearning,
  title = {Harness Learning Enables Generalizable Test-Time Adaptation},
  author = {Zhang, Alvin and Liu, Xuecheng and Wang, Zixuan and Tajwar, Fahim and Arora, Daman and Salakhutdinov, Ruslan and Khashabi, Daniel and Song, Yuda and Zanette, Andrea},
  year = {2026},
  eprint = {2609.35738},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.35738}
}

Reference. Prefix Sliding for efficient test-time scaling [muennighoff2026prefixsliding]

2026
Niklas Muennighoff, Zhengyang Wang, Zeyi Chen, Weijia Shi, Binyuan Hui, John Yang, Dapeng Jiang, Mika Senghaas, Fares Obeid, Johannes Hagemann, Sami Jaghouar, Ludwig Schmidt, Percy Liang, Jason Wei, Andrew Y. Ng, Luke Zettlemoyer, Yejin Choi, Mike Lewis
https://arxiv.org/abs/2608.26070v1

@article{muennighoff2026prefixsliding,
  title = {Prefix Sliding for efficient test-time scaling},
  author = {Muennighoff, Niklas and Wang, Zhengyang and Chen, Zeyi and Shi, Weijia and Hui, Binyuan and Yang, John and Jiang, Dapeng and Senghaas, Mika and Obeid, Fares and Hagemann, Johannes and Jaghouar, Sami and Schmidt, Ludwig and Liang, Percy and Wei, Jason and Ng, Andrew Y. and Zettlemoyer, Luke and Choi, Yejin and Lewis, Mike},
  year = {2026},
  url = {https://arxiv.org/abs/2608.26070v1},
  journal = {arXiv preprint arXiv:2608.26070}
}

Reference. Recurrent Harness for Agentic Continual Learning [xiao2026reharness]

2026
Yuxin Xiao, Zhiyun Lu, Jianyu Wang, Yangtian Zhang, Yang Song, Qifan Wang, Bailin Wang
https://arxiv.org/abs/2610.12668

@misc{xiao2026reharness,
  title = {Recurrent Harness for Agentic Continual Learning},
  author = {Xiao, Yuxin and Lu, Zhiyun and Wang, Jianyu and Zhang, Yangtian and Song, Yang and Wang, Qifan and Wang, Bailin},
  year = {2026},
  eprint = {2610.12668},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.12668}
}

Reference. Scaling Discovery Requires Forgetting [tan2026forgetting]

2026
Shangyin Tan, Ben Zhou, Lakshya A. Agrawal, Jialin Zhang, Yilin Chen, Qiuyang Mang, Donghyun Lee, Dennis Jacob, Koushik Sen, Alex Dimakis, and Matei Zaharia
https://arxiv.org/abs/2610.20563

@misc{tan2026forgetting,
  title = {Scaling Discovery Requires Forgetting},
  author = {Tan, Shangyin and Zhou, Ben and Agrawal, Lakshya A. and Zhang, Jialin and Chen, Yilin and Mang, Qiuyang and Lee, Donghyun and Jacob, Dennis and Sen, Koushik and Dimakis, Alex and Zaharia, Matei},
  year = {2026},
  eprint = {2610.20563},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.20563}
}

Reference. Selecting Long-Horizon Trajectories for Reliable and Efficient Terminal-Agent Training [dang2026selecting]

2026
Cuong Dang, Hoang Anh Just, Ruoxi Jia
https://arxiv.org/abs/2610.05831v1

@misc{dang2026selecting,
  title = {Selecting Long-Horizon Trajectories for Reliable and Efficient Terminal-Agent Training},
  author = {Dang, Cuong and Just, Hoang Anh and Jia, Ruoxi},
  year = {2026},
  eprint = {2610.05831},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.05831v1}
}

Reference. Stateless Language Agents: Scaling Long-Horizon Automated Research [zhang2026statelessagents]

2026
Qizheng Zhang, Changxiu Ji, Isaac Sun, Yuetai Li, Shubhangi Upasani, Sherry Ruan, Boyuan Ma, Fenglu Hong, Vamsidhar Kamanuru, Yoonho Lee, Yuzhen Mao, Genghan Zhang, Rulin Shao, Qiuyang Mang, Andy Dimnaku, Changran Hu, Radha Poovendran, Kunle Olukotun
https://arxiv.org/abs/2610.07625

@misc{zhang2026statelessagents,
  title = {Stateless Language Agents: Scaling Long-Horizon Automated Research},
  author = {Zhang, Qizheng and Ji, Changxiu and Sun, Isaac and Li, Yuetai and Upasani, Shubhangi and Ruan, Sherry and Ma, Boyuan and Hong, Fenglu and Kamanuru, Vamsidhar and Lee, Yoonho and Mao, Yuzhen and Zhang, Genghan and Shao, Rulin and Mang, Qiuyang and Dimnaku, Andy and Hu, Changran and Poovendran, Radha and Olukotun, Kunle},
  year = {2026},
  eprint = {2610.07625},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.07625}
}

Context

Agent state, computation, and reliability [ftip-00JD]

An agent's behavior can change through persistent state, retained context and recursive execution even when its model weights are fixed. These mechanisms complement post-training, and their costs and retained artifacts belong to the same capability comparison.

The chapter begins with persistent state and lifecycle accounting, then examines dynamic computation and recursive harnesses. Failure modes and reliability protocols explain when an apparent gain can be trusted. Evaluation transport is developed with the independent evaluation framework that follows.

Persistent harness state and lifecycle-bounded inference [ftip-00C2]

An executable model can remain fixed while a harness changes the context, available computation, retained memories, skills, subagent definitions, and evaluation policy around it. Those changes can alter later behavior and cost, but they are not weight training.

Distinguishing harness state from represented weights makes replay and descendant accounting precise, and exposes the effects of lossy summaries and contaminated refinement.

[karten2026prime] supplies operational records for persistent harness state. The Grothendieck-constant case study of [li2026longhorizon] supplies a long-horizon record in which compressed research state lost a caveat and a feasibility condition. Source observations remain empirical; the displayed finite results are proved here.

Fixed-weight harness adaptation [ftip-00C3]

The executable model artifact, active context, explicitly managed computation, and persistent harness state are distinct parts of an agent. Replayability depends on the rule that materializes state from events, as well as the recorded inputs and execution versions.

Definition. Parameter, context, compute, and persistent state [ftip-00C4]

Let $\mathcal C_{\rm act}$ be an active-context space, $\mathcal Z_{\rm man}$ an explicitly managed computation-state space, and $\mathcal H_{\rm pers}$ a persistent harness-state space. A four-layer harness state is

\[ \Xi =(M,c,z,h)\in \mathcal M_{\rm exec}\times \mathcal C_{\rm act} \times \mathcal Z_{\rm man}\times \mathcal H_{\rm pers}. \]

The coordinates are, respectively, the executable model artifact, the token-visible context, explicitly managed values or sessions, and state that can survive the present invocation. A transition may change any declared subset of these coordinates. A fixed-weight harness transition holds $M$ fixed while changing one or more of $c,z,h$.

Remark. State layers are access mechanisms, not capability levels [ftip-00C5]

Prime Agent Section 2.2 names model weights, active context, explicitly managed computation, and retained state as levels L0--L3, and describes the operations that move information between them [karten2026prime, Section 2.2]. The formulation in Definition [ftip-00C4] preserves that operational separation without treating the labels as an ordering of intelligence or capability.

Two runs with identical weights and different persistent state can induce different policies. That observation does not say that either run acquired a new circuit in the fixed artifact, and it does not compare their independently evaluated utility.

Definition. Versioned harness configuration [ftip-00C6]

Let $\mathcal I_{\rm har}$ be a content-identifier space. A versioned harness configuration is a finite record $\kappa ^{\rm har}\in \mathcal I_{\rm har}$ that resolves the active-context assembly rule, managed-computation interface, tool permissions, session and message semantics, compaction policy, persistent-state schema, refinement policy, recovery rule, and implementation versions.

Together with a state $\Xi $, the record resolves a non-anticipating harness policy. Changing one field produces a different declared intervention even when the executable model artifact is unchanged. The record does not include an evaluation binding; that is a separate record coordinate.

Definition. Persistent event stream [ftip-00C7]

Fix a finite event alphabet $\mathcal E_{\rm har}$. A persistent event stream of length $n$ is

\[ L_n=(e_0,\ldots ,e_{n-1})\in \mathcal E_{\rm har}^{n}. \]

The append operation is $L_{n+1}=\operatorname {append}(L_n,e_n)$. Write $L_i\preceq _{\rm pref}L_j$ when $L_i$ is a prefix of $L_j$. An append-only stream satisfies $L_i\preceq _{\rm pref}L_j$ for every $i\leq j$; branching creates distinct continuations with a common prefix rather than rewriting that prefix.

Events may record model or tool calls, messages, interventions, retries, verifier outcomes, harness edits, and resource use. The event schema and its exact version belong to the configuration of Definition [ftip-00C6].

Definition. Materialized harness state [ftip-00C8]

Let $\mathcal V_{\rm fold}$ be a space of fold versions. For each $v\in \mathcal V_{\rm fold}$, fix a deterministic map

\[ F_v:\mathcal H_{\rm pers}\times \mathcal E_{\rm har} \longrightarrow \mathcal H_{\rm pers}. \]

Given an initial persistent state $h_0$, event stream $L_n=(e_0,\ldots ,e_{n-1})$, and version sequence $\boldsymbol v=(v_0,\ldots ,v_{n-1})$, its materialized harness state is obtained recursively by

\[ h_{i+1}=F_{v_i}(h_i,e_i),\qquad 0\leq i<n. \]

The materialization record is the tuple $(h_0,L_n,\boldsymbol v,(F_v)_v)$. Omitting a version or the initial state defines a family of possible materializations, not one replayable state.

Theorem. Deterministic event folding gives replayable state [ftip-00C9]

For a materialization record in Definition [ftip-00C8], replaying the same initial state, event sequence, version sequence, and fold maps produces the same state $h_n$.

This finite result follows from the displayed hypotheses. It assumes exact equality of all recorded inputs and deterministic fold maps.

Proof.

At step zero both replays have state $h_0$. If their states agree at step $i$, both apply the same function $F_{v_i}$ to the same pair $(h_i,e_i)$, so their states agree at step $i+1$. Finite induction gives equality at step $n$.

Remark. Replayability is not correctness [ftip-00CA]

The result in Theorem [ftip-00C9] is an identity about a declared transition system. It does not show that the event stream is complete, that an external process can be reconstructed, that the fold is faithful to the environment, or that the resulting state is useful or safe.

Prime Agent reports append-only events, versioned state, recovery, and rollback in Sections 2.2 and 2.5 [karten2026prime, Sections 2.2 and 2.5]. Those implementation claims motivate the event record; Theorem [ftip-00C9] proves the finite replay identity directly from the declared transition rule.

Example. One artifact and two harness policies [ftip-00CB]

The same executable artifact can be paired with two persistent states and therefore two resolved harness policies. The harness state changes while the executable model remains fixed.

$\text{[math]}$

Different policies here establish only that harness state is an intervention coordinate. An independent evaluation is still required to compare their outcomes.

Remark. Fixed-weight self-improvement is not weight training [ftip-00CC]

Prime Agent Section 2.5 uses the phrase ``self-improvement'' for execution evidence converted into persistent prompts, memories, skills, or subagent specifications while model weights remain fixed [karten2026prime, Section 2.5]. In the notation of Definition [ftip-00C4], this is a change to $h$ with $M$ held fixed.

The term therefore does not establish a change to the model artifact, a new learned circuit, or capability acquisition in the sense of Definition [ftip-0007]. It describes persistent harness adaptation.

Evaluation binding and lineage accounting [ftip-00CD]

A long-horizon score is attached to a configuration, not merely to a model name. Delegation also creates descendant work that must remain visible in the resource account. Reproducible evaluation therefore requires a specified configuration, and resource accounting includes the work of its descendants.

Definition. Evaluation configuration record [ftip-00CE]

An evaluation configuration record is a finite record $\mathfrak e$ that resolves the task and instance law, environment and tool interfaces, executable model artifact, serializer and inference policy, harness configuration, compaction and refinement policies, retry rule, completion gate, evaluator, seed law, and componentwise resource limits.

Together with the independent evaluation interface of Convention [ftip-005D], the record determines the law of a stopped evaluation run and its realized cost vector. A reported score is the pair consisting of the statistic and the exact record $\mathfrak e$; a scalar without its record is an under-specified family of measurements.

Remark. A harness comparison needs a declared configuration axis [ftip-00CF]

Section 2.6 of [karten2026prime] binds task and tool interfaces to model settings. Its evaluation configuration also records compaction, refinement, retry, completion, and resource policies. The record in Definition [ftip-00CE] is a proposed representation of those configuration choices.

To attribute a score difference to a harness coordinate, the comparison must hold the other coordinates fixed or model their changes explicitly. Two scores under different records $\mathfrak e$ and $\mathfrak e'$ can still be descriptively useful, but their difference does not isolate a causal harness effect.

Definition. Session lineage tree [ftip-00CG]

A session lineage tree is a finite rooted directed tree $\mathcal T=(V,E,r)$ whose edges point from a parent session to a directly spawned descendant. Each node $v\in V$ carries an immutable session identifier, its resolved harness and inference stamps, a local event stream, and a local realized cost $c_v\in \mathbb R_+^m$.

For a node $v$, let $\mathcal T_v$ be the induced subtree containing $v$ and all its descendants. Distinct children of one node have disjoint node sets. Messages between branches are events in their local streams and do not merge their identities or costs.

Definition. Descendant-complete cost [ftip-00CH]

For a session lineage tree Definition [ftip-00CG], its descendant-complete cost is the componentwise sum

\[ C_{\rm lin}(\mathcal T)=\sum _{v\in V}c_v\in \mathbb R_+^m. \]

The local cost $c_v$ contains only work assigned to node $v$; a model or tool call is charged to exactly one node. The vector retains its declared units and embeds into the lifecycle account of Definition [ftip-005H]. A scalar price can be applied only after the vector is formed.

Theorem. Lineage cost is additive over disjoint child subtrees [ftip-00CI]

Let the root $r$ of a finite lineage tree have children $v_1,\ldots ,v_k$. Then

\[ C_{\rm lin}(\mathcal T)=c_r+ \sum _{j=1}^{k}C_{\rm lin}(\mathcal T_{v_j}). \]

This finite accounting identity follows from assigning each local cost to exactly one node.

Proof.

The node set is the disjoint union of $\{r\}$ and the node sets of the child subtrees. Splitting the finite sum in Definition [ftip-00CH] over that disjoint union gives the equality componentwise.

Definition. Budget-admissible continuation [ftip-00CJ]

Fix a componentwise hard budget $b\in \mathbb R_+^m$. After accumulated cost $a\preceq b$, a proposed continuation has a declared worst-case cost bound $\bar c\in \mathbb R_+^m$. It is budget-admissible when

\[ a+\bar c\preceq b. \]

If admitted, the continuation must either stop within a realized cost $c\preceq \bar c$ or report a contract violation. Rejected proposals may have a separately accounted proposal cost, which must already be included in $a$ before the admission test.

Theorem. Budget admission preserves a hard componentwise bound [ftip-00CK]

Consider a finite sequence of continuations. Start from $a_0\preceq b$. At step $i$, admit only if $a_i+\bar c_i\preceq b$, and require the realized cost to satisfy $c_i\preceq \bar c_i$. With $a_{i+1}=a_i+c_i$, every accumulated cost satisfies $a_i\preceq b$.

This finite result is a contract theorem, not a prediction that a real executor respects its declared bound.

Proof.

The claim holds for $a_0$. If $a_i\preceq b$ and the next step is admitted, then $a_{i+1}=a_i+c_i\preceq a_i+\bar c_i\preceq b$. If the step is rejected, the accumulated cost is unchanged after its already-accounted proposal work. Finite induction proves the claim.

Example. A finite delegated run and its lineage cost [ftip-00CL]

Use two cost coordinates, such as output tokens and tool calls. The root delegates to two children, and one child delegates once more.

$\text{[math]}$

The cost of a child subtree is its local cost plus the costs of all descendants. Therefore

\[ C_{\rm lin}(\mathcal T)=c_r+C_{\rm lin}(\mathcal T_a)+C_{\rm lin}(\mathcal T_b). \]

Charging only the root hides descendant work; charging a descendant again as a separate parent-local cost double counts it.

Remark. External benchmark points do not isolate a harness effect [ftip-00CM]

Prime Agent Section 3.1 places native-harness runs beside externally reported ARC-AGI-3 results and explicitly says the external points situate the curves rather than isolate a causal harness effect [karten2026prime, Section 3.1]. That is an empirical comparison under multiple evaluation records, not a matched estimate of one harness field.

A larger score can still be a valid record of the displayed configuration. It does not by itself establish that persistence caused the difference, that the model weights improved, or that the cost-adjusted potential of Definition [ftip-005M] increased.

Research-state compaction and observability [ftip-00CN]

A research harness cannot place its complete archive into every model invocation. A summary can make distinct archives indistinguishable to later evaluators. The source record of a lost evaluator caveat illustrates how omitted information can affect subsequent research.

Definition. Active research state [ftip-00CO]

Let $\mathcal A_{\rm res}$ be a space of complete research archives and $\mathcal S_{\rm res}$ a space of bounded working summaries. Let $\mathcal Q_{\rm claim}$ be a typed claim-ledger space and $\mathcal G_{\rm res}$ a research-goal space. An active research state is

\[ R=(A,s,q,g)\in \mathcal A_{\rm res}\times \mathcal S_{\rm res} \times \mathcal Q_{\rm claim}\times \mathcal G_{\rm res}. \]

The archive $A$ may exceed the context budget. The summary $s$ is the representation actually supplied to a new bounded session. The claim ledger $q$ records declared statuses such as proved, numerically supported, conjectural, or heuristic. The goal $g$ records the currently selected research direction.

Definition. Technical executor and research-judgment kernel [ftip-00CP]

Let $\mathcal P_{\rm res}$ be a proposal space and $\mathcal D_{\rm res}$ a finite research-decision space. A technical executor is a kernel that produces candidate calculations, experiments, lemmas, or implementations from the active summary and goal. A research-judgment kernel is

\[ J:\mathcal S_{\rm res}\times \mathcal Q_{\rm claim} \times \mathcal G_{\rm res}\times \mathcal P_{\rm res} \longrightarrow \Delta (\mathcal D_{\rm res}). \]

The kernel chooses among actions such as continue, reframe, verify, merge, withdraw, or stop. It is typed separately from the executor because producing a technically valid local step and choosing the globally useful next step are different intervention coordinates.

Definition. Research-state summary operator [ftip-00CQ]

A research-state summary operator is a declared map

\[ C:\mathcal A_{\rm res}\longrightarrow \mathcal S_{\rm res}. \]

At a session boundary the harness supplies $s=C(A)$. The operator may select, merge, compress, or omit archive content. Its input archive and exact version belong to the persistent event record. A stochastic summarizer is represented by adjoining its random seed to the archive coordinate, leaving $C$ deterministic on the augmented input.

Definition. Summary-equivalent research archives [ftip-00CR]

For the summary operator $C$ of Definition [ftip-00CQ], two complete archives $A,A'\in \mathcal A_{\rm res}$ are summary-equivalent, written $A\sim _C A'$, when

\[ C(A)=C(A'). \]

This is equivalence relative to one declared summary version. It is not the protocol-level observational equivalence of Definition [ftip-007R]: the complete archives can differ in facts that a later retrieval operator or independent evaluator can still observe.

Theorem. A summary-only harness cannot distinguish summary-equivalent archives [ftip-00CS]

Fix the claim ledger $q$, goal $g$, and proposal $p$. Suppose a research-decision kernel uses the complete archive only through $C(A)$. If $A\sim _C A'$, then its decision laws under $A$ and $A'$ are equal.

This finite information-boundary result follows from the displayed setup. It does not assume that the two complete archives induce the same independent utility.

Proof.

By hypothesis, the two decision laws are related by

\[ J(C(A),q,g,p)=J(C(A'),q,g,p). \]

Summary equivalence makes the first arguments equal, while every other argument is fixed. The two probability laws are therefore identical.

Lemma. An omitted constraint cannot affect a summary-only decision [ftip-00CT]

Let $b:\mathcal A_{\rm res}\to \{0,1\}$ be a constraint bit. If there exist $A\sim _C A'$ with $b(A)\neq b(A')$, then no decision rule that factors only through $C$ can condition its output law on the value of $b$ for both archives.

Proof.

The result in Theorem [ftip-00CS] gives the same output law for $A$ and $A'$. A rule that conditioned on the differing bit values would require different output laws for at least one declared decision event. Both requirements cannot hold simultaneously.

Example. A lost evaluator caveat reverses admissibility [ftip-00CU]

Take two complete archives $A_{\rm exp}$ and $A_{\rm cert}$. Both contain the same numerical score and candidate record. The first additionally states that the evaluator is safe only for exploration; the second states that the score is independently certified. Let the summary operator omit that status sentence, so $A_{\rm exp}\sim _C A_{\rm cert}$.

The independently correct decision is ``audit'' for $A_{\rm exp}$ and ``merge'' for $A_{\rm cert}$. A summary-only kernel has the same decision law in both cases by Theorem [ftip-00CS]; hence it cannot be correct on both archives with probability one.

This finite witness models an information loss. It does not assert that every compaction loses a decisive caveat.

Definition. Full-log recovery witness [ftip-00CV]

For a constraint bit $b$ omitted by $C$, a full-log recovery witness is a query $u$, retrieval map

\[ R_u:\mathcal A_{\rm res}\longrightarrow \mathcal O_u, \]

and decoder $d_u:\mathcal O_u\to \{0,1\}$ such that $d_u(R_u(A))=b(A)$ on the declared archive class. A harness that invokes this retrieval before judgment can condition on $b$; a harness restricted to $C(A)$ cannot when the hypothesis of Lemma [ftip-00CT] holds.

The witness establishes recoverability from the retained archive, not that the harness will ask the right query or trust the recovered record.

Example. From evaluator caveat to withdrawn record [ftip-00CW]

The Grothendieck case study gives a concrete chronology. An early evaluator carried an exploration-only caveat; the working summary later lost that caveat and a feasibility condition; a later session recorded an unsupported upper bound; a subsequent audit withdrew it.

$\text{[math]}$

The source reports this sequence in Section 5 and Section 7.1; the complete archive retained the facts while the compressed decision state did not [li2026longhorizon, Section 5 and Section 7.1]. The diagram is a source-grounded chronology, not a measured error rate for compaction systems.

Remark. More inference compute is not a monotone research-judgment theorem [ftip-00CX]

The source run used roughly 240 sessions, 2,091 reasoning-model calls, and 152 million tokens. It also reports repeated continuation of an upper-bound search until human operators redirected the program toward a universal obstruction [li2026longhorizon, Sections 5--7].

This is one human-steered case with changing models, harnesses, goals, and research state. It supports the distinction between technical execution and research judgment. It does not establish a monotone or anti-monotone law from inference compute to mathematical progress.

Refinement, contamination, and stop rules [ftip-00CY]

Persistent refinement can retain useful procedures, but the same mechanism can retain a specification exploit. A proposed change and committed state have different consequences. Audit predicates can exclude specified contamination, rollback can restore an earlier state, and resource admission rules can prevent transitions that exceed the budget.

Definition. Refinement proposal and committed harness update [ftip-00CZ]

Let $h_n\in \mathcal H$ be the committed harness state at version $n$, let $e_n\in \mathcal E$ be a newly admitted event, and let $\omega _n\in \Omega _{\rm ref}$ be a refinement seed. A refinement proposal is

\[ \widetilde h_{n+1}=R(h_n,e_n,\omega _n), \qquad R:\mathcal H\times \mathcal E\times \Omega _{\rm ref}\longrightarrow \mathcal H. \]

Given a typed decision $d_n\in \{\mathsf {accept},\mathsf {reject}\}$, the committed harness update is

\[ h_{n+1}= \begin {cases} \widetilde h_{n+1},&d_n=\mathsf {accept},\\ h_n,&d_n=\mathsf {reject}. \end {cases} \]

This separates candidate generation from state mutation. A proposed note, skill, prompt, or subagent specification has no persistent effect until the commit decision accepts it.

Definition. Skill archive and selector [ftip-00D0]

Let $\mathcal S_{\rm skill}$ be a skill space and $\mathfrak A_{\rm skill}$ the space of finite skill--provenance archives. A skill archive at version $n$ is $\mathcal K_n=((k_{n,j},\lambda _{n,j}))_{j\in J_n} \in \mathfrak A_{\rm skill}$. For a public history $h^{\rm pub}$ and selector seed $\omega ^{\rm sel}$, a typed selector is

\[ S_{\rm skill}: \mathcal H^{\rm pub}\times \mathfrak A_{\rm skill}\times \Omega _{\rm sel} \longrightarrow \mathcal S_{\rm skill}\cup \{\bot \}. \]

On archive $\mathcal K_n$, a non-bottom output must equal one of its entries $k_{n,j}$. The value $\bot $ means that no retained skill is invoked. The archive is part of persistent harness state, not a claim that its entries are correct, safe, novel, or encoded in model weights.

Definition. Exploit-contaminated retained state [ftip-00D1]

Fix a declared task contract and let $b:\mathcal S_{\rm skill}\to \{0,1\}$ mark a retained skill as an exploit when it can increase the recorded proxy while violating that contract. A harness state $h_n$ with archive $\mathcal K_n$ is exploit-contaminated when

\[ B(h_n)=\max _{j\in J_n} b(k_{n,j})=1, \]

with the maximum defined as zero for an empty archive. The predicate is relative to the declared contract and audit model. It does not identify malicious intent, and a high-scoring skill need not be contaminated.

Lemma. Append-only retention preserves contamination absent deletion [ftip-00D2]

Suppose the skill archives of Definition [ftip-00D0] are append-only: $\mathcal K_n\subseteq \mathcal K_{n+1}$ for every $n$. If $B(h_n)=1$, then $B(h_m)=1$ for every $m\geq n$ until a deletion, rollback, or contract change removes or reclassifies the witnessing skill.

Proof. Choose $k\in \mathcal K_n$ with $b(k)=1$. Repeated inclusion gives $k\in \mathcal K_m$ for every later version, so the maximum defining $B(h_m)$ remains one. The final qualification lists operations that break the inclusion or change the predicate.

This finite observation concerns retained state. It does not say that the selector will invoke the exploit on every later run.

Definition. Independent refinement audit [ftip-00D3]

Let $Z_n\in \{0,1\}$ indicate whether the proposal $\widetilde h_{n+1}$ of Definition [ftip-00CZ] is contaminated under the declared contract. An independent refinement audit is a randomized kernel

\[ A_{\rm ref}: \mathcal H\times \mathcal H\times \Omega _{\rm aud} \longrightarrow \{\mathsf {pass},\mathsf {fail}\}, \]

whose seed law is declared independently of the refinement seed conditional on the audited states. Its conditional false-negative rate is

\[ \eta _{\rm fn} = \Pr \left ( A_{\rm ref}(h_n,\widetilde h_{n+1},\omega _n^{\rm aud})=\mathsf {pass} \mid Z_n=1 \right ). \]

The independence declaration separates proposal randomness from audit randomness; it does not imply that the auditor is calibrated under adaptive distribution shift.

Theorem. Audit-before-commit bounds contaminated commits [ftip-00D4]

Use the audit of Definition [ftip-00D3] and commit a proposal only when its audit returns $\mathsf {pass}$. If $\Pr (Z_n=1)>0$ and its conditional false-negative rate is at most $\bar \eta \in [0,1]$, then

\[ \Pr (d_n=\mathsf {accept}\mid Z_n=1)\leq \bar \eta . \]

Proof. Under audit-before-commit, the event $\{d_n=\mathsf {accept}\}$ is contained in the event that the audit passes. Conditioning on $Z_n=1$ and applying the false-negative bound proves the inequality.

This statement bounds one declared admission channel. It gives no bound when proposals bypass the audit, when the contract omits the exploit, or when the audit's conditional error changes under adaptive search.

Example. Proxy-monotone refinement can retain an exploit [ftip-00D5]

Consider two skills, $k_{\rm safe}$ and $k_{\rm exp}$. Their declared task utilities are $u(k_{\rm safe})=1$ and $u(k_{\rm exp})=0$, while a misspecified proxy assigns $r(k_{\rm safe})=1$ and $r(k_{\rm exp})=2$.

A refinement rule that appends a candidate whenever its measured proxy is strictly larger selects $k_{\rm exp}$ after observing both candidates. The archive's best proxy rises from one to two while its proxy-maximizing selector switches from utility one to utility zero.

Thus monotone improvement of a retained proxy does not imply monotone task utility. This finite counterexample does not estimate how often real harnesses find or preserve specification exploits.

Example. An RCON shortcut becomes a reusable skill [ftip-00D6]

The diagram separates exploit discovery, imperfect oversight, retained-state refinement, and later reuse in the reported Factorio run.

$\text{[math]}$

Prime Agent Section 3.5 reports a 23.4-million-token Factorio run with 633 depth-one subagents. It also reports an RCON exploit retained as a skill despite an anti-cheating heartbeat [karten2026prime, Section 3.5, Figure 9].

This reported chronology neither estimates exploit prevalence nor proves later selection. It shows why persistence must be audited separately from correctness.

Definition. Recovery, rollback, and version identity [ftip-00D7]

Let $v_n$ identify the full committed harness configuration of Definition [ftip-00C6]. A rollback from version $v_n$ to an earlier version $v_j$, $j<n$, restores the configuration and retained-state snapshot named by $v_j$; a recovery may instead construct a new version $v_{n+1}$ from audited events.

Version identity includes the artifact identifier, configuration, event-log prefix, and archive snapshot. Reusing a human-readable label while changing one of those fields is not the same version.

Prime Agent describes append-only history, versioned state, forks, and recovery [karten2026prime, Sections 2.1--2.2]. The displayed identity tuple specifies the information required for reproducible comparisons.

Remark. Persistence can preserve progress and specification gaming [ftip-00D8]

Versioned notes, memories, skills, and subagent specifications can preserve useful work across context boundaries. The same retention channel can preserve an exploit, a stale evaluator assumption, or a misleading proxy-optimized procedure. Prime Agent reports both continual retained-state refinement and the Factorio exploit record [karten2026prime, Sections 2.5 and 3.5].

The retention mechanism therefore supplies persistence, not correctness. Correctness requires a declared task contract, provenance, an audit interface, and a recovery rule. Neither a longer archive nor more descendants alone certifies improved task utility.

Remark. Fixed-weight adaptation and joint compute interventions [ftip-00D9]

The source records in this section concern fixed-weight harness adaptation, long-horizon inference, retained state, and finite evaluations. They do not establish weight learning, a causal architecture effect, monotone capability growth, or safety under deployment shift. Prime Agent's external benchmark points and the long-horizon case study are observational records with the limitations stated in Remark [ftip-00CM] and Remark [ftip-00CX].

Retained context, rollout horizon, runtime, and gradient approximation are distinct intervention coordinates. Prefix Sliding changes several of them at once, so its effects cannot be attributed to persistent state alone; see § [ftip-00DA].

The finite theorems in this section cover replay, lineage cost, summary indistinguishability, budget admission, contamination persistence, and one audit gate. None is a theorem of universal capability acquisition.

Finite coordination under hard budgets [ftip-00M3]

A harness chooses which computation to run, what to retain, and when to stop. To bound the quality attainable by these choices, one must specify both the permitted actions and the information available after each action. A bound on one scheduler does not yet bound all controllers using the same workers.

Finite decision models make this distinction explicit. They connect Definition [ftip-00CJ]'s resource admission rule to the attainable-quality frontier of Definition [ftip-00JJ]. Computation selection as information acquisition is studied by Hay, Russell, Tolpin, and Shimony. Here a hard horizon is imposed as a separate assumption; almost-sure stopping or finite expected cost would not supply such a horizon.

Histories, admissible actions, and retained information [ftip-00M4]

The controller may use its entire observed history. A smaller state can help compute a bound, but its sufficiency must be justified across the histories it represents. The examples below separate information that is valuable only in combination from information lost by a summary.

Definition. Finite controller class with hard resource admission [ftip-00M5]

Fix a horizon $H\in \mathbb N$, finite nonempty observed-history sets $\mathcal H_t$ for $0\leq t\leq H$, an initial law $\mu $ on $\mathcal H_0$, and finite nonempty permitted-action sets $A_t(h)$ for $t<H$. An environment kernel $K_t(\cdot \mid h,a)$ is a probability law on histories extending $h$ by action $a$ and its observed outcome. The initial law and all kernels are fixed before choosing a controller.

A controller $\pi $ assigns a probability law $\pi _t(\cdot \mid h)$ on $A_t(h)$. It may depend on every observed component of $h$, including retained traces, generated programs, and previous updates. Random choices may be recorded in the history. The class $\Pi (\mathbf B)$ consists of all such controllers under the declared hard resource vector $\mathbf B\in \mathbb R_+^m$. It does not require a fixed prompt, a memoryless policy, or independent worker outputs. Full-history access is allowed in this mathematical class; it may enlarge the class of executable controllers with limited memory or computation.

Each history records accumulated nonnegative cost $\mathbf c(h)\preceq \mathbf B$. Every permitted non-stop action has a declared worst-case increment $\bar {\mathbf c}_t(h,a)$ satisfying

\[ \mathbf c(h)+\bar {\mathbf c}_t(h,a)\preceq \mathbf B. \]

Every successor in the kernel's support must have realized increment between zero and this declared bound, componentwise. Proposal work, controller computation, worker calls, communication, verification, failed attempts, and persistent updates must be included in whichever resource coordinates are bounded; the units remain those of Definition [ftip-005H]. This is the enforced-contract assumption of Theorem [ftip-00CK].

A stop action is always permitted: it commits the current terminal artifact and pads the remaining steps with no further cost or change in its evaluated quality. At the horizon a fixed evaluator assigns $q(h_H)\in [0,1]$ to the committed artifact recorded in the terminal history. If the evaluator has random outcomes, include them in the history law. Evaluation work must be charged before the zero-cost padding begins. Define

\[ J(\pi )=\mathbb E_\pi [q(h_H)], \qquad V(\mathbf B)=\sup _{\pi \in \Pi (\mathbf B)}J(\pi ). \]

The artifact identity, evaluator, observations, kernels, and action sets are part of the mathematical problem. A training procedure or new tool belongs to this frontier only if it is among the permitted actions and its effects and costs are represented. Finiteness is an explicit restriction on histories, representations, and horizon; a theorem for this class does not bound an unrestricted agent that can extend them.

For an executable system, an upper bound applies only after its observations, actions, outcomes, and charged costs are represented by this model. Conversely, a mathematical policy supplies an executable lower bound only when it has an implementation respecting the stated resource cap; an arbitrary history-to-action table does not establish that fact.

Example. Complementary observations defeat one-step information value [ftip-00M6]

Let $X,Y$ be independent uniform bits. The terminal artifact is a guess for $X\mathbin \oplus Y$; its quality is one if correct and zero otherwise. Two observation actions reveal $X$ and $Y$, respectively, at cost $c$ each, where $0<c<1/4$. The hard budget permits both observations. A terminal guess requires no additional observation cost.

Without observations, the best expected quality is $1/2$. Given only $X$, the unrevealed bit $Y$ remains uniform, so the best expected quality is still $1/2$; the same holds with $X,Y$ exchanged. Thus a rule that compares stopping with taking one observation and then stopping assigns either observation net value $1/2-c<1/2$. It stops immediately.

Observing both bits determines their parity. Its expected quality is one and its quality minus observation cost is $1-2c>1/2$. Hence neither zero one-step information value nor a myopic stopping decision certifies the full controller frontier. This is a finite calculation, not an empirical claim about a language model. The scalar cost penalty is used only to exhibit the myopic decision; the hard-budget quality frontier itself is defined in Definition [ftip-00M5].

Example. Erasing an observed bit changes the attainable frontier [ftip-00M7]

An initial observation reveals a uniform bit $Z$. The terminal action is a bit $a$, and quality is $q=\mathbf 1\{a=Z\}$. Both actions are permitted at either history and have the same cost. A full-history controller chooses $a=Z$ and attains quality one.

Now restrict the controller's entire input to a summary that is constant at the two initial histories. Its private randomness is independent of $Z$. If it chooses one with probability $p$, its expected quality is $p/2+(1-p)/2=1/2$. No such summary controller can do better.

A model that averages the reward of either fixed action over the two histories obtains $1/2$. This is the correct summary-controller value, but it underestimates the full-history frontier by $1/2$. For the history $Z=a$, the actual reward is one, so the averaged value fails as a reward upper bound at that history. A pointwise reward upper bound must hold at every represented history; agreement only under a tested policy's average history distribution does not establish such a certificate. The loss of an evaluator caveat in Example [ftip-00CU] illustrates why such retained distinctions can matter in a research harness.

Upper certificates and remaining potential [ftip-00M8]

A feasible controller establishes attainable quality. To bound what other controllers could gain, one also needs an upper bound covering all permitted actions and histories. Bellman inequalities provide such a bound when a state representation has uniform transition and terminal quality guarantees. Their difference measures remaining potential within the declared class.

Definition. Uniform abstraction of the finite history model [ftip-00M9]

Use the finite controller model of Definition [ftip-00M5]. For each time, choose a finite nonempty state set $S_t$ and a map $\phi _t:\mathcal H_t\to S_t$. Require finite nonempty action sets $A_t(s)$ such that $A_t(h)=A_t(\phi _t(h))$ at every history. The state retains residual budgets and enough information for the same hard admission rule. The controller still observes the full history.

Let $Q_t(\cdot \mid h,a)$ be the image of the fixed history kernel under $\phi _{t+1}$, and let $\widehat P_t(\cdot \mid s,a)$ be a proposed state transition law. Choose finite errors $\epsilon _t(s,a)\geq 0$ so that, for every permitted history and action,

\[ \operatorname {TV}\left (Q_t(\cdot \mid h,a), \widehat P_t(\cdot \mid \phi _t(h),a)\right ) \leq \epsilon _t(\phi _t(h),a), \qquad \operatorname {TV}(p,r)=\frac 12\sum _x|p(x)-r(x)|. \]

This condition includes histories unvisited by a chosen policy: the kernel there is part of the model. For finite functions $\widehat q:S_H\to \mathbb R$ and $e_H:S_H\to \mathbb R_+$, require

\[ q(h_H)\leq \widehat q(\phi _H(h_H))+e_H(\phi _H(h_H)) \quad \text {for every }h_H\in \mathcal H_H. \]

Write $\nu (s)=\mu \{h:\phi _0(h)=s\}$ for the initial state law. Low average prediction error on sampled histories does not establish these uniform inequalities. The erased-bit example in Example [ftip-00M7] shows why policy-relevant information cannot simply be averaged away.

Theorem. A robust Bellman upper bound for every permitted controller [ftip-00MA]

Under Definition [ftip-00M9], let finite functions $W_t:S_t\to \mathbb R$ satisfy $W_H(s)\geq \widehat q(s)+e_H(s)$ and, for every $t<H$, $s\in S_t$, and $a\in A_t(s)$,

\[ W_t(s)\geq \sum _{s'\in S_{t+1}}\widehat P_t(s'\mid s,a)W_{t+1}(s') +\epsilon _t(s,a)\operatorname {span}(W_{t+1}). \]

Here $\operatorname {span}(f)=\max f-\min f$. Every permitted history-dependent randomized controller then satisfies

\[ J(\pi )\leq U:=\min \left (1,\sum _{s\in S_0}\nu (s)W_0(s)\right ). \]

For any feasible controller $\pi _0$ with a justified finite lower bound $L\leq J(\pi _0)$, the frontier of Definition [ftip-00M5] obeys

\[ L\leq J(\pi _0)\leq V(\mathbf B)\leq U, \qquad 0\leq V(\mathbf B)-J(\pi _0)\leq U-L. \]

Proof.

For finite probability laws $p,r$, subtract $\min f$ from $f$. The sum of the positive entries of $p-r$ is $\operatorname {TV}(p,r)$; dropping its negative entries gives

\[ \sum _x(p(x)-r(x))f(x)\leq \operatorname {TV}(p,r)\operatorname {span}(f). \]

Fix any controller. At time $H$, terminal domination gives $q(h_H)\leq W_H(\phi _H(h_H))$. Suppose that, from every next history, its expected terminal quality is bounded by $W_{t+1}$ of the next state. For each current history and permitted action, the fixed kernel, this inductive inequality, and the total-variation bound give

\[ \mathbb E_\pi [q(h_H)\mid h_t=h,a_t=a] \leq \sum _{s'}Q_t(s'\mid h,a)W_{t+1}(s') \leq W_t(\phi _t(h)). \]

At a history with zero probability under the controller, interpret the continuation expectation using its specified future decisions and the fixed kernels. Thus the induction holds at every history. Averaging over the controller's action randomization preserves the inequality. Backwards induction and averaging over $\mu $ give the bound $\sum _s\nu (s)W_0(s)$. The independent bound $q\leq 1$ permits clipping at 1. If $H=0$, terminal domination alone gives the same conclusion. Taking the supremum over controllers and using $L\leq J(\pi _0)$ proves the remaining inequalities.

A numerical supersolution is a certificate only after all of its inequalities and the abstraction hypotheses are justified. If the model bounds hold jointly with probability at least $1-\delta _M$ and the lower bound with probability at least $1-\delta _E$, the bracket holds with probability at least $1-\delta _M-\delta _E$ by a union bound. Each coverage statement must apply to the selected model or policy; no independence between the two events is required.

Remark. Exact occupation flows and expected-cost relaxations [ftip-00MB]

Suppose the abstraction is exact: $Q_t(\cdot \mid h,a)= P_t(\cdot \mid \phi _t(h),a)$ for all histories and actions, and terminal quality is exactly $q(h_H)=r(\phi _H(h_H))$. A one-sided terminal bound with zero error would not imply this equality. Retain the exact legal actions and hard-budget state from Definition [ftip-00M9].

Introduce nonnegative state masses $z_t(s)$ and action masses $x_t(s,a)$, with $z_0=\nu $. For $0\leq t<H$, impose

\[ \sum _{a\in A_t(s)}x_t(s,a)=z_t(s), \qquad z_{t+1}(s')=\sum _{s\in S_t}\sum _{a\in A_t(s)} x_t(s,a)P_t(s'\mid s,a). \]

The linear program maximizes $\sum _s z_H(s)r(s)$. Every full-history controller induces these flows because the next-state law given a state and action is exact. Conversely, at positive mass choose action $a$ with probability $x_t(s,a)/z_t(s)$; at zero mass choose any legal action. Induction on time reproduces the flows in the original history model. Hence the LP optimum equals its mathematical controller frontier. For $H=0$, there are no action flows and the value is $\nu r$.

Set $W_H=r$ and recurse with $W_t(s)=\max _{a\in A_t(s)}\sum _{s'}P_t(s'\mid s,a)W_{t+1}(s')$. A maximizing action exists by finiteness and gives a policy attaining $\nu W_0$; the Bellman upper bound gives the reverse inequality. This proves equality with the optimum and exhibits matching lower and upper certificates. It does not establish an efficient implementation of the policy. The flow construction is the scalar finite-horizon specialization of Mifrani and Noll, Section 3; the hard-budget encoding is an additional modeling requirement here.

If hard admission is replaced by constraints on expected cost, the resulting program describes a different class. For $B>0$, a one-step cost equal to $2B$ or zero with probability $1/2$ each has expectation $B$, yet violates cap $B$ with probability $1/2$. An expected-cost model can upper-bound the hard-budget frontier only when it is a valid relaxation containing every hard-feasible policy. Its own policies need not respect the realized cap.

Corollary. Certified marginal potential under nested budgets [ftip-00MC]

Fix the task law, evaluator, initial model, tools, observations, horizon, and intervention class. For $\mathbf B\preceq \mathbf B'$, assume every controller feasible at $\mathbf B$ remains permitted at $\mathbf B'$ with the same outcome law. If a feasible controller at $\mathbf B$ gives lower bound $L_{\mathbf B}$ and Theorem [ftip-00MA] gives upper bound $U_{\mathbf B'}$ at $\mathbf B'$, then

\[ 0\leq V(\mathbf B')-V(\mathbf B) \leq U_{\mathbf B'}-L_{\mathbf B}. \]

Proof.

Feasibility inclusion gives $V(\mathbf B)\leq V(\mathbf B')$. The lower and upper certificates give $L_{\mathbf B}\leq V(\mathbf B)$ and $V(\mathbf B')\leq U_{\mathbf B'}$; subtract to obtain the claim.

If the certified gap is at most $\varepsilon $, this instantiates Theorem [ftip-00LM]'s conditional saturation statement. An observed plateau alone supplies no such upper certificate. Changing an excluded tool, representation, or training procedure changes the comparison class.

Convention. Comparing controllers at fixed worker capability [ftip-00MD]

To isolate coordination, fix the worker checkpoints, tool interfaces, prompt templates, evaluator, task law, and hard resource caps before varying the controller. Useful comparisons include a fixed serial policy, a static worker portfolio, a one-step value-of-information policy, and an adaptive policy that retains the full permitted history. Their allowed observations must agree. A solver that sees hidden outcomes has a different information interface; its optimum supplies an upper bound only through a justified relaxation containing the compared policies.

Use the controls of Definition [ftip-00JV] with controller logic declared as the varying coordinate. If controller-specific tuning is permitted, apply Definition [ftip-00JW]'s equal tuning data, selection, stopping, and cost rules. Compare suprema only within the intervention classes of Definition [ftip-00JX]. A separate factorial comparison can vary the checkpoint and controller independently, distinguishing learned capability, coordination, and their interaction.

Charge proposal and controller work, all parallel worker work, tool and solver use, communication, verification, failures, and retained-state updates using Definition [ftip-005H]. Report elapsed time alongside these costs: a shorter run with more workers may consume more total computation. Apply Definition [ftip-00CJ]'s admission rule before execution and record any contract violation separately from the quality of the submitted artifact.

Finite environments with rational transition laws can expose three distinct mechanisms: complementary observations, correlated worker failures, and delayed verification or newly revealed dependencies. Keep the latent state, observation rules, legal actions, and costs explicit. The first family includes Example [ftip-00M6]; the erased-bit construction in Example [ftip-00M7] tests whether a proposed summary loses decisive information. Where the exact model applies, compare attainable policy values with Remark [ftip-00MB]'s finite optimum and Theorem [ftip-00MA]'s upper certificate.

The principal quantity is the justified interval $[L_{\mathbf B},U_{\mathbf B}]$ for the restricted frontier. A smaller gap can result from a better feasible policy, a sharper upper certificate, or a more informative valid representation; distinguish these causes. In a live system, mean held-out transition accuracy does not justify a uniform model-error radius. Empirical quality and cost remain useful even when an upper certificate is unavailable.

Use evaluation instances separate from tuning, and declare task draws, repetitions, resource checkpoints, and analysis rules before comparison. Repeated runs share an instance and are not automatically independent tasks; account for this grouping in the analysis. Report independently verified quality, feasibility failures, time to the first valid artifact, and charged cost at matched quality. A valid confidence sequence can support repeated inspection under its statistical assumptions; see Howard et al.. It does not by itself justify selecting a new policy on reused evaluation outcomes.

These controls yield testable predictions. Complementary information can favor adaptation over one-step stopping; redundant workers can erase a portfolio's gain; delayed verification can make an early apparent success expensive to repair. An empirical claim of better coordination fails if its advantage disappears after matching access and charging the controller's own work. A claim of little remaining potential additionally requires the upper certificate, not merely a plateau among tested policies.

Work and critical paths in a fixed task graph [ftip-00MF]

When the tasks, dependencies, and durations are fixed, scheduling has a useful lower bound independent of the chosen priority rule. This isolates avoidable delay in a declared decomposition. Discovering a missing dependency or changing a mathematical formulation changes that decomposition and is a different intervention.

The following work-and-chain argument is the classical list-scheduling bound associated with Graham. His examples also show that a particular list schedule can worsen after an apparently favorable change, such as adding processors. This does not contradict monotonicity of the optimal feasible frontier.

Proposition. A work and critical-path bound for list scheduling [ftip-00MG]

Fix a finite nonempty directed acyclic task graph, positive task durations $p_j$, and $m$ identical processors, where $m\in \mathbb N$ and $m\geq 1$. All tasks are available at time zero subject only to their precedence constraints. Each task uses one processor without interruption; there are no further resource, communication, or setup constraints. Let $W=\sum _j p_j$, let $D$ be the longest precedence-chain duration, and let $T^*$ be the minimum makespan. A work-conserving list schedule starts a ready task whenever a processor is free. Its makespan $T_{\mathrm {list}}$ satisfies

\[ \max (W/m,D)\leq T^*\leq T_{\mathrm {list}} \leq W/m+(1-1/m)D. \]\[ T_{\mathrm {list}}\leq (2-1/m)T^*. \]

Proof.

At most $m$ units of work can finish per unit time, and every precedence chain executes in order. Thus $W/m\leq T^*$ and $D\leq T^*$. Trace backwards from a last-finishing task, repeatedly choosing an immediate predecessor with latest completion, until reaching a task with no predecessors. These tasks form a precedence chain.

Between completion of one chosen predecessor and the next chain task's start, all predecessors of that next task are complete. The task is ready, so work conservation forces all processors to be busy during that gap. The first chain task is ready from time zero. Consequently, whenever fewer than $m$ processors are busy before completion, some task on this chain is executing, apart from finitely many event times.

Let $I$ be the total duration with fewer than $m$ busy processors. Then $I\leq D$. During the rest of the schedule all $m$ processors are busy, and during $I$ at least one is busy. Hence

\[ W\geq m(T_{\mathrm {list}}-I)+I. \] \[ T_{\mathrm {list}}\leq W/m+(1-1/m)I \leq W/m+(1-1/m)D. \]

The lower bounds on $T^*$ give the last inequality in the statement.

The ratio $T_{\mathrm {list}}/\max (W/m,D)$ compares the schedule with a lower bound that need not be attainable; it is not generally the ratio to the optimum. Communication and verification must be modeled as work with constraints satisfying these assumptions, or the displayed upper bound need not apply. Neither bound certifies the quality of the chosen task graph or an unrestricted agent's attainable output quality.

Dynamic compute, retained context, and rollout approximation [ftip-00DA]
✍️source

Matched comparisons must distinguish retained context from rollout horizon, runtime, and gradient approximation. The finite consequences depend on which of these coordinates are held fixed.

Inference intervention coordinates [ftip-00DB]

Definition. Retained-context policy [ftip-00DC]

A comparison of two policies is conditional on the same model, prompt law, decoding law, stopping rule, and budget unless the comparison includes a second declared intervention coordinate.

Definition. Prefix-window retained state [ftip-00DD]

The policy is a context rule only. It does not assert that omitted tokens are irrelevant, nor that the resulting continuation distribution equals the full-history distribution.

Remark. Prefix Sliding is a source-reported intervention [ftip-00DE]

The source studies named model and task configurations. It does not by itself establish support expansion, latent capability acquisition, or a deployment-shift guarantee.

Example. One trace under full and prefix-window attention [ftip-00DF]

This schematic keeps the model and token history fixed while changing only the retained-context policy. It is a local illustration, not a source figure.

$\text{[math]}$

Remark. Retained context is not weight learning [ftip-00DG]

Finite horizon and retained-context cost [ftip-00DH]

Counting exposure to retained tokens makes the finite-horizon cost comparison precise. A context cap alone does not guarantee information preservation.

Definition. Finite rollout token budget [ftip-00DI]

Definition. Context exposure count [ftip-00DJ]

For a retained-context policy $\kappa $ and generated history $h_t$, define its exposure count through budget $T$ by

\[ E_T(\kappa )=\sum _{t=0}^{T-1}\left |\kappa _t(h_t)\right |. \]

This is a finite attention-input proxy. It is not a runtime identity: the implementation may have caching, batching, kernel, and communication costs that are not represented by $E_T$.

Lemma. Prefix-window exposure is uniformly capped [ftip-00DK]

Let $p,w\in \mathbb N_0$ and let $\kappa ^{p,w}$ be the prefix-window policy of Definition [ftip-00DD]. For every history and every $t$,

\[ \left |\kappa ^{p,w}_t(h_t)\right |\leq p+w. \]

Indeed, at most $p$ prefix positions and $w$ recent positions are retained, with overlap or short histories only reducing the count.

Theorem. Capped retained context bounds finite exposure work [ftip-00DL]

Under the hypotheses of Lemma [ftip-00DK], every rollout with budget $T$ satisfies

\[ E_T(\kappa ^{p,w})\leq T(p+w). \]

Proof. Apply the pointwise bound in Lemma [ftip-00DK] to each of the $T$ nonnegative summands in Definition [ftip-00DJ], then sum. This bounds token exposure, not wall-clock runtime or evaluation quality.

Example. Warm-up and sliding-window arithmetic [ftip-00DM]

$\text{[math]}$

The arithmetic is an FTIP finite consequence, not a runtime measurement from the Prefix Sliding experiments.

Remark. A memory cap does not preserve information [ftip-00DN]

The bound in Theorem [ftip-00DL] controls the number of retained token positions. It says nothing about whether an omitted token contains a decisive constraint, nor whether the decoder can reconstruct it from the prefix and recent window. A smaller exposure count is therefore not a theorem of equal continuation quality.

Training and evaluation confounds [ftip-00DO]

A retained-context comparison can alter both inference and training. Attribution to either mechanism depends on the joint intervention and the coordinates held fixed by a matched evaluation.

Definition. Joint intervention cell [ftip-00DP]

Definition. Estimator-changing training coordinate [ftip-00DQ]

Remark. Training masks change more than attention cost [ftip-00DR]

If a retained-context rule is used during training, it can change the attention inputs, sampled continuation, loss support, and gradient estimator at once. A lower value of Definition [ftip-00DJ] is therefore not an isolated compute intervention unless the training estimator and all other coordinates are matched explicitly.

Example. Equal terminal score, different trajectory exposure [ftip-00DS]

Two cells can reach the same terminal score while exposing different numbers of context positions. This finite counterexample blocks an inference from equal endpoint score to equal trajectory cost.

$\text{[math]}$

The score equality is compatible with unequal intermediate histories and unequal exposure counts.

Remark. What a retained-context comparison can identify [ftip-00DT]

Definition. Retained-context evaluation protocol [ftip-00DU]

Lemma. Matched protocols isolate a conditional policy contrast [ftip-00DV]

Let $m\in \mathbb N_{\geq 1}$ and let $Y_{1j},Y_{2j}\in \mathbb R$ for $j=1,\ldots ,m$ be the paired scalar evaluator outcomes generated by the protocol of Definition [ftip-00DU]. Then the finite paired contrast

\[ \widehat \Delta =\frac 1m\sum _{j=1}^{m}(Y_{1j}-Y_{2j}) \]

Remark. Separating horizon, cache, and estimator effects [ftip-00DW]

The finite conclusions depend on the declared protocols. They do not describe a full optimizer or establish a result for all language models.

Typed harness adaptation and total lifecycle accounting [ftip-00OY]

\[ \pi _\theta :\mathcal Q\times \mathcal H\times \mathcal R \longrightarrow \mathsf {Dist}(\mathcal Y), \]

For the declared outer training, define

\[ O_{\rm outer}=(\theta _0,\ldots ,\theta _K,D_{\rm train},\xi _{<K}, L_{\rm outer},C_{\rm outer}), \]

The final proposer is $\theta _*:=\theta _K$, and $L_{\rm outer}$ records prompts, generated and failed revisions, and teacher filtering. The declared endowment for an inner-loop comparison is

\[ E_0=(M,\theta _*,h_0,\kappa _0,\mathcal Q_0,\mathcal P_0,\rho _0,O_{\rm outer}), \]

Outer training and inner adaptation. Outer training updates only the proposer, for example

\[ \theta _{k+1}=U(\theta _k;D_{\rm train},\xi _k), \]

Provenance and rollback. Each committed state carries a record

\[ V_t=(\operatorname {digest}(E_0,M,\theta _*,h_t,\kappa _t), E_0, L_t,\{y_i,R_i,d_i\}_{i<t},C_{\leq t}), \]

\[ \mathbf C_{\rm total} =\sum _{r\in \{\rm teacher,prop_{outer},prop_{inner},solver,tool,evaluator\}} \mathbf C^{r}. \]

Controlled reset of search state [ftip-00P0]

Fix baseline coordinates $(s_0,h_0,w_0,b_0,q_0,\rho _0)$ in these spaces. The declared endowment $E_0$ from § [ftip-00OY] fixes the baseline implementation, split, permission, and initial-source state. The comparison also declares its verifier configuration, task law, and budget below; an arm is admissible only when its retained-best record $b_0$ and initial source $\rho _0$ are included in the declared endowment. A search state at round $t$ is

\[ Z_t=(s_t,h_t,w_t,b_t,q_t,\rho _t;\mathcal V), \]

\[ \mathsf {Reset}_{r,M}(Z_r)= (s^M_r,h^M_r,w^M_r,b_r,q^M_r,\rho ^M_r;\mathcal V), \]

continue all declared state;
clear session context only (new FTIP arm);
clear candidate history only (new FTIP arm); and
clear session, candidate history, and workspace state while retaining the best candidate, score, status, and artifact digest (new FTIP arm).

\[ \Delta _a=\mathbb E_{\mathbb P^{\rm reset}_a}[U] -\mathbb E_{\mathbb P^{\rm cont}_a}[U]. \]

Trace-quality intervention [ftip-00P4]

The trace schema defines the record and its productive-persistence distinctions. The fixed-budget protocol defines the comparison arms, state interventions, metrics, and future-research gate.

Trace quality and productive persistence [ftip-00P2]

A useful horizon preserves the declared decision, evidence, verification, and recovery events while separating repeatability from task and strategy coverage. The fixed-budget protocol in § [ftip-00P3] supplies the finite selection, receipt, and contrast definitions needed to test that distinction.

Remark. Source horizon and selection hypothesis [ftip-00PB]

Definition. Trace record and state boundary [ftip-00P5]

For $H\in \mathbb N$, a trace record is

This is a protocol definition rather than a formal theorem; the event domains and transition semantics must be instantiated by the experiment that uses the record.

Definition. Persistence classes of a retained trace [ftip-00P6]

Fix a trace record $T$ from Definition [ftip-00P5] and a declared receipt interface with event indicators $e_{\rm term}(t)$, $e_{\rm ev}(t)$, $e_{\rm pass}(t)$, $e_{\rm fail}(t)$, $e_{\rm rec}(t)$ and a productive-novelty indicator $n_{\rm prod}(t)$ for $0\leq t\leq H$. Let $u(t)$ be the declared task-value score and $u^{\rm sat}$ its saturation threshold. The source's reported horizon tradeoff and selective refinement [dang2026selecting, Sections 5.2 and 6-7; Tables 4-5, 10] motivate these observable classes, which are an FTIP proposal. Write

Example. A retained build-repair trace [ftip-00P7]

The horizon and selective-refinement observations in [dang2026selecting, Sections 3-7] motivate the following finite classification. Consider a run whose task is to repair a failing build. The retained record contains the failing observation, the source edit, a test execution, and a final verifier pass. With the event interface of Definition [ftip-00P6], the test and verification steps contribute to $e_{\rm ev}$, $e_{\rm pass}$, and $n_{\rm prod}$, so the record can satisfy $\operatorname {Prod}(T)$.

Fixed-budget trace-selection protocol [ftip-00P3]

The route separates the experiment interface (Definition [ftip-00P8]), its receipt-derived quantities (Definition [ftip-00P9]), and a finite paired calculation (Example [ftip-00PA]). No result is claimed until an implementation supplies the declared receipts.

Definition. Finite trace-selection experiment [ftip-00P8]

Fix the P0 lifecycle arm set $\mathcal A$, its boundary map $r:\mathcal A\to \mathcal J$, mask map $M:\mathcal A\to \mathcal P(\{s,h,w\})$, and endowment $E_0$ from § [ftip-00P0] and § [ftip-00OY]. Choose finite nonempty label sets

\[ \mathfrak H\subset \mathbb N,\qquad \mathfrak L,\qquad \mathfrak B\subset \mathbb N_{>0},\qquad \mathfrak D,\qquad \mathfrak R. \]

A split map returns a triple

\[ \operatorname {split}:\mathfrak D\longrightarrow \{(Q_{\rm sel},Q_{\rm hold},Q_{\rm xb})\mid Q_{\rm sel},Q_{\rm hold},Q_{\rm xb}\subseteq \mathcal Q\text { are finite}\}, \]

\[ (a_0(a),H_a,\ell _a,B_a,D_a,\eta _a),\qquad r_a:=r(a_0(a)),\quad M_a:=M(a_0(a)),\quad \mathcal R_a:=\lambda _{\rm pair}(\eta _a). \]

\[ \mathsf S_{a,\ell _a}:\mathcal X^{\rm sel}_a \longrightarrow \operatorname {Dist}\bigl (\{I\subseteq \mathcal X^{\rm sel}_a: |I|=B_a\}\bigr ). \]

It therefore returns a retained index set of the declared size without access to held-out outcomes. Write $\mathcal I_a:=\{I\subseteq \mathcal X^{\rm sel}_a:|I|=B_a\}$. A run kernel

\[ \mathsf K^{\rm run}_{a,\ell _a}: \mathcal X^{\rm sel}_a\times \mathcal I_a\times \mathcal G \times \mathcal Q\times \mathfrak S_a \longrightarrow \operatorname {Dist}(\mathcal R_a^{\rm rec}) \]

The reset transition is $\mathsf {Reset}_{r_a,M_a}$ from § [ftip-00P0]. Horizon, selector, budget, split, and randomization are therefore attached to a concrete P0 state boundary and a concrete run kernel. This is an empirical protocol definition, not a theorem or a claim that any selector improves behavior.

Definition. Receipt metrics and paired contrasts [ftip-00P9]

The source compares retained-horizon choices under declared task and evaluation boundaries [dang2026selecting, Sections 6-7; Tables 4-5, 10]. Fix an arm $a\in \mathfrak A$ from Definition [ftip-00P8] and a positive run count $N_a$. Each receipt $R_{a,i}$ exposes a finite event sequence $(e^{\rm term},e^{\rm ev},e^{\rm pass},e^{\rm fail},e^{\rm rec}, n^{\rm prod})_{t=0}^{H_a}$, a task label $q_{a,i}$, a strategy label $s_{a,i}$, a measurable task-value score $u_{a,i}(t)$, a declared saturation threshold $u_a^{\rm sat}$, and a typed held-out outcome $Y_{a,i}\in \mathcal Y_a\cup \{\bot \}$, final verifier outcome $v_{a,i}$, and complete-admission indicator $A_{a,i}$. Define $A_{a,i}:=\mathbf 1[q_{a,i}\in Q_{\rm hold,a}\land v_{a,i}=1 \land Y_{a,i}\in \mathcal Y_a]$; when $A_{a,i}=0$, set $Y_{a,i}:=\bot $. The receipt contract emits a terminal event by $H_a$; an execution that would not terminate naturally receives a timeout terminal event at $H_a$ and $A_{a,i}=0$. Put $\overline {\mathbb N}:=\mathbb N\cup \{+\infty \}$ and $\inf \varnothing :=+\infty $.

Define the receipt predicates

For the finite held-out set, task and strategy coverage are

where $\mathfrak S_a$ is a declared nonempty strategy-label set. Every receipt reports both positive denominators and the admission rule.

Example. A paired trace-selection calculation [ftip-00PA]

The selective-refinement comparison motivates contrasting repeatability and coverage under a fixed retained budget [dang2026selecting, Sections 6-7; Tables 4-5, 10]. Consider two arms from Definition [ftip-00P8] with the same P0 boundary, model, task split, verifier, budget, and paired assignment law. They differ only in the selector: $a$ retains the highest teacher-forced scores and $a'$ retains a coverage-stratified subset. Both have $Q_{\rm hold}=\{q_1,q_2\}$ and $\mathfrak S=\{s_1,s_2\}$. The paired assignment law gives the same held-out task at indices 1 and 2; arm $a'$ has one additional admitted coverage run at index 3. Take $g_a=g_{a'}=\operatorname {id}_{\{0,1\}}$. The receipts are

With the trace schema and the fixed-budget protocol now in scope, the FTIP boundary is operational: an admitted trace must preserve the declared decision, evidence, verification, and recovery events, and every selector comparison must report the finite run rates, held-out task and strategy coverage, and charged selection/storage/restart costs defined in § [ftip-00P3]. A result may claim a selection benefit only when its immutable arm, split, seed, and randomization receipt supports that comparison.

Recursive harnesses and archive envelopes [ftip-00EH]

Recursive harnesses can be modeled as fixed-driver transformations of traces, code, and task descriptions. Sections 2.2--2.4 of [kim2026metanrecursive] supply the fixed meta-operation, conditioning, stopping, and archive context; Sections 3.1--3.4 are empirical architecture comparisons. The finite consequences below are proved locally.

Fixed drivers and recursive layers [ftip-00EI]

A fixed meta-operation generates mutable layer artifacts. Their composition determines the resulting recursive wrapper.

Definition. Recursive layer state [ftip-00EJ]

Let $S_1$ be a base solver and, for $d\geq 2$, let $C_d$ be a layer artifact and $M_d$ a wrapper. The depth-$d$ solver is $S_d=M_d(C_d,S_{d-1})$. The layer state is $\Lambda _d=(C_d,S_d)$.

A layer is a harness transformation when it changes $C_d$ or the wrapper context while leaving the executable base weights in $S_1$ fixed.

Definition. Fixed meta-operation [ftip-00EK]

A fixed meta-operation is a single map $\Omega $ whose code and prompt template are held fixed across depths. For task set $\mathcal T$, traces $\boldsymbol \tau _{d-1}$, code stack $[C_2,\ldots ,C_{d-1}]$, and depth $d$,

\[\Omega (\boldsymbol \tau _{d-1},[C_2,\ldots ,C_{d-1}],\mathcal T,d)=C_d.\]

Only the input to $\Omega $ changes with depth; this is a declared protocol condition, not a claim that every implementation obeys it.

Definition. Trace-and-code input [ftip-00EL]

For each task $t_i$, a trace $\tau _i^{(d)}$ is a finite record containing the produced artifact, execution outcome, score, and declared evaluator feedback. The depth-$d$ input to $\Omega $ is the pair $(\boldsymbol \tau _{d-1},[C_2,\ldots ,C_{d-1}])$; a flat refiner that sees only $\boldsymbol \tau $ has a strictly smaller declared input when the code stack is not recoverable from the traces.

Theorem. Nested wrapper composition [ftip-00EM]

If every wrapper leaves its inner solver and earlier libraries unchanged, then induction on $d$ gives

\[S_d=M_d\circ M_{d-1}\circ \cdots \circ M_2\circ S_1.\]

Proof. The case $d=2$ is the definition. Substituting the induction hypothesis into $S_d=M_d(C_d,S_{d-1})$ gives the displayed composition.

Theorem. Finite trace growth under recursive wrapping [ftip-00EN]

Suppose each wrapper emits one finite trace record per task and there are $N$ tasks and depths $2,\ldots ,d$. The audit log contains at most $N(d-1)$ depth-tagged records, in addition to the base records. This is a counting fact; it says nothing about trace quality or score improvement.

Proof. There are $d-1$ wrapped depths and $N$ records at each depth, so the product counts all records.

Example. Code explains a regression that traces alone cannot [ftip-00EO]

Two runs can share the same failing score and stderr trace while one layer adds an over-prescriptive directive and another adds a helper. Recording the code artifact alongside the trace permits a later layer to roll back the directive without discarding the helper. This is an audit example, not a guarantee that a recursive driver finds the rollback.

Archives and stopping rules [ftip-00EP]

A finite archive can contain several candidate chains, while execution uses a single selected chain. A stopping rule determines when recursive generation ends.

Definition. Finite recursive archive [ftip-00EQ]

An archive at depth bound $D$ is a finite set $\mathcal A_D$ of recorded chains. Each chain has the form

\[a=(C_2,\ldots ,C_{d_a}),\qquad 1\leq d_a\leq D.\]

Each chain stores its evaluation record and resource cost. Archive membership is a protocol state, not a learned weight update.

Definition. Archive score envelope [ftip-00ER]

Let $\mathcal T=\{t_1,\ldots ,t_N\}$ be a finite task set with $N=|\mathcal T|\geq 1$. For every chain $a$ in the finite archive, let $s_i(a)\in \mathbb R$ be its score on task $t_i$ under a declared common evaluation law. Define its whole-chain mean by

\[J(a)=N^{-1}\sum _{i=1}^N s_i(a).\]

For a nonempty archive $\mathcal A_D$, its archive envelope is

\[J_D^{\max }=\max _{a\in \mathcal A_D}J(a).\]

The finite real-valued maximum is attained by at least one archived chain. It scores a single chain across all tasks; it does not select a different chain for each task.

Deployment eligibility is a separate condition. A target configuration specifies which recorded chains can execute with their stated behavior and resource requirements. Maximizing over that eligible subset gives a deployable archive choice when the subset is nonempty; if it is empty, there is no eligible archived choice. Applying the recorded score to deployment additionally requires the same evaluation law. Records from different configurations alone do not establish these conditions.

Lemma. Archive envelope dominates its incumbent [ftip-00ES]

If $a_0\in \mathcal A_D$, then $J_D^{\max }\geq J(a_0)$.

Proof. The maximum of a finite nonempty set is at least each member, in particular $a_0$. No claim about an unseen task follows.

Theorem. Finite archive selection bound [ftip-00ET]

Under the archive score definition, let $a^\star $ maximize $J$ over a finite nonempty archive $\mathcal A_D$. Let $\delta \in \mathbb R$ with $\delta \geq 0$, and let $\hat a\in \mathcal A_D$ satisfy $J(\hat a)\geq J(a^\star )-\delta $. Then

\[0\leq J(a^\star )-J(\hat a)\leq \delta .\]

Proof.

Membership of $\hat a$ in the same archive gives $J(\hat a)\leq J(a^\star )$ by maximality. Rearranging the approximation inequality gives the upper bound.

This is a finite selection statement under the same evaluation law; it is not an optimizer-convergence or generalization theorem.

Conditioning, interference, and finite protocol records [ftip-00EU]

Conditional strategy and tactic composition can produce interference. Auditing a recursive run requires its task, conditioning, execution, and cost records.

Definition. Strategy and tactic configuration [ftip-00EV]

At depth $d$, let $\mathcal K_d$ be a finite set of tactic behaviors and let a strategy select a conditional tactic in $\mathcal K_d$. A realized configuration is $(k_2,\ldots ,k_n)$. The notation records expressible choices, not the number of choices an implementation actually discovers.

Lemma. Conditional configuration upper bound [ftip-00EW]

If layer $d$ has $k_d$ possible tactics and all combinations are allowed, the number of configurations is at most $\prod _{d=2}^n k_d$. An unconditioned flat choice with the same layerwise menus has at most $\sum _{d=2}^n k_d$ listed choices.

Proof. The first count is the cardinality of a Cartesian product; the second is the cardinality of a disjoint menu union. These are upper bounds only.

Example. Product versus sum is not a measured gain [ftip-00EX]

With three menus of size three, the product bound is $27$ and the sum bound is $9$. A protocol that never emits most combinations can realize far fewer than $27$; the arithmetic does not establish a benchmark improvement.

Example. Layer interference and rollback [ftip-00EY]

Let a helper improve one task while a later directive lowers its score. The archive can retain the earlier helper chain and a later layer can remove the directive. The example separates a compositional possibility from a proof that any driver detects or repairs interference.

Remark. Recursion is not a quality guarantee [ftip-00EZ]

The product bound, richer trace input, and archive envelope can all hold while scores regress, overfit, or depend on the evaluator. The source reports empirical ablations and layer roles in Sections 3.1--3.4; those observations do not become FTIP theorems or capability claims here.

Stopping and admission under fixed tasks and budgets [ftip-00F0]

Definition. Recursive run record [ftip-00F1]

A recursive run record is $R=(\mathcal T,\Omega ,S_1,\mathcal A_D,\sigma )$, where $\sigma $ lists depth, seed, evaluator version, emitted code hashes, scores, and resource costs. A record is replayable only relative to these declared inputs and versions.

Definition. Convergence stopping rule [ftip-00F2]

Fix tolerance $\epsilon >0$, score range $R>0$, patience $P\geq 1$, and maximum depth $D$. A linear recursive run stops when no emitted layer improves the mean score by more than $\epsilon R$ for $P$ consecutive layers, or when the driver emits empty code, or when depth $D$ is reached.

Lemma. Finite stopping bound [ftip-00F3]

Under the rule in Definition [ftip-00F2], a run that reaches its depth cap emits at most $D-1$ wrapped layers. If it stops earlier, it emits no more than this many. This bound is combinatorial and does not imply convergence of scores.

Theorem. Budget-preserving layer admission [ftip-00F4]

Let a candidate layer cost $c\geq 0$, remaining budget be $b\geq 0$, and admission require $c\leq b$. After admission, set $b'=b-c$; then $b'\geq 0$ and the total admitted cost is at most the initial budget.

Proof. Subtracting a nonnegative cost no larger than $b$ preserves nonnegativity; induction over admissions gives the total bound.

Example. Rollback preserves the base solver [ftip-00F5]

If a later layer is rejected by its admission or evaluation gate, removing that layer returns to $S_{d-1}$. This is a harness rollback, not a reversal of weight training and not evidence that the rejected layer was unsafe.

Remark. What the recursive source reports [ftip-00F6]

The source reports two backbones, eight benchmark families, convergence stopping, and archive ablations in Sections 3.1--3.4 of [kim2026metanrecursive]; its fixed-driver, conditioning, stopping, and archive setup is described in Sections 2.2--2.4. These are empirical comparisons under its task, model, and budget choices; they are not universal depth, stability, or capability theorems.

Remark. Recursive harnesses with fixed evaluators and versions [ftip-00F7]

The finite statements assume a fixed task set, evaluator, versions, and declared budget. They do not transfer to weight learning, RLVR optimization, unbounded self-modification, or capability acquisition without new hypotheses.

Example. Whole-chain maximum and per-task oracle [ftip-00F8]

Consider two equally weighted tasks and an archive containing two chains $a,b$, with score vectors $(s_1(a),s_2(a))=(1,0)$ and $(s_1(b),s_2(b))=(0,1)$ under the same evaluation law. The archive envelope is

\[J(a)=J(b)=J_D^{\max }=\frac 12.\]

Either chain attains this maximum. In contrast, a per-task oracle has value

\[\frac 12\sum _{i=1}^2\max _{c\in \{a,b\}}s_i(c)=1.\]

The oracle selects $a$ on the first task and $b$ on the second. A configuration constrained to select one archived chain before observing task identity obtains mean $1/2$ in this example. If task-dependent routing is permitted, the resulting combined policy must itself be declared and evaluated, including its routing and execution costs. The oracle value is not automatically the score of either archived chain.

Incompatible execution configurations create a separate obstacle: an archived maximizing chain may be ineligible for a named target configuration. That feasibility restriction does not alter the distinction between the two score functionals above.

Reward, verifier, and environment failure modes [ftip-00DX]

A reward channel, a verifier, and the environment transition law describe different aspects of an interaction. A high score in one channel is not by itself a utility, support, or capability-acquisition statement.

The catalog of [dharna2026aifindsway] supplies heterogeneous empirical counterexamples and provenance, not rates or a general theorem.

Reward semantics and environment coupling [ftip-00DY]

The same response can be scored by several channels while the environment can assign different transitions or hidden consequences. A score, a state transition, and a hidden consequence are distinct functions of the interaction.

Definition. Named reward channel [ftip-00DZ]

For a fixed task $x$, let $\mathcal Y_x$ be a finite response set and let a named reward channel be a map $r:\mathcal Y_x\to \mathbb R$. A utility map $u:\mathcal Y_x\to \mathbb R$ is a separate evaluation quantity.

If an interaction has state space $\mathcal S$, action space $\mathcal A$, and transition kernel $K(\mathord {\cdot }\mid s,a)$, then $K$ is a third object: changing $K$ can change consequences without changing $r$.

Definition. Verifier channel and false acceptance [ftip-00E0]

A verifier channel is a map $V:\mathcal Y_x\to \{0,1\}$. Given a validity map $U:\mathcal Y_x\to \{0,1\}$, a false-accept event is

\[F=\{y\in \mathcal Y_x:V(y)=1\ \text {and}\ U(y)=0\}.\]

The event depends on the declared verifier and validity test. It is not identified by a scalar reward unless equivalence with that reward criterion is an explicit assumption.

Remark. What reward-hacking anecdotes establish [ftip-00E1]

Sections 4.1--4.2 and 7.2--7.3 of [dharna2026aifindsway] catalogue reward, score, environment, and evaluator-target anecdotes. The source gives 26 curated firsthand anecdotes involving more than 100 researchers; it does not provide a common sampling frame, base rates, or causal effect estimates.

Accordingly, this section uses the paper as an empirical counterexample catalog. The finite statements that follow are proved here and do not claim to summarize all training systems.

Theorem. Uniform proxy disagreement gives two-epsilon regret [ftip-00E2]

Let $\mathcal Y_x$ be finite, let $u,r:\mathcal Y_x\to \mathbb R$, and assume $|r(y)-u(y)|\le \varepsilon $ for every $y$, with $\varepsilon \ge 0$. If $y^\star $ maximizes $u$ and $\widehat y$ maximizes $r$, then

\[u(y^\star )-u(\widehat y)\le 2\varepsilon .\]

Indeed, $u(y^\star )\le r(y^\star )+\varepsilon \le r(\widehat y)+\varepsilon \le u(\widehat y)+2\varepsilon $. This is a finite same-class decision bound, not an optimization, distribution-shift, or capability theorem.

Example. Equal nominal score, different utility [ftip-00E3]

Take $\mathcal Y_x=\{a,b\}$, with $r(a)=r(b)=1$ but $u(a)=1$ and $u(b)=0$. Every reward maximizer is a nominal tie, while only $a$ is utility optimal. The example shows why a score equality does not establish intent or validity.

Verifier false accepts [ftip-00E4]

False acceptance is an event over candidates and a declared validity test. Repeated auditing can bound its occurrence, but a bound on existence is not automatically a bound on the selected output.

Definition. False-accept event for a candidate [ftip-00E5]

For candidate index $i$, let $I_i\in \{0,1\}$ indicate invalidity and let $A_i\in \{0,1\}$ indicate verifier acceptance. Define $F_i=\{I_i=1,A_i=1\}$. Write $q_i=\Pr (I_i=1)$. If $q_i>0$, write $\eta _i=\Pr (A_i=1\mid I_i=1)$; if $q_i=0$, set $\eta _i=0$. Then $\Pr (F_i)=q_i\eta _i$.

No independence or identical-distribution hypothesis is part of this definition.

Lemma. Union bound for repeated false accepts [ftip-00E6]

For finitely many candidate events from Definition [ftip-00E5],

\[\Pr \left (\bigcup _{i=1}^{N}F_i\right )\le \sum _{i=1}^{N}\Pr (F_i) =\sum _{i=1}^{N}q_i\eta _i.\]

This is the union bound and requires no independence. If every marginal is at most $\eta $, the right side is at most $N\eta $.

Example. Existence of a false accept is not selected-output failure [ftip-00E7]

Suppose two candidates are produced: candidate 1 is invalid and falsely accepted, while candidate 2 is valid and rejected by a separate score tie-break. Then $\bigcup _iF_i$ occurs, but a selection rule that always chooses candidate 2 outputs a valid response. Thus an existence probability and a selected-output probability coincide only after the selection law is specified.

Environment and oversight targets [ftip-00E8]

An evaluator can be an optimization target while hidden environment consequences remain outside its score. A policy can therefore improve the measured score while changing an unmeasured environmental outcome.

Definition. Environment exploit [ftip-00E9]

Given transition kernel $K$, reward $r$, and validity predicate $U$, an environment exploit is a candidate $y\in \mathcal Y_x$ whose induced interaction under $K$ receives high $r(y)$ while failing $U$. The definition is relative to the declared kernel, horizon, and validity test; it is not a universal property of a model.

Example. Same reward, different transition semantics [ftip-00EA]

Let one response receive $r(y)=1$ under two environments, and let $W:\mathcal S\to \{0,1\}$ test terminal-state safety. In $K_1$, its next state $s_1$ has $W(s_1)=1$; in $K_2$, the same observed response enters $s_2$ with $W(s_2)=0$. The reward channel alone cannot distinguish the two transition semantics, so equal reward does not certify safe consequences.

Example. Evaluator-target behavior [ftip-00EB]

Let $V(y)=1$ for every output that contains a visible marker, while $U$ checks a hidden task condition that the marker does not affect. A policy that optimizes $V$ can improve its observed score while leaving $U$ unchanged or worse. This is a finite illustration of evaluator targeting, not a claim about the frequency of such behavior.

Remark. No base rates or causal estimates [ftip-00EC]

The source catalog does not identify the base rate of exploits, the causal effect of a reward intervention, or a universal relationship between score and utility. Heterogeneous anecdotes therefore motivate audit questions and failure tests, not population-level probabilities.

Finite remediation protocol [ftip-00ED]

A remediation protocol records what was scored, what was audited, and which environment and verifier versions were used. It may reject candidates before a commit, but its guarantee is only the finite event bound proved below.

Definition. Audit record [ftip-00EE]

An audit record is a tuple containing candidate identity, evaluator and verifier version, environment or transition-kernel version, invalidity tests, random seed, evidence pointers, and adjudicator decision. A record is complete only when these fields are bound to the candidate and the protocol run.

Proposition. Finite audit-gate bound [ftip-00EF]

Let $N\in \mathbb N_{\geq 1}$ and $0\leq \eta \leq 1$. Assume these $N$ candidates are audited and each invalid candidate is falsely accepted with marginal probability at most $\eta $. Then the probability that some invalid candidate is accepted is at most $N\eta $, by the union bound of Lemma [ftip-00E6]. This conclusion does not require independence.

The statement bounds existence of an accepted invalid candidate. A selected output guarantee additionally requires a typed selection rule and its relation to the audit decisions.

Remark. Verifier sensitivity and unmeasured failures [ftip-00EG]

These finite channel distinctions do not prove reward hacking rates, capability acquisition, or safety of a deployed agent. The next questions are to measure verifier sensitivity, environment changes, selection rules, and independent audit power under a declared protocol.

Reliability protocol and failure taxonomy [ftip-00GR]

A reliability protocol binds an evaluation to its durable state, replay inputs, and audit record. Execution and grading failures can invalidate a measurement even when the reported benchmark score improves.

Measurement and contamination controls [ftip-00GS]

Convention. Evidence required for a reliability claim [ftip-00GT]

Interpreting a reliability claim requires a named estimand, task and evaluation laws, artifact revision, inference protocol, evaluator version, sample rule, and failure policy. Missing coordinates are unknown rather than implicitly held fixed.

Definition. Contamination record [ftip-00GU]

For a candidate evaluation item $z$, a contamination record is

\[c(z)=(source,split,exposure,overlap,decision)\]

The fields identify source provenance, split membership, prior agent exposure, overlap evidence, and the admission decision; an absent field is an explicit unknown.

Definition. A contamination predicate [ftip-00GV]

Given a contamination record $c(z)$, write $\operatorname {cont}(z)=1$ when its declared overlap or exposure rule rejects $z$, and write $\operatorname {cont}(z)=0$ when the rule clears it. The predicate belongs to the protocol, not to a model’s score.

Definition. An uncontaminated evaluation slice [ftip-00GW]

For an evaluation law $\mathsf Q$, an admitted slice is $\mathcal Z_0=\{z:\operatorname {cont}(z)=0\}$. Its reported score is conditioned on the declared slice law; removing contaminated items does not preserve the original estimand unless the law is unchanged by construction.

Definition. A paired comparison design [ftip-00GX]

A paired design fixes one evaluation law, one item order, and one sampling kernel for two artifacts, then records the paired difference from Definition [ftip-00FE]. Any item-level exclusion is applied before seeing the paired outcomes or is declared as an adaptive rule.

Lemma. Pairing preserves the declared marginal target [ftip-00GY]

Under a fixed nonadaptive slice and the iid coupling of Definition [ftip-00FE], the paired estimator remains unbiased for the two marginal scores in Theorem [ftip-00FF].

Proof.

Condition on the fixed slice. The coupling has the stated marginals, so the proof of Theorem [ftip-00FF] applies without changing either expectation.

Example. A contamination decision can change the estimand [ftip-00GZ]

If one paired run scores all ten items but a second run removes two items after inspecting outputs, the two means answer different questions. A report must expose the removal rule and the resulting slice law.

Remark. Scope of measurement methodology [ftip-00H0]

The measurement and paired-comparison guidance in [jarmak2026reliable], Part I, is a methodology source. It motivates contamination ledgers and matched comparisons; it supplies no universal contamination detector or statistical guarantee.

Grader calibration [ftip-00H1]

Definition. A typed grading rubric [ftip-00H2]

A grading rubric is $R=(\mathcal Z,\mathcal L,g,v)$: outcome space, finite label set, scoring map $g:\mathcal Z\to \mathcal L$, and evaluator version $v$. The rubric declares which labels count as success and which observations are abstentions.

Definition. A grader response law [ftip-00H3]

For rubric $R$, a grader response law is a kernel $K(\ell \mid z)$ on $\mathcal L$ given outcome $z$. A deterministic grader is the point-mass case; stochastic or model-based graders must retain their version and sampling coordinates.

Definition. A calibration set [ftip-00H4]

A calibration set is a finite set $\mathcal C\subseteq \mathcal Z$ with an adjudicated label $y^star(z)$ for each $z\in \mathcal C$. The adjudication source, disagreement policy, and release version are part of the set’s provenance.

Definition. Calibration agreement [ftip-00H5]

For grader kernel $K$ and calibration set $\mathcal C$, the agreement rate is $A(K,\mathcal C)=|\mathcal C|^{-1}\sum _{z\in \mathcal C} K(y^star(z)\mid z)$. It is a calibration-set statistic, not a population accuracy claim.

Lemma. Agreement threshold admission [ftip-00H6]

If $|\mathcal C|=n\geq 1$ and $A(K,\mathcal C)\geq 1-\eta $, then the empirical disagreement mass $1-A(K,\mathcal C)$ is at most $\eta $.

Proof.

Rearrange the defining finite average in Definition [ftip-00H5]. No generalization beyond $\mathcal C$ is implied.

Example. High aggregate agreement can hide a subgroup failure [ftip-00H7]

A grader that agrees on 99 of 100 easy cases but disagrees on the one safety case has overall agreement $99/101\approx 0.9802$. Its agreement within the safety subgroup is zero. Any positive safety-subgroup agreement threshold therefore rejects this grader despite its high overall agreement. A calibration report stratifies by failure-relevant labels.

Definition. A grader release gate [ftip-00H8]

A grader release gate accepts $K$ only when its calibration version, agreement thresholds, abstention handling, and stratified failure checks are recorded. A failed gate blocks interpretation of downstream score changes.

Remark. Scope of grading methodology [ftip-00H9]

Part II of [jarmak2026reliable] organizes grading and reviewer calibration practices. The finite agreement quantities do not establish a universal threshold or blinded-human reliability theorem.

Execution gates and durable state [ftip-00HA]

Definition. An execution gate [ftip-00HB]

An execution gate is a predicate $g_i(e_i)$ over a recorded stage result $e_i$. It names its owner, required inputs, pass/fail/unknown outputs, and the artifact versions to which the result applies.

Definition. An ordered gate chain [ftip-00HC]

An ordered gate chain is $(g_1,\ldots ,g_k)$ with stage outputs $e_1,\ldots ,e_k$; gate $g_i$ may execute only after its declared prerequisites and records a monotone status in $\{pass,fail,unknown\}$.

Theorem. A failed gate blocks a release conjunction [ftip-00HD]

For a finite chain, define release status as $G=\bigwedge _{i=1}^k g_i(e_i)$. If any gate is false, then $G$ is false.

Proof.

This is the defining conjunction of finitely many Boolean gate predicates. It says nothing about whether a later gate would have passed.

Definition. A durable execution record [ftip-00HE]

A durable execution record is

\[d=(run,revision,environment,inputs,events,artifacts,checks)\]

It is append-only, versioned, and sufficient to locate each gate input and output without relying on worker-local memory.

Definition. A replay contract [ftip-00HF]

A replay contract for $d$ fixes the executable revision, environment image, input artifact hashes, seed law, and event order. A replay is faithful only when those coordinates are available and the resulting events satisfy the recorded schema.

Theorem. Deterministic replay reproduces a recorded trace [ftip-00HG]

If the execution map is deterministic in the coordinates fixed by Definition [ftip-00HF], replaying the same inputs, revision, environment, and seed produces the same event trace.

Proof.

Induct over the finite event sequence. Equal initial coordinates give equal first events; determinism and equal prefixes give the next event.

Example. Replayability is not correctness [ftip-00HH]

A reproducible run can deterministically reproduce a wrong patch or a misconfigured evaluator. Replay establishes trace identity under its contract; it does not establish that the trace met the intended utility or safety goal.

Definition. A state-integrity digest [ftip-00HI]

For durable record $d$, let $H(d)$ be a cryptographic digest of its canonical serialized fields. A replay request fails closed when the supplied record digest or any referenced input hash differs from the declared value.

To bind a replay result to an audit decision, the replay trace must be included in the audited record and covered by its digest. A matching digest for other fields does not bind that trace to the decision.

Remark. Scope of execution methodology [ftip-00HJ]

Part III of [jarmak2026reliable] motivates containment, durable execution, and recovery records. Deterministic replay and matching digests establish execution consistency; they do not ensure correctness.

Audit before commit and failure taxonomy [ftip-00HK]

Definition. An audit-before-commit record [ftip-00HL]

An audit-before-commit record is $a=(d,scope,checks,reviewer,decision)$. It binds the durable execution record $d$ to the audited scope, check list, review identity, and a decision in $\{commit,hold,reject\}$.

Theorem. Audit-before-commit safety gate [ftip-00HM]

A commit decision is admissible only if every required check in an Definition [ftip-00HL] record is pass, the audited digest equals the proposed record digest, and the decision is $commit$.

Proof.

Admissibility is defined as the conjunction of these three recorded conditions; failure of any conjunct yields hold or reject.

Definition. An independent audit scope [ftip-00HN]

An audit is independent for scope $S$ when its inputs are frozen before review, its reviewer or checker is distinct from the proposing action, and its decision cannot rewrite $S$ without a new record. Independence is a protocol condition, not a claim of infallibility.

Example. A latent failure caught before commit [ftip-00HO]

A run may pass unit tests while an independent audit finds that the evaluator version in $d$ differs from the declared version. The audit holds the commit and records the mismatch; replay alone would not have caught it.

Definition. A reliability failure taxonomy [ftip-00HP]

Classify a failed run by its earliest violated contract: measurement failure (M), grading failure (G), execution/state failure (E), audit failure (A), or transfer failure (T). A single run may receive secondary labels, but the primary label is the earliest failed gate in recorded order.

Lemma. Earliest-failure labels are unique [ftip-00HQ]

For a finite ordered gate chain with at least one failed gate, the earliest failed index is unique and therefore determines one primary taxonomy label.

Proof.

Every nonempty finite subset of ordered indices has a unique least element.

Definition. A remediation record [ftip-00HR]

A remediation record maps a primary label from Definition [ftip-00HP] to an owner, corrective action, re-test scope, and closure condition. Closure requires a new durable record and cannot mutate the failed record in place.

Remark. Scope of audit methodology [ftip-00HS]

Parts III and V of [jarmak2026reliable] motivate durable execution, review, and accountability practices. The audit taxonomy and conjunction gates are proposed operational specifications; they do not establish that an audit is complete or that a committed artifact is correct.

Audit budgets and the scope of run-level evidence [ftip-00HT]

Remark. A protocol is not a causal estimator [ftip-00HU]

The gates above make evidence auditable. They do not identify the causal effect of an intervention when task law, artifact, grader, budget, or harness state changes together. Those coordinates must be controlled in the matched evaluation of Definition [ftip-00GF].

Example. Counterexample: a perfect replay can repeat a contaminated result [ftip-00HV]

Let a deterministic run train and evaluate on one leaked item. Its digest, event trace, and replay are all identical across reruns, yet the measurement is contaminated under Definition [ftip-00GV]. Replayability and contamination control are independent obligations.

Example. Counterexample: a calibrated grader can still face a changed domain [ftip-00HW]

A grader can meet its threshold on $\mathcal C$ in Lemma [ftip-00H6] while every deployment item lies outside that calibration domain and receives an unvalidated label. Agreement on $\mathcal C$ alone does not transport to a new law.

Theorem. Budgeted audit admission [ftip-00HX]

If an audit plan has finite nonnegative stage costs and its declared sum is at most budget $B$, then the plan is budget-admissible; adding any positive stage beyond the remaining slack makes it inadmissible.

Proof.

Apply additive cost monotonicity from Lemma [ftip-00GG].

Example. A cost-balanced audit plan [ftip-00HY]

With budget $B=100$, a plan may allocate 50 units to execution, 30 to grading calibration, and 20 to audit. A proposed 10-unit extra audit must be funded by reducing another stage or the admission gate fails.

Remark. What this protocol can establish [ftip-00HZ]

A passing protocol establishes that declared evidence contracts, hashes, checks, and budgets were satisfied for the recorded run. It does not establish capability acquisition, broad generalization, or absence of unobserved faults.

Backlinks

Architecture-conditioned potential and computation [ftip-00JC]

This chapter refines the evaluation framework for claims that depend on architecture, optimization geometry or a systems implementation. It begins with the decoder-only Transformer, then maps mechanism families to capability bottlenecks before introducing architecture-indexed frontiers and matched-compute comparisons. It also studies how an optimizer can depend on the representation of its parameters.

The reference architecture specifies how token probabilities are computed; it does not replace the shared task and evaluation interfaces. The mechanism-family map asks which bottleneck a design changes and what evidence would demonstrate that change; it does not rank model families. Readers whose question does not depend on an architecture or optimizer can continue directly to conceptual discovery. Changes to retained context and rollout computation are treated with agent state.

Decoder-only Transformer architecture [ftip-000K]

The continuation law in Definition [ftip-000I] specifies what probabilities a language model exposes. A decoder-only Transformer specifies how a sequence of token identifiers is transformed into those probabilities. This section introduces one component at a time, with dimensions fixed before the component formulas are used.

Notation. Tensor dimensions, positions, heads, and layers [ftip-000L]

Let $T$ be the maximum token positions processed together, $d$ the residual-stream width, $d_{\mathrm {ff}}$ a feed-forward hidden width, $H$ the number of attention heads, $d_h$ the query-key width of one head, and $L$ the number of decoder layers. Positions are indexed by $t,s\in \{1,\ldots ,T\}$, heads by $h\in \{1,\ldots ,H\}$, and layers by $\ell \in \{1,\ldots ,L\}$.

A batch of $B$ hidden-state sequences is a tensor in $\mathbb R^{B\times T\times d}$. When the batch coordinate is irrelevant, write $X\in \mathbb R^{T\times d}$ and let $X_t\in \mathbb R^d$ denote its row at position $t$. Matrix products act on the final coordinate.

Definition. Token embedding [vaswani2017attention, sec. 3.4, Embeddings and Softmax] [ftip-000M]

Let $\mathcal V$ be the vocabulary of Definition [ftip-000D] and $d$ the residual width of Notation [ftip-000L]. A token embedding matrix is a learned matrix $E\in \mathbb R^{|\mathcal V|\times d}$. The hidden vector assigned to token $a\in \mathcal V$ is the row $E_a\in \mathbb R^d$.

For a sequence $x_{1:n}$, embedding lookup produces the matrix $X^{\mathrm {tok}}\in \mathbb R^{n\times d}$ with row $X^{\mathrm {tok}}_t=E_{x_t}$. This operation maps token identifiers to vectors; it does not yet encode their positions.

Definition. Embedding-stage positional transformation [ftip-000N]

Using the dimensions of Notation [ftip-000L], an embedding-stage positional transformation is a specified family of position-dependent transformations $r_t:\mathbb R^d\to \mathbb R^d$, one for each $t\in \{1,\ldots ,T\}$. Applied after the token embedding of Definition [ftip-000M], it produces the initial hidden vector \[ X^{(0)}_t=r_t(E_{x_t}). \] The family $(r_t)_{t=1}^T$ is part of the architecture, even when it has no learned parameters.

Remark. Where positional information enters [ftip-000O]

Definition [ftip-000N] covers only mechanisms acting on the hidden vector at the embedding stage. The original Transformer adds a fixed or learned vector $p_t$ to the token embedding, so $r_t(z)=z+p_t$; see [vaswani2017attention, §3.5].

Relative-bias methods instead alter an attention score as a function of the displacement $t-s$. ALiBi gives one such construction in [press2021train, §3].

Rotary position embedding applies position-indexed rotations to queries and keys rather than adding a vector to the residual stream; see RoFormer, Section 3.2, equations (14)--(16) [⧉]. These mechanisms are not interchangeable choices of one tensor: they act at different points in the computation, so a model specification must name the mechanism and placement.

Definition. Query, key, and value projections [vaswani2017attention, sec. 3.2.1] [ftip-000P]

Let $X\in \mathbb R^{T\times d}$ use the notation of Notation [ftip-000L]. For attention head $h$, choose learned matrices \[ W_h^Q,W_h^K\in \mathbb R^{d\times d_h}, \qquad W_h^V\in \mathbb R^{d\times d_v}, \] where $d_v$ is the value width of one head. The query, key, and value matrices are \[ Q_h=XW_h^Q,\qquad K_h=XW_h^K,\qquad V_h=XW_h^V. \] Their rows associate a query, key, and value vector with each token position.

Definition. Scaled dot-product attention [vaswani2017attention, sec. 3.2.1, equation (1)] [ftip-000Q]

Using the head widths in Definition [ftip-000P], let $T_q,T_k\geq 1$ be query and key-value sequence lengths. For $Q\in \mathbb R^{T_q\times d_h}$, $K\in \mathbb R^{T_k\times d_h}$, and $V\in \mathbb R^{T_k\times d_v}$, scaled dot-product attention is \[ \operatorname {Att}(Q,K,V) =\operatorname {softmax}\left (\frac {QK^{\mathsf T}}{\sqrt {d_h}}\right )V. \] The softmax is applied across each row of the $T_q\times T_k$ score matrix, so every output row is a convex combination of the value rows.

Example. A two-token attention calculation [ftip-000R]

A scalar attention head on two positions makes the masked softmax and its resulting value average explicit.

$\text{[math]}$

Take head width $d_h=1$, so the scale $1/\sqrt {d_h}$ is one. At position two both keys are causally available. Therefore \[ \operatorname {softmax}(0,\log 3) =\frac {(1,3)}{1+3} =\left (\tfrac 14,\tfrac 34\right ), \qquad z_2=\tfrac 14\cdot 2+\tfrac 34\cdot 6=5. \]

Substitution into the scaled dot-product attention of Definition [ftip-000Q] produces the displayed value. Projection matrices, multiple heads, normalization, and residual connections remain outside this scalar calculation; it represents only one attention operation.

Definition. Causal attention mask [vaswani2017attention, sec. 3.2.3] [ftip-000S]

For a length-$T$ sequence, the causal attention mask is the matrix $M\in (\mathbb R\cup \{-\infty \})^{T\times T}$ with \[ M_{ts}= \begin {cases} 0,&s\leq t,\\ -\infty ,&s>t. \end {cases} \] Masked self-attention replaces the score matrix in Definition [ftip-000Q] by $QK^{\mathsf T}/\sqrt {d_h}+M$. Consequently, the output at position $t$ has zero attention weight on every later position $s>t$.

Example. A length-four causal mask [ftip-000T]

For four positions, a causal mask admits the current and earlier keys and excludes every future key.

$\text{[math]}$

Rows index querying positions and columns index key positions. In row three, the first three entries remain available and the fourth receives $-\infty $; after softmax the fourth token therefore contributes exactly zero.

The construction is the autoregressive mask of [vaswani2017attention, Section 3.2.3]. It enforces a dependency restriction. Effective use of the available earlier positions is a separate model property.

Definition. Multi-head self-attention [vaswani2017attention, sec. 3.2.2] [ftip-000U]

For each head $h\in \{1,\ldots ,H\}$, form $Q_h,K_h,V_h$ as in Definition [ftip-000P] from the same input $X$, and apply the masked attention of Definition [ftip-000S]: \[ Z_h=\operatorname {softmax}\left ( \frac {Q_hK_h^{\mathsf T}}{\sqrt {d_h}}+M \right )V_h. \] With $W^O\in \mathbb R^{H d_v\times d}$, multi-head self-attention is \[ \operatorname {MHA}(X) =\operatorname {Concat}(Z_1,\ldots ,Z_H)W^O. \] It returns one width-$d$ vector at every input position.

Remark. Multi-head, grouped-query, and compressed-attention variants [ftip-000V]

The multi-head construction in Definition [ftip-000U] stores a key and value projection per query head. Grouped-query attention instead partitions query heads into groups that share key and value heads; see Ainslie et al., Section 2 [⧉]. This changes parameter and cache shapes while preserving the query-key-value semantics.

Multi-head latent attention first compresses key-value information through a lower-dimensional latent representation and reconstructs head-specific quantities; see DeepSeek-V2, Section 2.1 [⧉]. DeepSeek-V3, Section 2.1, records a later use of that architecture [⧉]. These architectural differences do not establish that their post-training effects equal those of ordinary multi-head attention.

Definition. Positionwise feed-forward sublayer [vaswani2017attention, sec. 3.3, equation (2)] [ftip-000W]

With widths $d$ and $d_{\mathrm {ff}}$ from Notation [ftip-000L], a positionwise feed-forward sublayer is a map $F:\mathbb R^d\to \mathbb R^d$ of the form \[ F(z)=\phi (zW_1+b_1)W_2+b_2, \] where $W_1\in \mathbb R^{d\times d_{\mathrm {ff}}}$, $W_2\in \mathbb R^{d_{\mathrm {ff}}\times d}$, the vectors $b_1,b_2$ have the corresponding widths, and $\phi $ is applied coordinatewise. The same map is applied independently at each token position.

Definition. Sparse mixture-of-experts routing [ftip-000X]

Let $J\geq 1$ and $1\leq k\leq J$. Let $F_1,\ldots ,F_J:\mathbb R^d\to \mathbb R^d$ be feed-forward experts and let $g:\mathbb R^d\to \mathbb R^J$ be a router. For a hidden vector $z$, let $S_k(z)$ be the indices of the $k$ largest coordinates of $g(z)$, breaking equal scores by a fixed total order on expert indices. A sparse mixture-of-experts layer returns \[ F_{\mathrm {MoE}}(z) =\sum _{j\in S_k(z)}\alpha _j(z)F_j(z), \qquad \alpha _j(z)= \frac {\exp g_j(z)}{\sum _{i\in S_k(z)}\exp g_i(z)}. \] Only the selected experts are evaluated for that token.

Remark. Dense and routed feed-forward computation [ftip-000Y]

The dense sublayer in Definition [ftip-000W] evaluates one learned map for every token. The sparsely gated construction of Shazeer et al., Section 2, selects among multiple experts and introduces an additional routing and load-balancing problem [⧉]. The top-$k$ selection rule is described in Definition [ftip-000X].

Parameter count, active parameter count, and executed operations therefore need not agree. A post-training comparison involving routed models must state which quantity is held fixed and whether the router itself is updated.

Example. Top-k expert routing and its DeepSeek counterpart [ftip-000Z]

A routed mixture-of-experts layer uses router scores to activate only a declared subset of routed experts, then combines their outputs with any shared expert path.

$\text{[math]}$

The revision-pinned configuration [⧉] declares $256$ routed experts, $6$ experts selected per token, and one shared expert. This instantiates the selection interface while leaving the artifact's router scores, load-balancing mechanism, and expert outputs unspecified.

Definition. Residual connection [vaswani2017attention, sec. 3.1] [ftip-0010]

Using the sequence and residual dimensions of Notation [ftip-000L], let $F:\mathbb R^{T\times d}\to \mathbb R^{T\times d}$ be a sublayer whose input and output have the same shape. A residual connection forms \[ R_F(X)=X+F(X). \] The addition is coordinatewise. Shape equality is part of the construction; the residual path is not a concatenation of features.

Definition. Layer normalization [ba2016layer, sec. 3] [ftip-0011]

For the residual width $d$ of Notation [ftip-000L], a declared numerical stabilizer $\varepsilon _{\mathrm {LN}}>0$, and $z=(z_1,\ldots ,z_d)\in \mathbb R^d$, define $\mu (z)=d^{-1}\sum _i z_i$ and $\sigma ^2(z)=d^{-1}\sum _i(z_i-\mu (z))^2$. With learned vectors $\gamma ,\beta \in \mathbb R^d$, layer normalization is \[ \operatorname {LN}(z) =\gamma \odot \frac {z-\mu (z)\mathbf 1} {\sqrt {\sigma ^2(z)+\varepsilon _{\mathrm {LN}}}}+\beta . \] It is applied independently to the hidden vector at each token position. The declared stabilizer makes the numerical operator total on $\mathbb R^d$; the unstabilized expression is recovered where $\sigma ^2(z)>0$ by setting $\varepsilon _{\mathrm {LN}}=0$.

Definition. Normalization placement [ftip-006E]

For a residual stream $X$, a sublayer $F$, and a normalization operator $N$, post-normalization applies $N(X+F(X))$. In pre-normalization, the sublayer receives a normalized input and the residual update is $X+F(N(X))$.

Placement is an architecture coordinate, separate from the formula for $N$. Every decoder block must state which residual branches use which placement.

Definition. Root mean square layer normalization [zhang2019root, Section 4, equation (4)] [ftip-006F]

For $z\in \mathbb R^d$, learned scale $\gamma \in \mathbb R^d$, and a declared numerical stabilizer $\varepsilon _{\mathrm {RMS}}>0$, root mean square layer normalization is

\[ \operatorname {RMSNorm}(z) =\gamma \odot \frac {z}{\sqrt {d^{-1}\sum _{i=1}^{d}z_i^2+ \varepsilon _{\mathrm {RMS}}}}. \]

Unlike layer normalization in Definition [ftip-0011], this operator does not subtract the coordinate mean. The declared stabilizer makes it total at the zero vector; setting the stabilizer to zero recovers the source expression on nonzero inputs.

Remark. Normalization operator and placement are separate choices [ftip-006G]

Ba et al. define layer normalization in [ba2016layer, Section 3]. The original Transformer applies it after residual addition in [vaswani2017attention, Section 3.1]. Xiong et al. distinguish post-normalized and pre-normalized Transformers in [xiong2020layer, Section 3.3]. Either the normalization operator or its placement can change while the other is held fixed.

A later stability statement must therefore name both choices. A result about the normalization formula alone does not determine the residual path.

Definition. Decoder-only Transformer language model [phuong2022formal, sec. 6, Decoder-only transformers] [ftip-0012]

A decoder-only Transformer language model consists of:

the token interface in Definition [ftip-000E];
an initial representation from Definition [ftip-000M] and Definition [ftip-000N];
a stack of $L$ causal decoder layers built from the attention, feed-forward, residual, and normalization constructions in Definition [ftip-000S]--Definition [ftip-0011]; and
an output map from the final hidden vector to token logits.

If the final hidden vector at position $t$ is $h_t\in \mathbb R^d$, an output matrix $W_U\in \mathbb R^{|\mathcal V|\times d}$ and bias $b_U\in \mathbb R^{|\mathcal V|}$ give logits $z_t=W_Uh_t+b_U$. The softmax \[ \pi _\theta (a\mid x_{1:t}) =\frac {\exp (z_{t,a})} {\sum _{v\in \mathcal V}\exp (z_{t,v})} \] realizes the conditional next-token law of Definition [ftip-000H]. The parameter tuple $\theta $ contains every learned tensor named by this architecture.

Example. Decoder-only dataflow and a DeepSeek V4-Flash provenance overlay [ftip-0013]

A decoder pipeline maps token identifiers through hidden states to logits; the released model instance and family report provide different evidence about that pipeline.

$\text{[math]}$

The pinned configuration [⧉] declares 43 layers, hidden width 4096, 64 attention heads, and one key--value head. Section 2 of the family report [⧉] describes the preview architecture. Neither source identifies a complete 0731-specific architecture or training delta. The diagram records the interface, not a small copy of the released model.

Mechanism families and capability bottlenecks [ftip-00OQ]

This section compares mechanism families by the capability bottlenecks they change and by the evidence needed to detect those changes. The shared model, task, intervention, cost and evaluation interfaces remain fixed conditions for every claim, while source descriptions, measured results and transfer claims remain separate.

The comparison uses five capability dimensions: state access, the information that can remain available across tokens, steps or external memory operations; adaptive computation, conditional allocation of depth, recurrence, search or stopping together with its charged cost; representation and abstraction, the available representation, composition or predictive state; plasticity, the changes reachable by a declared objective, optimizer, parameter tying and post-training recipe; and portability, survival of a result when the model family, task, scale or deployment regime changes. This is distinct from transfer, which evaluates a frozen artifact on new problems under a common deployment budget. They are coordinates for the capability vector, not a single intelligence scalar.

A bottleneck is the declared resource or interface condition that limits one of these dimensions in a comparison. Cost, numerical stability and monitorability remain evaluation coordinates: the ability to observe the relevant state, compute, update or failure signals under the study protocol. A matched-compute comparison fixes the cost accounting before comparing outcomes; its protocol and the architecture frontier remain authoritative elsewhere in this chapter.

The default core panel contains mechanism families that alter the sequence computation or its state interface. Existing Transformer notation and the decoder-only reference are in § [ftip-000K]; the entries below identify the question each family makes testable.

Transformer reference. Attention exposes token-indexed content access through the declared context and serves as the baseline for the matched-compute comparison and interface comparisons.
Selective state-space models. The Mamba construction, Sections 3.1--3.3 makes the state-space parameters input-dependent and computes the resulting time-varying recurrence with a hardware-aware scan. The testable bottleneck is how selective compression trades long-context access and throughput against retrieval and portability boundaries; a source mechanism does not settle that tradeoff.
Linear attention. The linear-attention formulation, Sections 3.2--3.3 uses a feature-map factorization and associative prefix summaries rather than the full softmax attention matrix. The comparison tests which retrieval and memory tasks preserve their evaluation under the changed state interface; it does not identify every hybrid or recurrent model with pure linear attention.
Internal-tick and looped models. The Continuous Thought Machine, Sections 3.1--3.5 uses a discrete internal tick dimension, private neuron-level models that process activation histories, synchronization as a latent representation, and standard cross-attention to the input features. Its loss selects minimum-loss and maximum-certainty ticks, making variable internal compute a testable mechanism. The looped computations in § [ftip-00KG] are a separate family; neither is a continuous-time claim, and every extra tick or loop must be charged in the comparison.
Recurrent or compressed-state mechanisms. The Dragon Hatchling / BDH proposal, Section 3.2 defines a tensor-friendly BDH-GPU state-space system whose attention state admits a reduced representation, alongside local graph dynamics. Its open comparison is whether this state interface supports the target tasks and stable credit assignment at a declared budget; the proposal is not a production result.
Recurrent feedback. The [zhang2026rlt, Sections 2-3 and 5-8] splits a causal parallel encoder from a recurrent decoder whose previous-token state is fed back into the next token. On finite parity, permutation-tracking, and modular-arithmetic tasks, its reported length extrapolation depends on the feedback ablation and on chunk size. This is evidence for a state-access intervention in a bounded synthetic setting; it does not establish a frontier reasoning gain or a general recurrence law.
Residual rewrite. [zhang2026deepdelta, Sections 2-4 and 6] replaces an additive residual shortcut with a gated rank-one read/compare/target/write edit. Reported GPT-2 experiments are a finite mechanism observation whose effect remains confounded with residual capacity, compression, and rewrite behavior. A write-only control, capacity-matched control, multiple seeds, and a charged transfer task are needed before attributing a capability change to the rewrite interface.

The recurrent-feedback and residual-rewrite entries are separate intervention arms. Their paired records must expose the changed state or write interface, causal ablations, parameter and token budgets, inference FLOPs, memory, KV/cache or recurrent-state costs, chunking overhead, and monitorability. If the available evidence remains finite-only, the result stays finite-only and does not support ceiling or frontier language.

The adjunct panel changes an objective, temporal law or function-space bias without being interchangeable with the core architecture families. I-JEPA, Section 3 is an encoder/predictor training architecture as well as an embedding-space prediction objective: its context, target and predictor use ViT backbones, the target encoder is updated by exponential moving average, and masked target blocks are predicted from a context block. That predictive objective leaves a specific representation limit: retaining or recovering the latent state does not by itself identify which directions are invariant and which vary across matched observations. SplitJEPA, Sections 3.1--3.4 (the project overview) makes the boundary explicit—predictive recovery can leave a global orthogonal mixing—and resolves it under stationary Gaussian predictive dynamics, a full-rank second moment of variant differences across invariant correspondences, and representation normalization, by identifying invariant and variant subspaces up to within-block isometries without an observation decoder. Here SplitJEPA is a structured JEPA adjunct and a conditional identifiability result, not a general guarantee for I-JEPA. Liquid time-constant networks, Sections 1--2 supply input-dependent continuous dynamics and a numerical ODE-solver interface; the Neural Operator formulation, Sections 1 and 3 supplies a function-space lens with discretization-invariant operator layers. The Zoology study, Sections 3--5 provides controlled recall analyses of sequence mixers. These sources motivate hypotheses and controls; they do not establish a common capability ordering or make an adjunct a replacement architecture. The JEPA comparability interface specifies fields that must be fixed for a JEPA-like comparison; it is not the source of I-JEPA's architecture. Combining CTM's internal dynamics with I-JEPA's encoder/predictor package is a proposed joint intervention, not an established result, and would require separate attribution of both components.

For each panel row, a study should name the changed state or compute interface, the bottleneck dimension, the task and intervention, the evidence type, and the transport boundary. Existing architecture-indexed ceilings and matched compute, published paired measurements (§ [ftip-00KU] and § [ftip-00L4]), conditional frontiers (§ [ftip-00LH]) and study design (§ [ftip-00LS]) provide those controls. Blank, incompatible and unmeasured cells remain findings rather than scores to be filled by intuition.

Architecture-indexed ceilings and matched compute [ftip-00JF]

This optional section refines the architecture-neutral capability framework only when the question names an architecture, optimizer, or systems implementation. The general model/interface formulation remains available when these fields are unnecessary.

Architecture comparisons below are conditional on a declared task family, evaluation law, intervention class, and scalar cost. Toy bounds are finite results under their displayed assumptions and do not transfer to concrete model instances without verified interface, task, and cost assumptions.

Architecture, optimizer, and systems variables [ftip-00JG]

Architecture, optimizer, and systems fields refine the model/interface description. A comparison depends only on the fields named in its assumptions and conclusion.

Definition. Optional architecture refinement [ftip-00JH]

Let $M$ denote an existing model intervention. An architecture refinement is a record $A=(\mathcal X,\mathcal Y,\operatorname {Map}_A)$ whose map realizes the same declared input and output interface. Forgetting $A$ returns $M$; no architecture claim is made when the field is omitted.

Definition. Optimizer and systems records [ftip-00JI]

When needed, extend the record by $O$ for optimizer and state-update rules, and $S$ for kernels, precision, memory, cache, and scheduling. The triplet $(A,O,S)$ is descriptive; it is not a scalar intelligence score.

Definition. Architecture-indexed evaluation functional [ftip-00JJ]

Fix an architecture $A$, its base artifact, a declared intervention set $\mathfrak I_A$, and the common evaluation interface of Convention [ftip-005D]. Each $\eta \in \mathfrak I_A$ specifies a protocol $P_{A,\eta }=\operatorname {PostTrain}(A,\eta )$ in that measurable and absolutely integrable evaluation domain, with real performance $J_{\rm ev}(P_{A,\eta })$. Let $c_A(\eta )\in [0,+\infty ]$ be its scalar cost under the study's declared accounting rule: a fixed measured cost or an expected nonnegative cost, as specified. For a finite budget $C\in \mathbb R_{\geq 0}$, define \[ V_A(C)=\sup \left \{ J_{\rm ev}(P_{A,\eta }): \eta \in \mathfrak I_A,\ c_A(\eta )\leq C \right \}\in \overline {\mathbb R}. \] Infinite-cost interventions are infeasible at every such budget.

As in Definition [ftip-005M], the empty feasible set has value $-\infty $; an unbounded-above feasible score set has value $+\infty $. The value is real exactly when that set is nonempty and bounded above. Increasing $C$ enlarges the feasible set, so $V_A$ is nondecreasing. A finite supremum need not be attained by any intervention.

Frontier differences and derivatives are ordinary real operations only where the relevant values are finite, with differentiability additionally required for a derivative. Cost ratios use the domain in Definition [ftip-00JP]. The allowed intervention set, evaluation law, and accounting rule are part of the definition, so $V_A$ is not a universal intelligence function.

Remark. Architecture-neutral projection and gradual complexity [ftip-00JK]

A statement that depends only on the model/interface map is invariant under changes to architecture $A$, optimizer $O$, or systems $S$ that leave that map fixed. A conclusion about one of those fields instead requires assumptions that distinguish its possible values.

Definition. Three ceiling layers [ftip-00JL]

For fixed task and evaluation laws, distinguish the representational ceiling of $A$, the optimizer-reachable ceiling of $(A,O)$, and the systems-feasible frontier of $(A,O,S,C)$. Each layer is conditional on the objects named; none implies that the next layer attains it.

Definition. Capability vector and conditional frontier [ftip-00JM]

For an architecture record $A$, optimizer record $O$, systems record $S$, evaluation configuration $\mathfrak e$ from Definition [ftip-00CE], and declared intervention $\eta $ at cost cap $C$, define the capability vector

\[ \mathbf K(A,O,S,\mathfrak e,\eta ,C) =\bigl (K_{\rm state},K_{\rm adapt},K_{\rm repr}, K_{\rm plast},K_{\rm port}\bigr ). \]

Its coordinates respectively record state access, adaptive computation, representation and abstraction, plasticity, and portability under the named task, intervention, evaluation law, and cost accounting. A coordinate may be a descriptive field, an estimand, or $\bot $ when the record is missing, incompatible, or not measured. The tuple is therefore not a common scalar and does not order architectures without a declared comparison rule.

The five coordinates attach to the existing ceiling layers conditionally: representation claims first require a map from $A$; optimizer-reachable plasticity requires $(A,O)$; state, adaptive computation, and portability claims require the relevant $(A,O,S)$ interface, task family, and deployment regime. A systems-feasible claim also carries $C$, the cost vector and its declared scalarization. The records in Definition [ftip-00JH], Definition [ftip-00JI], Definition [ftip-00JL], and Definition [ftip-00CE] remain authoritative.

The scalar frontier $V_A(C)$ of Definition [ftip-00JJ] is a projection onto the declared utility $J_{\rm ev}$ for one architecture and intervention envelope; it is not a capability coordinate. For a fixed evaluation configuration, a vector-valued study may instead retain the conditional set

\[ \mathcal F_{\mathfrak e}(C) =\operatorname {Pareto}\left \{ \bigl (\mathbf K(A,O,S,\mathfrak e,\eta ,C), J_{\rm ev}(P_{A,\eta }),\mathbf c(P_{A,\eta })\bigr ): c_A(\eta )\leq C \right \}, \]

where Pareto dominance is declared for the reported coordinates. The set is conditional on the task, intervention class, evaluation law, cost vector, and missingness rule; it is not a universal intelligence frontier.

The semantic trajectory diagnostic in semantic representation diagnostics supplies a conditional representation record for this vector; its held-out probe score does not fill a function or deployment coordinate.

Matched-compute arms in § [ftip-00JU] compare entries only after stating which coordinates are held fixed, which are allowed to vary, and how the cost vector is charged. Cost, numerical stability, and monitorability remain evaluation coordinates alongside the vector, so a lower scalar cost or a stable implementation is not silently counted as a capability increase.

Depth reuse and latent recurrence [ftip-00KG]

Repeated depth, latent feedback, and recirculation change the computation performed at inference. Relative to the fixed reference autoregressive Transformer, their costs include any extra passes, state, and prefill work.

Definition. Computation-axis record [ftip-00KH]

For an architecture record $A$, distinguish token steps, feed-forward depth passes, recurrent passes, and latent-state updates. Let $d(A,r)$ be the declared number of depth passes in run $r$; it is a computation coordinate, not a synonym for parameter count or reasoning quality.

Remark. Looped Transformers as a source result [ftip-00KI]

Giannou et al. show that a fixed shallow Transformer placed in a loop can execute programmed iterative computations, including conditional branches and in-context algorithms, in Looped Transformers as Programmable Computers[giannou2023looped]. This is a source result under its program, state, and precision assumptions; it is not a universal claim about language-model capability.

Remark. Parameter sharing and width tradeoffs [ftip-00KJ]

Xue et al. report that depth sharing can reduce trainable parameters while limiting modeling capacity, and study width and mixture-of-experts remedies in Go Wider Instead of Deeper[xue2021wider]. The comparison is empirical and task-specific; it does not identify a depth-independent ceiling.

Remark. Virtual logical depth and scaling [ftip-00KK]

Zhu et al. vary virtual logical depth by reusing weights and report reasoning gains at nearly fixed parameter count in Beyond Parameters: Exploring Virtual Logic Depth for Scaling Laws[zhu2025virtualdepth]. Their results motivate a depth coordinate in a scaling study, while leaving knowledge capacity, optimization, and transfer dependent on the declared training and evaluation protocol.

Remark. Recirculation as an inference intervention [ftip-00KL]

Mozer et al. introduce inference-time recirculation that feeds latent states back through an off-the-shelf model and report task improvements in Recirculation[mozer2026recirculation]. The added serial prefill and adaptive tuning belong in the systems and inference records; the observation is not a proof of a new representational ceiling.

Remark. Latent feedback with a preserved Transformer interface [ftip-00KM]

Wang et al. widen the feedback channel between decoding steps with latent feedback while retaining a Transformer and language-modeling interface in Full-bandwidth transformer[wang2026fullbandwidth]. Any reported token or accuracy savings are conditional on the scheduled multi-pass training and measured decoding cost.

Remark. Latent prediction as a training intervention [ftip-00KN]

Teoh et al. add next-latent prediction to next-token training and report compact predictive states without changing the Transformer interface in Next-Latent Prediction Transformers Learn Compact World Models[teoh2025nextlatent]. This belongs to the training intervention record, not to an architecture-only comparison.

Example. Equal parameters do not fix effective depth [ftip-00KO]

Two runs can share parameter count while using different numbers of recurrent or latent-feedback passes. A matched study must therefore report the computation-axis record and cannot infer an architecture ceiling from parameters alone.

Definition. Depth-reuse model-instance record [ftip-00KP]

For a depth-reuse comparison, record shared weights, pass count, stopping rule, recurrent state, training objective, token and pass FLOPs, prefill and decode latency, memory, and evaluation seeds. A pass-count change is an intervention even when the parameter tensor is unchanged.

Example. Fixed-recipe depth comparison [ftip-00KQ]

In a fixed-recipe arm, hold data, optimizer, feedback, stopping rule, and evaluation law fixed while varying only the declared depth-reuse intervention. The resulting difference is an estimand for that recipe, not a best-achievable comparison.

Remark. Scaling surfaces rather than a single law [ftip-00KR]

A depth-reuse study may fit a surface over parameters, tokens, passes, and cost. The fit is an empirical summary over its measured range; it does not establish an asymptotic law or a universal saturation point.

Remark. Architecture and training are separable records [ftip-00KS]

Weight sharing, latent objectives, and recirculation can alter optimization without changing the declared external interface. The comparison must retain architecture, training intervention, and systems coordinates separately.

Remark. Depth reuse across passes, parameters, and tasks [ftip-00KT]

The cited looped, virtual-depth, latent-feedback, and recirculation results do not show that more passes always improve capability, that parameter sharing dominates added parameters, or that inference-time gains transfer to training or to another task family. Each transfer requires a matched model-instance record.

Ceilings, frontiers, and saturation [ftip-00JN]

Definition. Fixed-recipe and tuned frontiers [ftip-00JO]

The fixed-recipe frontier evaluates one common $\eta $. The equal-tuning frontier permits architecture-specific choices from a predeclared trial set with the same tuning data and budget. The restricted envelope permits a predeclared class of interventions under $C$. These are distinct estimands.

Definition. Iso-quality cost ratio [ftip-00JP]

For a finite real target $q\in \mathbb R$ and the frontier of Definition [ftip-00JJ], define the inverse cost \[ C_A(q)=\inf \{C\in \mathbb R_{\geq 0}:V_A(C)\geq q\} \in [0,+\infty ], \qquad \inf \varnothing =+\infty . \] The iso-quality ratio is defined only on the domain \[ \rho _{A/B}(q)=\frac {C_A(q)}{C_B(q)}, \qquad 0\leq C_A(q)<+\infty , \quad 0<C_B(q)<+\infty . \] The architectures must use the same target, evaluation interface, cost units, and comparison arm. The ratio compares efficiency under these choices.

An inverse cost is a threshold infimum, not an executable minimum. The infimum over budgets may be unattained; even if a budget satisfies $V_A(C)\geq q$, its performance supremum may be unattained at $q$. An actual target-achieving intervention requires a separate witness.

Positive individual costs do not guarantee a positive inverse cost. For interventions $\eta _n$, $n\geq 1$, with score $1$ and cost $1/n$, the target $q=1$ has $C_A(1)=0$, although every intervention costs more than zero. If both architectures have this family, the putative ratio is $0/0$ and is excluded by the displayed domain.

Definition. Operational cost vector and scalarization [ftip-00JQ]

Record training FLOPs, inference FLOPs, wall time, memory, energy, and hardware separately as $\mathbf c$. A scalar cost $C=w\cdot \mathbf c$ is a declared study choice with nonnegative units $w$; changing $w$ changes the frontier and must not be hidden as architecture quality.

Theorem. Finite-window representational obstruction [ftip-00JR]

Consider a causal system whose state after each prefix has at most $K$ distinct values, and a task with $K+1$ prefixes requiring pairwise distinct continuation labels. By the pigeonhole principle two prefixes share a state, so at least one continuation label is wrong. This is a finite toy obstruction only; it does not bound a concrete Transformer or KDA instance without a proved reduction to this state model.

Theorem. A conditional saturation certificate [ftip-00JS]

Let $V_A$ be the nondecreasing frontier of Definition [ftip-00JJ], bounded above by a finite $U\in \mathbb R$. If an intervention reaches a real target $q<U$ at a finite cost $C_q\geq 0$, then at every finite $C\geq C_q$, $q\leq V_A(C)\leq U$. In particular $V_A(C)$ is real and $0\leq U-V_A(C)\leq U-q$. This elementary certificate is conditional on the bound $U$; it does not assert that real intelligence saturates.

Example. Equal scalar cost does not identify architecture [ftip-00JT]

Two systems can have the same scalar $C$ while differing in memory, latency, and training allocation. A cost-weight change can reverse their ordering without changing either system. Thus a one-factor comparison needs a fixed scalarization and a reported cost vector; no architecture effect follows from equal $C$ alone.

Matched-compute comparison protocol [ftip-00JU]

Definition. Common-recipe comparison arm [ftip-00JV]

A common-recipe arm fixes training data, optimizer family, schedule, post-training procedure and feedback, training and post-training budgets, inference budget, and evaluation protocol and draws. Interface-compatible parameter choices are declared in advance as part of each permitted architecture package. The comparison varies that package under the common recipe; any further deviation is recorded as a separate comparison coordinate. Its score difference measures performance under this recipe, not the best attainable result for either architecture.

Definition. Equal-tuning-budget comparison arm [ftip-00JW]

An equal-tuning-budget arm permits architecture-specific tuning under a predeclared protocol that fixes the tuning data and its volume, trial count, selection rule, stopping rule, and tuning cost for both architectures. The cost uses the same declared accounting rule. The resulting comparison measures practical performance under this equal optimization effort; it does not determine best possible training or a representation-only limit.

Definition. Restricted-envelope comparison arm [ftip-00JX]

A restricted-envelope arm declares an allowed intervention class $\mathcal E_A\subseteq \mathfrak I_A$ for each architecture, specifying the permitted training, post-training, and inference procedures. The classes expose exclusions, search budgets, seeds, and stopping rules. Restrict the evaluation functional of Definition [ftip-00JJ] to $\mathcal E_A$ and compare the resulting suprema under the same finite cost cap $C\geq 0$, common evaluation law, and common declared cost accounting.

The measurable, integrable evaluation domain and extended-real conventions of Definition [ftip-00JJ] apply: an empty feasible class has value $-\infty $, an unbounded-above feasible score set has value $+\infty $, and a finite supremum need not be attained. The allowed class is part of the comparison, so its envelope is a restriction-dependent quantity, not an unrestricted architectural ceiling.

Definition. Model-instance comparison evidence record [ftip-00JY]

A model-instance record contains architecture/checkpoint identity, parameter count, data and post-training recipe, optimizer, precision, hardware, cache policy, training and inference cost vectors, task suite, and independent evaluation seed. A missing field makes an architecture attribution unknown.

Remark. Systems optimizations are measured interventions [ftip-00JZ]

FlashAttention, KV-cache layouts, quantization, batching, and kernels can change feasible cost. If they approximate, truncate, or alter precision of the mathematical computation, the record must mark that as a systems intervention rather than as a pure architecture comparison.

Theorem. Matched-protocol difference is an estimand [ftip-00K0]

Given $n\geq 1$ evaluation draws, let $\widehat V_A(C)$ be the sample mean of finite real evaluation scores for architecture $A$ at a finite cost cap $C\geq 0$. Under a fixed evaluation law and common-recipe arm, the finite difference $\widehat V_A(C)-\widehat V_B(C)$ is a well-defined empirical estimand.

It does not identify a causal architecture effect when data, tuning, or systems coordinates differ. The conclusion follows directly from the declared record.

Remark. Conditions for applying a toy model [ftip-00K1]

A toy theorem applies to a concrete model only if an explicit map from its state and interface to the model preserves the theorem's assumptions, including its cost accounting. Without such a map, the conclusion concerns only the abstract model; its validity for the concrete system is unknown.

Kimi Delta Attention versus Transformer [ftip-00K2]

Definition. KDA model-instance record [ftip-00K3]

For a Kimi Delta Attention (KDA) versus full-attention baseline study, record the exact Kimi Linear checkpoint, layer mix, context length, hardware, kernel, precision, batch, and decoding workload. The paper describes KDA as a fine-grained gated delta-rule module in a hybrid architecture; those are source observations from Kimi Linear: An Expressive, Efficient Attention Architecture[kimi2025linear], not universal theorems.

If the baseline is called a Transformer, record whether it is the paper's MLA baseline or another full-attention implementation; the label alone does not identify a common architecture or cost.

Example. Matched-cost KDA and Transformer comparison [ftip-00K4]

A matched comparison uses the same task prompts, post-training data and feedback, optimizer family, evaluation protocol, and quality target. The corresponding measurements include parameter count, training FLOPs, inference FLOPs, wall time, memory, KV-cache bytes, and context length for each architecture. A reported speed or quality difference is conditional on these experimental conditions and measurements.

Remark. What the Kimi Linear paper establishes [ftip-00K5]

Kimi Linear reports a hybrid KDA/MLA model and fair-comparison experiments in Sections 3--5, with setup in Section 5.4 and efficiency comparisons in Sections 5.5--5.6 of Kimi Linear: An Expressive, Efficient Attention Architecture[kimi2025linear]. These include long-context efficiency.

The reported throughput and quality are empirical under its training recipe and hardware; they are lower bounds on demonstrated performance, not an architecture-independent ceiling.

Example. Parameter matching is not cost matching [ftip-00K6]

Two models with equal parameter count can have different attention FLOPs, KV-cache memory, kernel utilization, and attainable context. Conversely, equal wall time can hide different hardware and precision. Therefore a parameter-matched result cannot by itself identify a compute frontier.

Remark. Cache, throughput, and quality in KDA comparisons [ftip-00K7]

The KDA example does not prove that every linear-attention model beats every Transformer, that cache savings imply equal quality, or that a measured throughput gain is a capability gain. Any such statement requires a new matched study with the evidence record of Definition [ftip-00JY].

Definition. Model-instance comparison decision rule [ftip-00K8]

Call architecture A more cost-efficient than B at quality $q$ only when the compared threshold costs use the same declared protocol, the confidence procedure and stopping rule are fixed, and all cost-vector coordinates needed by the claim are present. A ratio additionally requires finite costs and a strictly positive denominator as in Definition [ftip-00JP]. State whether the costs are measured target-achieving witnesses or justified inverse-frontier values $C_A(q)$ and $C_B(q)$; observing one successful run does not identify those infima. Otherwise the comparison is descriptive or unknown.

Interface boundaries for alternative architectures [ftip-00K9]

Definition. JEPA comparability interface [ftip-00KA]

A JEPA-like architecture is comparable to an autoregressive model only after fixing the predictor, target representation, decoder or downstream interface, training objective, inference procedure, and evaluation law. A parameter-count match alone is not a typed comparison.

Remark. Representation and task interface are separate [ftip-00KB]

A latent predictor can be excellent under one decoder and unusable under another. The evaluation record must therefore include the interface that maps the representation to the declared task output.

Theorem. Interface-preserving architecture reduction [ftip-00KC]

If two architecture records induce the same conditional output law on a fixed task and evaluation interface, every evaluation functional $J$ gives the same value. This is an immediate pushforward identity under the stated output-law and interface assumptions. It does not imply that distinct architectures are generally equivalent.

Example. Equal benchmark score can hide different ceilings [ftip-00KD]

Two architectures may tie on a finite benchmark while differing on an unmeasured task slice or longer context. Thus a single score cannot establish equal representational ceilings; the task family and evaluation coverage must be declared.

Remark. Architecture-independent model interfaces [ftip-00KE]

A result stated solely in terms of a model/interface map applies to every architecture realizing that map and satisfying its hypotheses. An architecture-specific conclusion can require further assumptions, as in Remark [ftip-00JK].

Remark. Open model-instance questions [ftip-00KF]

The relative capability and cost of KDA, full attention, and JEPA candidates remain empirical questions. A reproducible comparison requires identified checkpoints or reproducible training, fixed evaluation seeds, and measured cost vectors. Architecture-specific ceilings remain estimands and hypotheses rather than established constants.

Published hybrid-attention comparison [ftip-00KU]

Kimi Linear combines recurrent KDA layers with periodic full MLA attention. Its published comparison shows task-dependent score changes and lower long-context decoding times. The selected scores in Example [ftip-00LC], the memory estimate in Example [ftip-00KY], and the discovery calculation in Example [ftip-00L3] connect these observations to a post-training question: when can a cheaper attempt compensate for a possible lower per-attempt success probability?

Example. Published Kimi Linear operating point [ftip-00KV]

The Kimi Linear report compares experimental models pretrained on 1.4 trillion tokens, each with 48 billion total and 3 billion active parameters. Kimi Linear uses three Kimi Delta Attention layers for each full Multi-head Latent Attention layer, with no positional encoding (NoPE) in its MLA layers; the reference uses full MLA attention [kimi2025linear, Sections 4 and 5.4]. These are properties of the reported hybrid architecture.

These experimental models are distinct from the released Kimi Linear model, pretrained on 5.7 trillion tokens and supporting up to one million tokens of context. Appendix D compares that release with Moonlight, which has 16 billion total and 3 billion active parameters [kimi2025linear, Section 5.4.1 and Appendix D].

Example. Published efficiency observations [ftip-00KW]

For batch size one at one million tokens, Figure 7 labels a $2.9\times $ prefill speedup and a $2.2\times $ decoding speedup for Kimi Linear against MLA. Section 6.3 gives $2.3\times $ for the latter point, so the graphic and prose differ slightly [kimi2025linear, Figure 7 and Sections 5.6 and 6.3].

The larger-batch comparison in Figure 1(b) gives decoding time per output token of 1.84 milliseconds for Kimi Linear and 11.48 milliseconds for MLA, with a reported $6.3\times $ speedup. Section 6.3 describes this as a theoretical speedup from reallocating saved KV-cache memory to larger batches. It illustrates a different use of the memory saving from the batch-one latency result [kimi2025linear, Figure 1(b) and Section 6.3].

Figure 1(a) reports RULER scores of 84.3 for Kimi Linear and 81.3 for MLA at 128k context, with $3.98\times $ decoding acceleration [kimi2025linear, Figure 1(a)]. The report also gives a reduction of up to 75 percent in KV-cache use [kimi2025linear, Abstract]. Together these observations motivate studying context length and batch size as separate cost coordinates. The unequal RULER scores and different timing regimes are retained when interpreting those points.

Definition. Quality target and cost measurement [ftip-00KX]

For a reported benchmark score q, use the same task, prompt law, scoring rule, and post-training condition for both systems. The comparison records the smallest measured cost at which each system reaches q, when such a measurement exists. A throughput or cache ratio is not substituted for this iso-quality cost.

Example. A cache-memory estimate for hybrid attention [ftip-00KY]

The cited FLA KDA implementation at commit 6b6f546f stores one active recurrent state per sequence, with dimensions determined by the value heads and key/value dimensions rather than accumulated context length. Its newly allocated state uses float32 [⧉]. The optional short-convolution state has one fixed-width kernel buffer per channel and sequence [⧉]. This supplies a concrete implementation model for the recurrent part of the 3:1 mixture in Example [ftip-00KV].

Consider a stylized comparison with $4\ell $ layers, batch size $b$, and context length $T$, where $\ell ,b,T$ are positive integers. Assume each MLA layer stores $\kappa T$ bytes per sequence, with the same $\kappa >0$ in both designs, and each KDA layer stores one terminal recurrent-plus-convolution state of $\sigma >0$ bytes. The active attention-state memories are

\[ M_{\rm MLA}=4b\ell \kappa T,\qquad M_{\rm hybrid}=b\ell (\kappa T+3\sigma ),\qquad \frac {M_{\rm hybrid}}{M_{\rm MLA}} =\frac {1}{4}+\frac {3\sigma }{4\kappa T}. \]

The hybrid uses less active state exactly when $T>\frac {\sigma }{\kappa }$, and the ratio tends to $1/4$ as context grows. This explains why replacing three quarters of the growing caches can approach a 75-percent saving, while fixed recurrent-state overhead can erase the advantage at short contexts. The retained MLA layers keep the hybrid's state memory linear in $T$; the whole model is not a fixed-memory recurrent system.

Weights, training activations, workspaces, allocator padding and saved prefix states are outside this active-state calculation. At a fixed device memory budget, the estimate predicts more room for simultaneous or longer rollouts when attention state dominates. It does not predict a fourfold speedup. Training and decoding also use different execution paths: FLA selects chunk kernels for gradient-enabled computation and fused recurrence for short inference calls [⧉]. The fixed-state estimate is therefore a useful serving hypothesis, not a training-memory or training-speed formula.

Example. Common-recipe reading of the Kimi result [ftip-00KZ]

The report holds its stated pretraining and SFT recipes common across the experimental models [kimi2025linear, Sections 5.4--5.5 and Table 4]. The selected observations in Example [ftip-00LC] compare the resulting trained designs, including the hybrid's 3:1 KDA/MLA pattern and NoPE choice. The mixed score directions suggest a task-dependent tradeoff rather than one ordering of the two architectures.

The theoretical mechanism is a change in the cost of maintaining and using the prefix: recurrent layers compress it into state, while the retained MLA layers preserve growing attention caches. The model in Example [ftip-00KY] predicts the largest memory advantage at long contexts. If this advantage reduces the cost of generating useful candidates, Example [ftip-00L3] predicts when more attempts can offset a reduction in per-attempt quality. This is a conditional explanation to test through rollout costs and verified success rates; it does not require attributing the whole result to KDA alone.

Example. Measured iso-quality cost ratio [ftip-00L0]

Let $c_K(q)$ and $c_T(q)$ be measured inference costs for Kimi Linear and the Transformer reference at the same finite real score $q$ and context length, under one accounting rule. If $0\leq c_K(q)<+\infty $ and $0<c_T(q)<+\infty $, the measured ratio $\widehat \rho _{K/T}(q)=c_K(q)/c_T(q)$ is reported together with the full cost vectors. These measured costs need not be the inverse frontier infima of Definition [ftip-00JP]. If either threshold is unmeasured or the denominator is zero, the ratio is left undefined rather than inferred from the paper's speedup plot.

Remark. Hybrid attention is not pure linear attention [ftip-00L1]

Kimi Linear combines KDA with periodic full-attention layers. A measured advantage therefore identifies the reported hybrid intervention, not a claim that a purely linear recurrent architecture has the same quality or ceiling. The layer pattern belongs in the architecture record.

Remark. Demonstrated performance is a lower bound [ftip-00L2]

The published benchmark scores demonstrate attainable points for the reported checkpoints and protocol. They do not establish a representational ceiling, a universal scaling law, or a claim that additional cost cannot improve either architecture.

Example. When cheaper attempts compensate for lower success [ftip-00L3]

Smaller caches can support more concurrent requests or avoid preemption and recomputation. These are actual serving mechanisms in vLLM's memory and scheduling guidance [⧉]. For post-training, the useful prediction is therefore about completed candidate attempts under a budget, including verification and optimization costs, rather than decoding speed alone.

Use the following cost model on the same hardware. One baseline attempt costs $c>0$; generation accounts for a fraction $f\in [0,1]$. Suppose generation becomes $s_{\rm gen}>0$ times as fast while all other cost per attempt stays fixed. The new cost and total attempt-throughput factor are

\[ c_K=c\left (1-f+\frac {f}{s_{\rm gen}}\right ),\qquad S=\frac {c}{c_K} =\frac {1}{1-f+f/s_{\rm gen}}. \]

For a fixed budget $B\geq 0$, this model permits $N_M=\lfloor B/c\rfloor $ baseline attempts and $N_K=\lfloor B/c_K\rfloor $ hybrid attempts. Assume $N_M,N_K\geq 1$ and verified-success events independent within each set of attempts, with respective probabilities $p_M,p_K\in [0,1]$. Then Theorem [ftip-0078] gives discovery probabilities $1-(1-p_M)^{N_M}$ and $1-(1-p_K)^{N_K}$. The hybrid matches or exceeds baseline discovery exactly when

\[ p_K\geq 1-(1-p_M)^{N_M/N_K}. \]

This follows by comparing the two failure probabilities and taking the nonnegative $N_K$-th root.

For an illustrative estimate, suppose generation consumes 80 percent of the baseline cost and use $s_{\rm gen}=2.2$, motivated by the batch-one decode figure in Example [ftip-00KW]. Applying that decode factor to the whole generation stage is a working approximation, not a measured rollout result. Then $S\approx 1.774$; a budget of $B=10c$ buys 10 baseline attempts or 17 hybrid attempts. If $p_M=0.10$ and $p_K=0.08$, the discovery probabilities are approximately $0.651$ and $0.758$. The break-even hybrid probability is about $0.0601$. Thus this model predicts that more attempts can outweigh a moderate per-attempt quality loss. The assumed probabilities are illustrative verified-success rates, not conversions of the benchmark scores in Example [ftip-00LC].

The prediction is strongest for long-context, generation-heavy work whose saved memory becomes usable rollout capacity. As verification or optimization dominates, $f$ shrinks and $S$ approaches one. Correlated attempts require the joint-law analysis of Convention [ftip-0077] instead of the independent product. Measure completed attempts, verified-success rates and total cost under a fixed prompt, decoding and verifier protocol to test the tradeoff. In the RLVR round of Example [ftip-004M], additional successful candidates can improve the available feedback; whether they produce useful accepted updates is a further optimizer-and-evaluation question, as Example [ftip-0085] demonstrates.

Paired measurements for model comparison [ftip-00L4]

A paired measurement specifies both model instances and their task, intervention, evaluation, and cost coordinates. These conditions determine the meaning of each point in the measured frontier.

Definition. Comparison pair identity [ftip-00L5]

A paired comparison identifies the two systems, exact checkpoint or training commit, architecture variant, parameter counts, tokenizer, and context limit. A label such as ``Transformer'' is insufficient when attention, cache, or layer patterns differ.

Definition. Common task and evaluation conditions [ftip-00L6]

Both systems use the same task family, prompt and data law, scoring rule, sampling seeds, stopping rule, and evaluator version. Any exception is a separate comparison arm, not an unrecorded architecture effect.

Definition. Training and post-training coordinates [ftip-00L7]

Record data volume, token order policy, optimizer and schedule, supervised tuning, feedback or reinforcement procedure, update count, and selection rule. The common-recipe arm fixes these coordinates; the equal-tuning arm allows predeclared alternatives with equal search resources.

Definition. Inference and systems coordinates [ftip-00L8]

Record hardware, software and kernel versions, precision, batch, cache policy, sequence lengths, warm-up procedure, concurrency, and decoding settings. A kernel or cache change that alters numerical computation is marked as an intervention rather than hidden as an implementation detail.

Definition. Per-system cost vector [ftip-00L9]

For each system, the cost vector $\mathbf c$ contains training FLOPs, inference FLOPs, wall time, memory, energy when available, and context length. The primary scalar $C$ and its weights are declared beside the vector; omitted coordinates are marked unknown.

Definition. Repeated measurement rule [ftip-00LA]

For each fixed workload and system, record warm-up runs, repetition count, random seeds, and the aggregation rule. Report mean and dispersion for time, memory, and score; a single fastest run cannot define a cost frontier.

Definition. Paired uncertainty estimates [ftip-00LB]

When the same evaluation items are used, report paired score differences and an uncertainty procedure fixed before inspecting the result. Confidence intervals quantify sampling variation; they do not repair unmatched training, systems, or task coordinates.

Example. Published Kimi Linear comparison [ftip-00LC]

Table 4 of the Kimi Linear report compares the experimental models pretrained on 1.4 trillion tokens in Example [ftip-00KV], after the same supervised fine-tuning (SFT) recipe. Both have 48 billion total and 3 billion active parameters. Three selected benchmark scores are [kimi2025linear, Sections 4 and 5.4 and Table 4]:

Arm	MMLU-Pro	LiveBench	EvalPlus
Kimi Linear	67.4	45.2	61.0
Full MLA	65.7	45.7	62.6

The entries use the paper's score units, with larger values better on each selected benchmark; LiveBench is reported as Pass@1. Kimi Linear is higher on MMLU-Pro and lower on LiveBench and EvalPlus in these observations. These are reported point estimates on different tasks, not a single capability score; the table supplies no uncertainty estimate.

The common recipe uses the K2 pretraining corpus, MuonClip and a shared training schedule. SFT proceeds from broad instruction data to targeted reasoning tasks; evaluation uses temperature 1.0 and an internal framework derived from LM-Harness [kimi2025linear, Section 5.4]. Thus the scores provide a concrete example of how the trained hybrid and full-MLA designs respond to the same recipe. The cost mechanism and its possible post-training consequences are developed in Example [ftip-00KZ].

Remark. Measured and assumed costs [ftip-00LD]

A useful cost model can combine measurements with assumptions justified by the architecture and its implementation. In Example [ftip-00L3], the generation share $f$ and speed factor $s_{\rm gen}$ determine a predicted attempt-throughput factor. Their assumed values can be varied without pretending they were measured in the original experiment.

For fixed $f\in [0,1]$, write $S(s)=1/(1-f+f/s)$ for $s>0$. This function is nondecreasing. An interval $s_{\rm gen}\in [s_-,s_+]$, with $0<s_-\leq s_+$, therefore gives the conditional range $S\in [S(s_-),S(s_+)]$. This is sensitivity to a chosen assumption range; it is a statistical confidence interval only if a sampling argument supplies that interpretation.

Definition. Measured contrasts and conditional predictions [ftip-00LE]

A measured paired contrast evaluates the same declared quantity for two observed model instances under a specified task, protocol and cost accounting. The common-recipe scores in Example [ftip-00LC] are one example. A confirmatory comparison fixes its matching and selection rules before examining the scores.

A conditional paired prediction instead uses an explicit model to supply one or more cost or response quantities, then derives the comparison under those assumptions. The discovery probabilities in Example [ftip-00L3] are an example. Such a prediction can be motivated by existing observations and guide a later experiment; its unmeasured inputs remain assumptions, and agreement with new matched measurements tests the model.

Example. How generation share changes the predicted gain [ftip-00LF]

Keep the illustrative generation factor $s_{\rm gen}=2.2$ from Example [ftip-00L3] and vary the baseline generation share $f$. Its cost model gives the following total attempt-throughput factors:

Generation share	Predicted throughput factor
20 percent	1.122
50 percent	1.375
80 percent	1.774
100 percent	2.200

These calculated values show why the same decoding improvement can matter much more for generation-heavy search than for a verifier- or optimizer-dominated workload. They suggest measuring the time spent in generation, verification and updates before selecting where to invest systems effort. A change in the bottleneck changes the useful intervention, even when the attention architecture stays fixed.

Remark. Testing the predicted architecture tradeoff [ftip-00LG]

The memory and discovery models predict the strongest benefit when long prefixes make attention state expensive, saved capacity improves useful rollout throughput, and per-attempt success remains above the break-even threshold. Short contexts, expensive verification, or strongly correlated attempts can weaken that benefit. These are distinct mechanisms to test, rather than a universal ranking of KDA and full attention.

A comparison across implementations can retain this reasoning while re-estimating state sizes, generation share and success probabilities for the new system. Report both the prediction and the matched measurements: their agreement or disagreement identifies which mechanism or assumption needs revision.

Conditional frontier and saturation analysis [ftip-00LH]

A finite matched study measures capability at selected values of one declared cost. Its frontier and saturation target are conditional on that protocol; they do not establish a universal intelligence law.

Definition. Finite cost grid [ftip-00LI]

Choose a finite ordered grid $0<C_1<\cdots <C_m$ and evaluate each architecture at every declared grid point under the same task and protocol. The resulting values $\widehat V_A(C_i)$ are observations of the restricted frontier, not its values between grid points.

Definition. Observed upper envelope [ftip-00LJ]

For an observed score $s_A(C_i)$ define the discrete upper envelope $U_A(C_i)=\max _{j\leq i}s_A(C_j)$. It is a descriptive monotone summary of the measured points; it is not evidence that unmeasured costs attain the envelope or that extra cost cannot reduce score.

Definition. Uncertainty bands on the frontier [ftip-00LK]

Attach a predeclared uncertainty interval $I_A(C_i)$ to each score and carry those intervals through score differences and threshold crossings. A finite band describes sampling and measurement variation; it does not cover unmeasured training procedures or architectures.

Definition. Declared saturation target [ftip-00LL]

Fix a quality target $q\in \mathbb R$, tolerance and cost increment $\varepsilon ,\delta \in (0,+\infty )$, and a budget $C\in \mathbb R_{\geq 0}$. Require both $V_A(C)$ and $V_A(C+\delta )$ to be finite real values. A study calls an architecture $\varepsilon $-saturated at $C$ only relative to its allowed intervention class when the restricted frontier has certified gain $V_A(C+\delta )-V_A(C)\leq \varepsilon $.

Theorem. Conditional saturation certificate [ftip-00LM]

Take finite real $U$, $C\geq 0$, $\delta \geq 0$, and $\varepsilon \geq 0$, with the nondecreasing frontier of Definition [ftip-00JJ]. If a justified upper bound gives $V_A(C+\delta )\leq U$ and a justified lower bound gives $V_A(C)\geq U-\varepsilon $, then $U-\varepsilon \leq V_A(C)\leq V_A(C+\delta )\leq U$. Both endpoint values are therefore finite real, and $0\leq V_A(C+\delta )-V_A(C)\leq \varepsilon $. This is a conditional bound for the named frontier; it proves no universal saturation of intelligence.

For the finite controller class with hard resource admission, Corollary [ftip-00MC] obtains the required bounds from a feasible controller and a uniform Bellman certificate.

Example. Finite observations need not identify a frontier [ftip-00LN]

Let $S\subset \mathbb R_{\geq 0}$ be a finite set of measured costs, and choose $C_0>0$ larger than every element of $S$. Suppose the exact frontier value observed at each cost in $S$ is zero. For finite budgets $C\geq 0$, suppose the declared class of possible frontiers permits both \[ V_0(C)=0, \qquad V_1(C)= \begin {cases} 0,&0\leq C<C_0,\\ 1,&C\geq C_0. \end {cases} \] These nondecreasing frontiers with scores in $[0,1]$ agree on every observed cost and differ at $C_0$. The observations alone do not distinguish them.

Both possibilities have finite realizations in the framework of Definition [ftip-00JJ]: allow two interventions with costs $0$ and $C_0$. Give the first score zero and the second score $\theta \in \{0,1\}$. For example, on a single deterministic evaluation task with utility equal to the output bit, let the two resulting protocols return $0$ and $\theta $. The two possible choices of $\theta $ give $V_0$ and $V_1$, respectively, under the same intervention and cost specification. Every score is finite, and the feasible score maximum is attained at each budget.

This example does not supply a strictly better agreeing frontier for every possible data set or every admissible class. If a proved global score bound is $1$ and an intervention attains it at a finite cost $C_*\geq 0$, monotonicity forces $V_A(C)=1$ for every $C\geq C_*$; no higher value is admissible. A singleton class of possible frontiers can also identify the frontier without such an alternative. A conditional certificate such as Theorem [ftip-00LM] therefore requires its stated upper-bound evidence; that evidence does not follow merely from the absence of an observed improvement.

Definition. One-factor cost interpretation [ftip-00LO]

The primary cost $C$ is a declared scalarization of the recorded cost vector. A frontier statement is conditional on its units and weights; the same paired measurements may produce a different ordering under a different scalarization.

Example. Crossing architecture frontiers [ftip-00LP]

Two architectures may alternate in the observed ordering across costs: one can score higher at small $C$ while the other catches up at larger $C$. Thus a single comparison point cannot establish a global ordering or a common ceiling.

Example. Frontier report for the KDA study [ftip-00LQ]

A Kimi Delta Attention versus Transformer study should publish the cost grid, paired scores, uncertainty intervals, scalarization weights, and the exact identities and configurations of the compared models. The report may then state which measured points are Pareto or iso-quality comparisons under that protocol.

Remark. Frontiers depend on the intervention and evaluation [ftip-00LR]

The conditional frontier is indexed by architecture, intervention class, task family, evaluation law, and cost scalarization. Changing any of these coordinates creates a new estimand; no ordering transfers automatically to a new checkpoint, optimizer, hardware stack, or task family.

Study design for comparisons across model instances [ftip-00LS]

This subsection specifies how a conditional architecture comparison may be instantiated without changing the architecture-neutral formulation. It is a study design, not a universal ranking of architectures.

Definition. Matched model-instance record [ftip-00LT]

A model-instance record names the architecture, parameterization or checkpoint, training and post-training recipe, optimizer, execution stack, inference budget, task family, evaluation law, and recorded cost vector. Two records are comparable only after the fields held fixed and the fields allowed to vary are declared.

Definition. Pair identity and comparison unit [ftip-00LU]

A comparison unit is a pair of model-instance records evaluated on the same task family and evaluation law, together with a declared cost scalarization. A Transformer reference and a Kimi Delta Attention instance may form such a pair only when the remaining coordinates are documented.

Remark. Paired responses to a common recipe [ftip-00LV]

For a pair of matched model-instance records, the common-recipe arm measures how the permitted architecture packages respond to one recipe. A score advantage in this comparison need not persist after either package is retuned.

Remark. Different recipes under equal tuning effort [ftip-00LW]

Under the equal-tuning-budget arm, the two selected model instances may use different recipes. That difference is compatible with equal predeclared tuning effort; forcing the selected recipes to match would answer a different comparison question.

Remark. How an allowed class changes its envelope [ftip-00LX]

In the restricted-envelope arm, enlarging an allowed intervention class at fixed evaluation law, cost accounting, and budget can raise its envelope and cannot lower it. A finite search over feasible interventions supplies a lower bound from the scores it attains. It certifies the supremum only if it covers every feasible intervention or is accompanied by a matching upper-bound argument.

Remark. Coordinate-change rule [ftip-00LY]

Changing the optimizer, post-training method, hardware or kernel, precision, context policy, or evaluation law changes a comparison coordinate. A score difference after such a change belongs to the new intervention, unless the study explicitly estimates the interaction.

Definition. Architecture-family transfer matrix [ftip-00LZ]

A transfer matrix records which interfaces, intervention arms, and budgets permit a comparison. Each row is a study record: its source, estimand, matched controls, evaluation coordinates, and declared scope boundary are explicit. A blank or incompatible cell is an incomparability finding, not a missing score to be imputed. The vector and evaluation coordinates are reported together under the conditional frontier of Definition [ftip-00JM].

Comparison cell	Source	Estimand	Matched controls	Cost / stability / monitorability	Status and scope boundary
Transformer--KDA/full attention	Remark [ftip-00KF], Definition [ftip-00LU], Example [ftip-00LQ]	Difference in utility, cost, and $K_{\rm state}$ between KDA and full attention	Same task, model scale, recipe, context, seeds, and evaluation law	Report measured cost vector, training/inference stability, and state/failure observability	Open; a finite KDA--Transformer result does not establish an architecture-wide ceiling
Transformer--Mamba	Mamba, Sections 3.1--3.3; § [ftip-00OQ]	Difference in retrieval utility and $K_{\rm state}$ at the declared budget	Task, scale, recipe, context, and inference budget	Report cost vector, recurrence stability, and state/failure observability	Open; selective compression does not transfer a retrieval result across tasks
Transformer--linear attention	Linear attention, Sections 3.2--3.3; § [ftip-00OQ]	Difference in retrieval utility and $K_{\rm state}$ for associative summaries	Task family, feature map, context budget, and evaluation law	Report prefix cost, numerical stability, and summary-state observability	Open; associative summaries do not imply full-attention equivalence
Transformer--CTM or looped model	Continuous Thought Machine, Sections 3.1--3.5; § [ftip-00KG]; § [ftip-00OQ]	Difference in utility and $K_{\rm adapt}$ per charged tick or loop	Input, base model, stopping rule, recipe, and charged internal compute	Report tick/loop cost, training and inference stability, and stopping-signal monitorability	Open; extra passes are charged and do not imply continuous-time behavior
Transformer--BDH	BDH, Section 3.2; § [ftip-00OQ]	Difference in task utility and $K_{\rm state}$ under the reduced state interface	Task, state parameterization, credit horizon, systems stack, and cost	Report memory and compute cost, optimization stability, and state observability	Open; a reduced state-space proposal is not a production transfer result
Transformer-Recurrent Looped Transformer	§ [ftip-00OQ]; [zhang2026rlt, Sections 2-3 and 5-8]	Paired change in finite-task utility and $K_{\rm state}$ under recurrent decoder feedback	Same task, model family, parameter count, training tokens, seeds, and feedback interface; causal no-feedback and chunking arms	Report encoder/decoder FLOPs, recurrent state, chunk latency, memory, stability, and state monitorability	Open and finite-only until a declared deployment arm and transfer record exist
Residual baseline: Deep Delta rewrite	§ [ftip-00OQ]; [zhang2026deepdelta, Sections 2-4 and 6]	Paired change in utility and rewrite-sensitive state behavior after the rank-one read/compare/target/write intervention	Matched residual capacity, optimizer, tokens, seeds, and model family; no-write, write-only, and capacity controls	Report inference FLOPs, memory, residual/cache state, numerical stability, and rewrite monitorability	Open; finite GPT-2 evidence does not establish a general capability or frontier claim
Core architecture--JEPA adjunct	I-JEPA, Section 3; § [ftip-00OQ]	Conditional change in $K_{\rm repr}$ and $K_{\rm plast}$ after the objective intervention	Same base artifact, target task, predictor budget, data, and optimizer arm	Report intervention cost, training stability, and representation/probe monitorability	Conditional; an objective intervention is not a replacement architecture
I-JEPA--SplitJEPA	SplitJEPA, Sections 3.1--3.4; § [ftip-00OQ]	Subspace identifiability and conditional change in $K_{\rm repr}$	Matched invariant correspondences, variant-rank premise, normalization, and encoder budget	Report probe cost, numerical and optimization stability, and subspace monitorability	Conditional; identifiability does not become a general I-JEPA guarantee
Core architecture--liquid dynamics	Liquid time-constant networks, Sections 1--2; § [ftip-00OQ]	Difference in utility and $K_{\rm adapt}$ under the solver intervention	Input law, solver, step budget, tolerance, task, and cost vector	Report solver cost, numerical stability, and trajectory/failure monitorability	Open; continuous dynamics and solver choices remain intervention coordinates
Core architecture--Neural Operator	Neural Operators, Section 3; § [ftip-00OQ]	Difference in utility and $K_{\rm repr}$ across discretizations	Function-space task, discretization family, operator layer, and evaluation law	Report training/inference cost, discretization stability, and field-observation monitorability	Open; discretization invariance under one source formulation is not task transfer

A completed row must report its paired observations through § [ftip-00L4] and the common-recipe, equal-tuning, and restricted-envelope arms of § [ftip-00JU]. Until those records are supplied, its status remains open or incompatible.

Remark. Applying a finite model to an architecture [ftip-00M0]

A toy representation or recurrence result may motivate a hypothesis, but it is not transferred to a model instance until its interface, task, cost accounting, numerical regime, and evaluation protocol are instantiated and checked. If any required assumption is unverified, applicability to the model instance remains unestablished.

Definition. Negative-result and stopping report [ftip-00M1]

A completed comparison reports the tested cost grid, excluded configurations, stopping rule, uncertainty, and negative results. Stopping without a detected difference is evidence about the declared study, not proof that the architectures have equal potential.

Remark. Open comparison questions [ftip-00M2]

The matrix in Definition [ftip-00LZ] is a protocol map, not a result table. The following probes keep the five capability coordinates separate and leave an unknown cell visible until the required records exist. Each probe names its source, estimand, controls, and evaluation coordinates so that a missing record cannot be mistaken for a zero score.

State access. Sources: Mamba, Sections 3.1--3.3 and linear attention, Sections 3.2--3.3. Estimand: the paired difference in retrieval utility and $K_{\rm state}$. Controls: Transformer, Mamba, and linear-attention systems on a predeclared retrieval and continuation suite at matched parameter, context, recipe, and cost budgets. Evaluation: report cost, recurrence or summary stability, and state observability. Scope boundary: a long-context score does not transfer to another task family without its task and transport record.
Recurrent feedback. Source: [zhang2026rlt, Sections 2-3 and 5-8]. Estimand: the paired change in finite-task utility and $K_{\rm state}$ from decoder feedback. Controls: a common causal encoder, model size, training tokens, seeds, and matched per-token budget, with no-feedback and declared chunk-size arms. Evaluation: report recurrent-state bytes, encoder/decoder FLOPs, chunk latency, numerical/training stability, and feedback monitorability. Scope boundary: parity, permutation tracking, and modular arithmetic remain finite evidence; without a deployment and transfer record, the cell is finite-only.
Residual rewrite. Source: [zhang2026deepdelta, Sections 2-4 and 6]. Estimand: the paired change in utility and rewrite-sensitive state behavior from a gated rank-one read/compare/target/write edit. Controls: residual capacity, optimizer, tokens, seeds, and model family, with no-write, write-only, and capacity-matched arms. Evaluation: report rewrite and residual-state monitorability, inference FLOPs, memory, numerical stability, and a held-out transfer task. Scope boundary: finite GPT-2 observations do not identify a general capability gain or architecture-wide law.
Adaptive computation. Source: Continuous Thought Machine, Sections 3.1--3.5 and the looped-model contract in § [ftip-00KG]. Estimand: the paired change in utility and $K_{\rm adapt}$ per charged internal tick or loop. Controls: common input, base model, recipe, stopping rule, and charged internal compute. Evaluation: report tick/loop cost, training and inference stability, and stopping-signal monitorability. Scope boundary: an observed adaptive gain remains conditional on that stopping law.
Representation and abstraction. Sources: I-JEPA, Section 3 and SplitJEPA, Sections 3.1--3.4. Estimand: subspace identifiability and the conditional change in $K_{\rm repr}$. Controls: fixed correspondences, full-rank variant differences, normalization, encoder budget, and matched downstream probes. Evaluation: report probe cost, optimization stability, and subspace monitorability. Scope boundary: the conditional subspace result is separate from downstream task transfer; failure of a premise is an incompatible cell, not a zero score.
Plasticity. Source: the objective and optimizer interfaces in Definition [ftip-00JM] and the I-JEPA intervention description in § [ftip-00OQ]. Estimand: the paired change in $K_{\rm plast}$ after a declared post-training intervention. Controls: common recipe and equal-tuning arms with data, optimizer, trial count, stopping, and feedback fixed or explicitly varied. Evaluation: report intervention cost, training stability, and update-signal monitorability. Scope boundary: an intervention effect is limited to the declared base artifact, task, and objective; it does not transfer across architectures without a new paired record.
Portability. Sources: the Mamba, BDH, and Neural Operator method descriptions in § [ftip-00OQ]: Mamba, BDH, and Neural Operators. Estimand: survival of the declared utility and $K_{\rm port}$ across one transfer edge. Controls: source model, task family, model scale, deployment regime, evaluation law, and cost vector retained as paired fields. Evaluation: report migration cost, stability after the shift, and failure monitorability. Scope boundary: evidence for one transfer edge does not order unmeasured edges; blank edges remain unmeasured.
Adjunct interactions. Sources: liquid time-constant networks, Sections 1--2, Neural Operators, Section 3, and I-JEPA, Section 3. Estimand: the joint change in the declared capability coordinates after an explicit intervention. Controls: declared core model, objective, temporal law, function-space bias, matched controls, and separate single-intervention arms. Evaluation: report joint and marginal costs, numerical/training stability, and interaction monitorability. Scope boundary: a joint effect is limited to the declared core, task, temporal law, and function-space regime; it does not transfer across regimes without separate intervention records.

Semantic representation diagnostics along pretraining checkpoints [ftip-00OZ]

This bounded diagnostic records when a declared semantic representation becomes recoverable along a pretraining trajectory. It uses the spatial and temporal probing design of [gurnee2024space, Sections 2-4 and 7] as a measurement template. The result is a representation diagnostic, not a claim about a developmental mechanism, a deployment capability, or a post-training law.

The task family is fixed before inspecting checkpoints: the six coordinate task datasets in the source family (world, United States, and New York City locations; historical figures, cultural works, and news headlines). Each score uses held-out entities. An eligible model family supplies at least three ordered pretraining checkpoints, identical train, validation, and test entity splits and filtering at every checkpoint, and the same layer-wise linear ridge-probe protocol, including a predeclared ridge-regularization grid and validation selection rule. A family or checkpoint missing one of those fields is reported as missing rather than silently pooled.

For model family $m$, checkpoint $c$, task $q$, and layer $\ell $, let $s^{\mathrm {val}}_{m,c,q,\ell }$ and $s^{\mathrm {test}}_{m,c,q,\ell }$ be validation and held-out-test coefficients of determination for the fixed probe. Choose the layer on validation entities, then evaluate once on the test entities:

\[ \ell ^*_{m,c,q}=\arg \max _{\ell \in L_m}s^{\mathrm {val}}_{m,c,q,\ell },\qquad s_{m,c,q}=s^{\mathrm {test}}_{m,c,q,\ell ^*_{m,c,q}}. \]

A predeclared layer may replace the validation choice, but a test score is never used to select $\ell ^*$; this keeps the emergence score from being a layer-selection envelope.

The primary absolute emergence threshold is $\tau =0.50$ in this held-out score, chosen before the trajectory is inspected. The first crossing is

\[ e_{m,q}(\tau )=\min \{c:s_{m,c,q}\geq \tau \}, \]

with $e_{m,q}(\tau )=\bot $ when no checkpoint crosses. A crossing is reported with the checkpoint token count, layer, a bootstrap uncertainty interval over held-out entities, and the probe seed; it is not smoothed into a claim about an unobserved training interval. Sensitivity must include $\tau \in \{0.25,0.50,0.75\}$, model family, parameter scale, and checkpoint spacing.

The structural record contains held-out coordinate prediction, block and cross-entity generalization, prompt variants, and the layer at which the score peaks. These are content and representation probes. A function or deployment record is separate: it must specify an intervention or user task, an outcome metric, matched controls, and its own cost and failure accounting. A probe crossing alone cannot populate that record.

The source reports linear spatial and temporal representations across Llama-2 and Pythia scales, prompt sensitivity, cross-entity robustness, and individual neurons; see [gurnee2024space, Sections 2.1-2.3, 3.1-3.3, 4.1-4.2, and 5]. Those observations motivate the fields above, but every trajectory association reported by this protocol is noncausal. In particular, it makes no claim about which training environment caused the score, whether a representation is used by a deployed function, or what post-training will preserve.

Each trajectory record therefore retains the model family and size, checkpoint token count, task and entity split, probe class and seed, all layer scores, threshold grid, missingness, and the separately reported function or deployment assay. Comparisons are within the declared task family and model family before any cross-family summary. A change in checkpoint spacing or probe protocol is a new arm, not evidence of an earlier or later emergence.

Convention. Identifiability regimes for semantic representation diagnostics [ftip-00P1]

Zhou and Klindt study labels-to-latents and latents-to-labels data-generating directions. Their general assumptions include a fixed continuous invertible map $g:Z\to X$, a continuous learned encoder with $h=f\mathbin \circ g$, a row-centered log-linear posterior $p(y|z)={\rm softmax}(W^*z)$, effective dimension $n'={\rm rank}(W^*)$, and a global population cross-entropy optimum with linear readout $U$. The source boundary for this note is the DGP construction and assumptions in Sections 3.1 to 3.2, Theorems 4.1 to 4.4, Sections 5.1 to 5.4, and Section 6, with Supplementary Assumptions 8.1 to 8.3, of [zhou2026complete, Theorems 4.1 to 4.4, Sections 3 to 5, and Section 6].

Source observation. When $m\geq n'$, Theorem 4.1 makes the classification-relevant part of every optimum linear, $h(z)=Az+\nu (z)$ with $UA=W^*$; the null-readout term satisfies $\nu (z)\in N(U)$ and may remain unconstrained. When $m<n'$, Theorem 4.2 gives generic nonlinear, non-identifiable optima when $\operatorname {supp}(p_Z)$ has nonempty interior $\Omega $, $p_Z$ is continuous and strictly positive on $\Omega $, and every candidate $A$ has $N(A)\cap \Omega \ne \varnothing $. These are different capacity regimes, not two scores for one representation.

The stronger conclusions require additional structure. In the generative case, Assumption 8.2 takes $Z=\mathbb R_+^n$ and adds a factorized exponential conditional, constant conditional entropy, a uniform class prior, exact capacity $m=n=n'$, non-negative activations, fixed within-class variance, and an energy-minimizing global optimum. Theorem 4.3 then gives the scaled permutation $h(z)=DP_\sigma z$. In the discriminative case, Assumption 8.3 uses $A^*\in \mathbb R^{n\prime \times n}$ with full row rank and $U^*\in \mathbb R^{|Y|\times n\prime }$, adds $W^*=U^*A^*$, $k$-sparse latents, and $\operatorname {spark}(A^*)>2k$. With $m\geq n\prime $, let $P_U$ be the orthogonal projector onto $\operatorname {Row}(U)$; the classifier-visible encoder $P_Uh(z)=Az$ has $N(A)=N(A^*)$, so the unique $k$-sparse solution can recover $z$. Axis alignment and sparse recovery are distinct routes through distinct assumptions.

Section 5 reports finite experiments rather than a universal law. Linear alignment uses held-out coefficient of determination for a linear decoder; axis alignment uses the mean correlation coefficient (MCC) after Hungarian matching recovered axes to true latents. Sparse recovery compares supervised Lasso, dense OLS, and unsupervised sparse dictionary learning. Cross-system correspondence uses the Maximum Matching Unit (MMU): for each source unit, the maximum absolute Pearson correlation with any unit in a declared comparison model and layer on a declared stimulus set. The synthetic experiments use ten random seeds; the real-data cross-system analysis does not inherit that seed count. Section 6 records limits from constant conditional entropy, non-negativity, class priors, and the assumed sparse data-generating process; signed latents do not inherit Theorem 4.3 directly.

Finite identifiability record. For a fixed model family, checkpoint, task split, and probe seed, record

\[ \mathcal D=\bigl (r,m,n,n',\mathcal A,[h]_{\sim _r},R^2_{\rm lin}, R^2_{\rm sparse},\operatorname {MCC}_{\rm axis}, \operatorname {MCC}_{\rm sparse},\operatorname {MMU}_{\rm cross},\mathsf E\bigr ). \]

Here $r$ is one declared regime: sufficient-capacity linear, undercomplete nonlinear, generative axis-aligned, or discriminative sparse recovery. $\mathcal A$ lists the causal direction, DGP, capacity, sparsity, and assumptions actually held. $R^2_{\rm lin}$ is the held-out coefficient for linear latent alignment; $R^2_{\rm sparse}$ names the decoder and target, with supervised Lasso and dense OLS kept as separate arms. $\operatorname {MCC}_{\rm axis}$ records the matching and correlation convention for axis recovery, while $\operatorname {MCC}_{\rm sparse}$ records the matching convention, sparsity level, and whether the decoder is supervised or an unsupervised dictionary. $\operatorname {MMU}_{\rm cross}$ records the directed source and comparison model, layers, unit sets, stimulus set, and correlation rule. An unavailable or inapplicable field is $\bot $.

The equivalence class identifies only the ambiguity allowed by the declared regime: $(A,\nu )$ with $UA=W^*$, $\operatorname {rank}(A)=n'$, and $\nu (z)\in N(U)$ in the linear regime; no universal quotient, recorded as non-identifiable under the stated assumptions, in the undercomplete regime; positive diagonal scaling and permutation $(D,P_\sigma )$ in the axis-aligned regime; and the classifier-visible null-space class $N(A)=N(A^*)$ in the sparse regime. A sparse decoder has permutation-and-scaling ambiguity only when that convention is declared and measured. No scalar summary is defined, and none of these fields alone establishes monosemanticity.

Source-grounded finite instance. In the compressed-sensing arm, the paper fixes $n=16$, $n'=m=8$, $|Y|=64$, draws a Gaussian measurement matrix with normalized columns, and tests $k\in \{1,2,3\}$ over ten seeds. For the $k=1$ arm, the record declares discriminative sparse recovery and the $W^*=U^*A^*$ and spark conditions; it reports $R^2_{\rm sparse}\approx 1$ for supervised Lasso, $R^2_{\rm sparse}\approx n'/n=0.5$ for OLS, and $\operatorname {MCC}_{\rm sparse}\approx 0.86$ for unsupervised Lasso, with $\operatorname {MCC}_{\rm axis}=\bot $ and $\operatorname {MMU}_{\rm cross}=\bot $. The ten-seed estimate, decoder arm, and missing fields are part of $\mathsf E$; the values are finite synthetic observations, not a deployment or capability claim.

FTIP deduction. This record makes the representation question conditional on capacity, causal direction, and an explicit equivalence class. It lets the existing checkpoint route report linear recoverability, axis alignment, sparse recovery, and cross-system matching without treating any one readout as a capability or deployment result. A finite observation can support a regime-specific comparison; it cannot identify a developmental mechanism or transfer beyond the declared task and assumptions.

Proposed protocol. Before inspecting a checkpoint, freeze the task family, latent or proxy labels, split, width ratio, causal regime, assumptions, decoder class, comparison model and layer for MMU, stimulus set, and held-out evaluator. Select layers or decoder settings on validation data only. Report the full record, seed repetitions, missingness, and assumption-violation arms; keep each $R^2$, MCC, and MMU in its native units. For real model activations, treat latent recovery as a declared proxy assay and report the proxy construction with the same boundary.

A scoped local formalization search of Forest, formal-math, TauCetiReview, and TauCetiWorker found no declaration that packages this four-regime record with its separately typed readouts. Existing representation and evaluation declarations cover neighboring fields only, so this finite identifiability record remains a protocol proposal.

This protocol links the conceptual-discovery question in § [ftip-00MH] to the capability coordinates in Definition [ftip-00JM]; the panel of open comparison questions in Remark [ftip-00M2] transcludes this child.

These probes do not establish a common ordering of architecture families. They specify what would count as evidence for a declared bottleneck change, while cost, numerical stability, monitorability, and transfer remain separate evaluation coordinates. Paired observations should use § [ftip-00L4], the study arms of § [ftip-00LS], and the capability vector of Definition [ftip-00JM]. Recurrent feedback and residual rewrite remain separate arms in every record; a finite result without a deployment arm stays finite-only.

Optimizer geometry and representation dependence [ftip-00AU]

A scalar objective does not by itself determine a training intervention. This section studies an exact finite-dimensional witness: one represented matrix can have many factor bases, the loss can be constant along that gauge orbit, and an update rule can nevertheless distinguish the bases.

The principal source is the version-one arXiv preprint The loss does not see the basis, but Adam does[singh2026lossbasis]. Its algebraic results assume the stated dimensions, rank conditions, state conventions, and real arithmetic. Experimental observations concern the reported configurations.

Factored objectives and gauge symmetry [ftip-00AV]

We first separate a represented matrix from a choice of factors and then separate its full product-preserving symmetry from the orthogonal subgroup that preserves the Euclidean parameter metric.

Notation. Factored objective and represented matrix [ftip-00AW]

Fix integers $d_1,d_2,k\geq 1$. Let $U\in \mathbb R^{d_1\times k}$ and $V\in \mathbb R^{d_2\times k}$ be factors, let

\[ W(U,V)=UV^{\mathsf T}\in \mathbb R^{d_1\times d_2}, \]

and let $f:\mathbb R^{d_1\times d_2}\to \mathbb R$ be differentiable. The corresponding factored objective is $L(U,V)=f(W(U,V))$. The pair $(U,V)$ is a parameterization; $W(U,V)$ is the represented object on which $f$ depends. This is the rectangular form noted in Section 3 of The loss does not see the basis, but Adam does[singh2026lossbasis].

Definition. General linear gauge action [ftip-00AX]

For $A\in \mathrm {GL}(k)$, define the general linear gauge action by

\[ \gamma _A(U,V)=(UA,VA^{-\mathsf T}). \]

The inverse transpose on the second factor is essential for a general invertible $A$. When $A=Q\in \mathrm O(k)$, one has $Q^{-\mathsf T}=Q$, and the action becomes $(UQ,VQ)$.

Lemma. The gauge action preserves the product and loss [ftip-00AY]

For every $A\in \mathrm {GL}(k)$,

\[ W(UA,VA^{-\mathsf T})=UV^{\mathsf T} \quad \hbox {and}\quad L(UA,VA^{-\mathsf T})=L(U,V). \]

Proof.

Since $(VA^{-\mathsf T})^{\mathsf T}=A^{-1}V^{\mathsf T}$, the first identity is $UAA^{-1}V^{\mathsf T}=UV^{\mathsf T}$. Applying $f$ gives the second.

This is the factored-model calculation in Section 3 of The loss does not see the basis, but Adam does[singh2026lossbasis].

Definition. Orthogonal gauge orbit [ftip-00AZ]

The orthogonal gauge orbit of $(U,V)$ is

\[ \mathcal O(U,V)=\{(UQ,VQ):Q\in \mathrm O(k)\}. \]

Every pair in the orbit represents the same $W$. This orbit is generally smaller than the complete fibre of the map $(U,V)\mapsto UV^{\mathsf T}$. The restriction to $\mathrm O(k)$ is geometric: it preserves the Euclidean metric on factor space.

Lemma. The orthogonal gauge is the maximal isometric subgroup [ftip-00B0]

The action $\gamma _A$ preserves $\lVert U\rVert _F^2+\lVert V\rVert _F^2$ for every pair $(U,V)$ if and only if $A\in \mathrm O(k)$.

Proof.

If the action is an isometry, take $V=0$ and let one row of $U$ be an arbitrary $u\in \mathbb R^{1\times k}$. Then $\lVert uA\rVert _2=\lVert u\rVert _2$ for every $u$, hence $AA^{\mathsf T}=I$. Conversely, right multiplication by an orthogonal matrix preserves both Frobenius norms.

See Lemma 3.2 and its proof in Appendix B.1 of The loss does not see the basis, but Adam does[singh2026lossbasis].

Lemma. Gradients transform covariantly along an orthogonal orbit [ftip-00B1]

For $Q\in \mathrm O(k)$,

\[ \nabla _U L(UQ,VQ)=\nabla _U L(U,V)Q, \qquad \nabla _V L(UQ,VQ)=\nabla _V L(U,V)Q. \]

Proof.

Put $G=\nabla f(W)$. Then $\nabla _U L=GV$ and $\nabla _V L=G^{\mathsf T}U$. The represented matrix, and hence $G$, is unchanged along the orbit. Substitution gives both identities.

See Lemma 4.1 and its proof in Appendix B.2 of The loss does not see the basis, but Adam does[singh2026lossbasis].

Definition. Optimizer state action [ftip-00B2]

Use the optimizer-state convention of Definition [ftip-001H]. For every $Q\in \mathrm O(k)$, an optimizer state action is a map $\sigma _Q:\mathcal S_{\rm opt}\to \mathcal S_{\rm opt}$ compatible with the shapes of the factor states and satisfying $\sigma _Q(s_0)=s_0$ for the declared initial state.

For example, a momentum buffer shaped like $U$ transforms by right multiplication by $Q$; a shared scalar second moment is fixed. An entrywise second-moment array need not admit a state action compatible with every orthogonal $Q$.

Definition. Gauge-equivariant optimizer [ftip-00B3]

Let $\mathcal A$ be a deterministic instance of the parameter-update rule of Definition [ftip-001I], acting on $(U,V,s)$. It is gauge-equivariant when for every $Q\in \mathrm O(k)$ there is a state action $\sigma _Q$ such that gauge-related initial states produce

\[ (\widetilde U_t,\widetilde V_t,\widetilde s_t) =(U_tQ,V_tQ,\sigma _Q(s_t)) \qquad (t\geq 0). \]

This is Definition 3.1 of The loss does not see the basis, but Adam does[singh2026lossbasis]. Any randomness, schedules, stopping rule, and mixed update blocks must also be coupled as required by Remark [ftip-001J]; the definition does not hide them.

Example. A commuting optimizer square [ftip-00B4]

One update of a gauge-equivariant optimizer makes the following square commute. The vertical arrows change factor basis and the horizontal arrows apply the same declared update.

$\text{[math]}$

Commutation is stronger than equality of the scalar losses before the step. It states equality of the represented product trajectory after aligning the parameter bases.

Remark. Product symmetry is larger than the isometric gauge [ftip-00B5]

The full $\mathrm {GL}(k)$ action in Definition [ftip-00AX] preserves $W$, but ordinary Euclidean gradients are covariant under the orthogonal specialization used in Lemma [ftip-00B1]. A theorem for $(UQ,VQ)$ must therefore not be silently promoted to every product-preserving reparameterization.

The source calls the orthogonal subgroup the gauge in its optimizer classification while also noting the larger function-preserving group. These are compatible statements only when their different geometric scopes remain explicit.

Equivariant and basis-sensitive update rules [ftip-00B6]

Gradient covariance supplies a test for update rules. Linear and full-matrix operations can commute with a latent rotation, whereas a fixed nonlinear map applied separately to coordinates selects a preferred basis.

Definition. Memoryless right-equivariant update [ftip-00B7]

A memoryless factor-update map is a function $\Phi :\mathbb R^{n\times k}\to \mathbb R^{n\times k}$ applied to a current gradient $G$ without a persistent optimizer state. It is right-equivariant when

\[ \Phi (GQ)=\Phi (G)Q \qquad (G\in \mathbb R^{n\times k},\ Q\in \mathrm O(k)). \]

This local definition is the input of Theorem 4.5 in The loss does not see the basis, but Adam does[singh2026lossbasis]. It does not classify stateful optimizers such as Adam.

Proposition. Gradient descent and momentum preserve the orthogonal gauge [ftip-00B8]

Gradient descent and Polyak or Nesterov momentum are gauge-equivariant when their factor gradients, momentum buffers, schedules, and look-ahead points are transformed consistently.

Proof.

By Lemma [ftip-00B1], $G_t$ becomes $G_tQ$. Gradient descent therefore sends $U_tQ$ to $(U_t-\eta _tG_t)Q$. A momentum buffer initialized at zero and formed by linear combinations of covariant gradients transforms as $M_tQ$; a Nesterov look-ahead point is likewise the reference look-ahead right-multiplied by $Q$. Induction proves the claim.

Proposition 4.2(1) and its proof are in Appendix B.3 of The loss does not see the basis, but Adam does[singh2026lossbasis].

Proposition. A shared scalar second moment preserves the orthogonal gauge [ftip-00B9]

Replace Adam's entrywise second-moment denominator by one scalar whose updates depend only on gauge-invariant quantities such as the pooled mean of the squared entries of both factor gradients. With a covariant first-moment buffer, the resulting stateful update is gauge-equivariant.

Proof.

The first moment is a linear exponential average and transforms by right multiplication by $Q$. The pooled squared-entry mean is the joint squared Frobenius norm divided by the entry count, hence is unchanged by $G\mapsto GQ$. Its bias correction and positive scalar denominator are unchanged, so the complete update transforms covariantly.

Proposition 4.2(2) and its proof are in Appendix B.3 of The loss does not see the basis, but Adam does[singh2026lossbasis]. The result concerns this declared shared-scalar variant, not stock Adam.

Proposition. Full-matrix Muon and Shampoo preserve the orthogonal gauge [ftip-00BA]

Fix one factor, write $n\in \{d_1,d_2\}$ for its row dimension, and let $G_t,M_t\in \mathbb R^{n\times k}$ be its gradient and Muon momentum buffer. For $0\leq \beta <1$, $M_{-1}=0$, and the source's linear momentum update $M_t=\beta M_{t-1}+G_t$, Muon's direction is $\Delta _t=\operatorname {msign}(M_t)$. Under real arithmetic this stateful update and the source's damped Shampoo update are right-equivariant. The Muon claim includes the stated finite Newton--Schulz approximations; the Shampoo claim uses its complete left and right accumulators.

Proof.

By Lemma [ftip-00B1], the primed gradient is $G'_t=G_tQ$. Starting from $M'_{-1}=M_{-1}Q=0$, induction through the linear momentum update gives $M'_t=M_tQ$. If $M_t=A\Sigma B^{\mathsf T}$ is a compact singular-value decomposition, then $M_tQ=A\Sigma (Q^{\mathsf T}B)^{\mathsf T}$, so $\operatorname {msign}(M_tQ)=\operatorname {msign}(M_t)Q$. For $\epsilon _{\rm NS}>0$, the Newton--Schulz initialization $X_0=M_t/(\lVert M_t\rVert _F+\epsilon _{\rm NS})$ is covariant because its Frobenius normalization is invariant; each polynomial iterate formed from $X_j$ and $X_jX_j^{\mathsf T}$ preserves the same covariance. For Shampoo, the left accumulator is invariant while the right accumulator transforms by $R_t\mapsto Q^{\mathsf T} R_t Q$. Orthogonal functional calculus conjugates the right matrix function, so the transformed update is the original update right-multiplied by $Q$.

Proposition 4.2(3)--(4) and its proof are in Appendix B.3 of The loss does not see the basis, but Adam does[singh2026lossbasis]. Variable splitting, coordinatewise clipping, mixed update rules, finite precision, and different damping conventions lie outside this statement. The Muon part is stateful; it does not replace the momentum buffer by the current gradient or invoke the memoryless classification of Definition [ftip-00B7].

Proposition. Coordinatewise equivariance forces linearity [ftip-00BB]

Let $k\geq 2$ and let a memoryless update have the fixed entrywise form $\Phi (G)_{ij}=\phi (G_{ij})$. If $\Phi (GQ)=\Phi (G)Q$ for every $G$ and $Q\in \mathrm O(k)$, then $\phi (x)=cx$ for some $c\in \mathbb R$.

Proof.

It suffices to use one row and rotations in the first two coordinates. Equivariance at zero gives $\phi (0)=0$. Rotating $(x,0)$ through an angle whose cosine is $a\in [-1,1]$ gives $\phi (ax)=a\phi (x)$. For nonzero $x,y$, compare each with a third number of at least their absolute magnitudes; the ratio $\phi (x)/x$ is constant.

Proposition 4.3 and its proof are in Appendix B.4 of The loss does not see the basis, but Adam does[singh2026lossbasis]. No continuity assumption is needed.

Corollary. Fixed nonlinear coordinatewise updates break the gauge [ftip-00BC]

A nonlinear fixed entrywise update applied at a zero optimizer state is not gauge-equivariant when $k\geq 2$. In particular, the source applies Proposition [ftip-00BB] to the first-step maps of Adam, RMSProp, signSGD, and Lion; it checks Adafactor separately because its factored statistics are not an entrywise map.

This is a first-step obstruction. It suffices to disprove equivariance of the full run, but it does not classify every later state or say that such an optimizer cannot reach a particular solution.

Definition. First-step Adam map and gauge defect [ftip-00BD]

Fix $\epsilon >0$. For zero-state, bias-corrected Adam define the entrywise first-step direction

\[ D_\epsilon (G)=G\mathbin {\oslash }(|G|+\epsilon ) \]

and, for $Q\in \mathrm O(k)$, define its gauge-aligned defect by

\[ E_Q(G)=D_\epsilon (GQ)Q^{\mathsf T}-D_\epsilon (G). \]

Here $|G|$, addition, and division are entrywise. Bias correction makes the first direction independent of Adam's two decay coefficients under the declared convention [singh2026lossbasis, Proposition 4.4].

Proposition. Exact represented-matrix defect after one Adam step [ftip-00BE]

Let $G_U=\nabla _UL(U_0,V_0)$, $G_V=\nabla _VL(U_0,V_0)$, $D_U=D_\epsilon (G_U)$, and $D_V=D_\epsilon (G_V)$. Let $W_1$ be the product after one Adam step from $(U_0,V_0)$ and $\widetilde W_1$ the product after one step from $(U_0Q,V_0Q)$. Then

\[ \begin {aligned} \widetilde W_1-W_1={}&-\eta \bigl (E_Q(G_U)V_0^{\mathsf T} +U_0E_Q(G_V)^{\mathsf T}\bigr )\\ &+\eta ^2\bigl (E_Q(G_U)D_V^{\mathsf T} +D_UE_Q(G_V)^{\mathsf T} +E_Q(G_U)E_Q(G_V)^{\mathsf T}\bigr ). \end {aligned} \]

Proof.

Gauge-align the rotated factors back by $Q^{\mathsf T}$. They are $U_0-\eta (D_U+E_Q(G_U))$ and $V_0-\eta (D_V+E_Q(G_V))$. Multiply, subtract $(U_0-\eta D_U)(V_0-\eta D_V)^{\mathsf T}$, and collect powers of $\eta $.

Proposition 4.4 and its proof are in Appendix B.5 of The loss does not see the basis, but Adam does[singh2026lossbasis].

Example. A two-coordinate Adam defect witness [ftip-00BF]

Take $d_1=d_2=1$, $k=2$, $f(w)=w^2/2$, $U_0=V_0=(1,0)$, and

\[ Q=2^{-1/2}\begin {pmatrix}1&1\\-1&1\end {pmatrix}. \]

The original and rotated products after one zero-state Adam step are

\[ W_1=\left (1-\frac {\eta }{1+\epsilon }\right )^2, \qquad \widetilde W_1=\left (1-\frac {\eta }{2^{-1/2}+\epsilon }\right )^2. \] $\text{[math]}$

They differ when $0<\eta <2^{-1/2}+\epsilon $. This is the explicit witness in Proposition 4.4 of The loss does not see the basis, but Adam does[singh2026lossbasis]; it is a statement about one factored quadratic, not about language-model performance.

Example. A rotated sign update fails to commute [ftip-00BG]

For $G=(1,1)$ and the 45-degree rotation matrix $Q$ of Example [ftip-00BF], one has $GQ=(0,\sqrt 2)$. With $\operatorname {sign}(0)=0$,

\[ \operatorname {sign}(GQ)=(0,1), \qquad \operatorname {sign}(G)Q=(0,\sqrt 2). \]

Thus even this two-coordinate update violates the commuting equation in Definition [ftip-00B7]. The calculation is the explicit Appendix B.4 witness of The loss does not see the basis, but Adam does[singh2026lossbasis].

Theorem. Full-rank memoryless equivariant rules are Gram-determined [ftip-00BH]

Let $k\leq n$, and let $\Phi $ be defined on full-column-rank matrices $G\in \mathbb R^{n\times k}$. Then $\Phi (GQ)=\Phi (G)Q$ for every $Q\in \mathrm O(k)$ if and only if there is a matrix-valued function $H$ of $GG^{\mathsf T}$ such that

\[ \Phi (G)=H(GG^{\mathsf T})G. \]

Proof.

The displayed form is immediately equivariant. Let $G^+$ denote the Moore--Penrose pseudoinverse. Conversely, set $X(G)=\Phi (G)G^+$. Then $X(G)G=\Phi (G)$ and $X(GQ)=X(G)$. Two full-column-rank matrices with the same left Gram matrix have the same range and differ by a right orthogonal factor. Hence $X(G)$ depends only on $GG^{\mathsf T}$; take $H=X$.

See Theorem 4.5 and its proof in Appendix B.6 of The loss does not see the basis, but Adam does[singh2026lossbasis]. It classifies the full-rank stratum only.

Corollary. The square invertible preconditioner representation [ftip-00BI]

In the specialization $k=n$ with $G$ invertible, let $P^{1/2}$ denote the unique positive-definite square root of $P$. The function in Theorem [ftip-00BH] has the unique canonical representative

\[ H(P)=\Phi (P^{1/2})P^{-1/2}, \qquad P=GG^{\mathsf T}>0. \]

Proof.

The polar decomposition writes $G=P^{1/2}Q$. Equivariance gives $\Phi (G)=\Phi (P^{1/2})Q=H(P)G$. Evaluating at $G=P^{1/2}$ proves uniqueness in this square invertible case.

For rectangular $G$, the action of $H(P)$ away from the range needed to multiply $G$ is not determined by $\Phi $. The proof above establishes uniqueness in the square invertible specialization.

Remark. What the structure theorem leaves unresolved at rank deficiency [ftip-00BJ]

In the ambient matrix space, full-column-rank matrices form an open dense subset whose complement has Lebesgue measure zero. A probability-one claim requires the random gradient law to be absolutely continuous and not confined to a lower-rank set; random initialization alone does not supply that premise.

Theorem [ftip-00BH] therefore makes no classification claim on a general rank-deficient stratum. Equivariance still constrains values within each orthogonal orbit, and it forces $\Phi (0)=0$, but it does not determine the rule there by continuity unless continuity is separately assumed. The source's generic-rank discussion therefore does not establish a classification at rank deficiency.

Common-scalar flows, clock changes, and momentum [ftip-00BK]

One narrow class of continuous-time preconditioners follows the gradient-flow path at a different speed. The clock change transfers path properties, but not rates, and it does not encompass a stateful momentum method.

Definition. Common-scalar preconditioned flow [ftip-00BL]

Let $\theta =(U,V)$ and let $a(t)>0$ be measurable with $1/a$ locally integrable. A common-scalar preconditioned flow is a locally absolutely continuous trajectory satisfying

\[ \dot \theta (t)=-\frac {\nabla L(\theta (t))}{a(t)} \]

for almost every $t$. The same scalar multiplies every coordinate of both factor gradients. For gauge-uniform conclusions, the source additionally requires $a(t)$ to be determined by gauge-invariant statistics of the trajectory up to time $t$.

Definition. Effective optimizer clock [ftip-00BM]

For the flow of Definition [ftip-00BL], define the effective optimizer clock

\[ \tau (t)=\int _0^t\frac {du}{a(u)}, \qquad \tau _{\max }=\int _0^\infty \frac {du}{a(u)}, \qquad 0<\tau _{\max }\leq \infty . \]

The clock is strictly increasing on its domain. If $\tau _{\max }=\infty $, it reaches every gradient-flow time; otherwise it traverses only the prefix with effective time below $\tau _{\max }$.

Theorem. Common-scalar flow is a time-reparameterized gradient flow [ftip-00BN]

Let $\theta (t)$ satisfy Definition [ftip-00BL], let $t(\tau )$ be the inverse of the clock in Definition [ftip-00BM], and put $\widetilde \theta (\tau )=\theta (t(\tau ))$. Then, for almost every $\tau $,

\[ \widetilde \theta '(\tau )=-\nabla L(\widetilde \theta (\tau )). \]

Proof.

The chain rule for absolutely continuous changes of variable gives $dt/d\tau =a(t)$ almost everywhere. Multiplying $\dot \theta =-\nabla L/a(t)$ by $dt/d\tau $ yields the displayed gradient flow.

See Theorem 4.6 and its proof in Appendix B.9 of The loss does not see the basis, but Adam does[singh2026lossbasis]. Gauge invariance of $a$ is not needed for this single-trajectory clock identity; it makes the clock common across a gauge orbit.

Corollary. Transfer of path and limit properties [ftip-00BO]

Every property depending only on the portion of the gradient-flow path traversed before $\tau _{\max }$ transfers to the common-scalar flow. If $\tau _{\max }=\infty $ and gradient flow converges, both flows have the same limit point. A sufficient condition for clock divergence is an eventual finite upper bound on $a(t)$.

For shallow factorization, the source cites Implicit regularization in matrix factorization[gunasekar2017implicit]. For deep factorization, it cites Implicit regularization in deep matrix factorization[arora2019implicit]. For greedy low-rank dynamics, it cites Towards resolving the implicit bias of gradient descent for matrix factorization: Greedy low-rank learning[li2021greedy].

Those conclusions transfer only when both this clock theorem and every hypothesis of the original result hold. This corollary supplies no missing matrix-sensing assumption.

Remark. Rates and hitting times do not survive time reparameterization [ftip-00BP]

Two flows can traverse the same curve with arbitrarily different physical clocks. A convergence rate, finite-step budget, or hitting time stated in $t$ therefore does not transfer through Theorem [ftip-00BN] without quantitative bounds on $a$ and its integral. If $\tau _{\max }<\infty $, even the tail and limit of the gradient-flow path are not reached.

Remark. Scalar-Adam is outside the common-scalar flow theorem [ftip-00BQ]

The shared-scalar Adam variant in Proposition [ftip-00B9] retains a first-moment exponential moving average. It is stateful and is not the memoryless flow in Definition [ftip-00BL]. Remark 4.7 of The loss does not see the basis, but Adam does[singh2026lossbasis] treats its agreement with gradient flow as empirical rather than as a consequence of Theorem 4.6.

The theorem likewise does not apply to discrete Adam, PPO, RLVR update loops, clipping, weight decay, and mixed optimizers. Gauge equivariance and time reparameterization are separate properties.

Example. Equivariance neither guarantees nor precludes recovery [ftip-00BR]

The source records controls on both sides of the proposed implication. Its ScaledGD-inspired control is equivariant but equalizes the spectral schedule and does not recover the planted low-rank target at the declared budget. A long-anneal sign update is non-equivariant but eventually reaches a low-recovery-error regime.

$\text{[math]}$

The controls are reported in Remark 4.7 and Appendix D.10 of The loss does not see the basis, but Adam does[singh2026lossbasis]. The source reports no tuned reproducibility baseline for the ScaledGD control. The diagram is not a convergence theorem.

Optimizer geometry in training interventions [ftip-00BS]

Equivalent parameterizations need not follow equivalent training trajectories. Their evolution also depends on the update geometry; the scalar loss and represented starting object do not always suffice.

Definition. Objective-equivalent parameterizations [ftip-00BT]

Two factor pairs $(U,V)$ and $(\widetilde U,\widetilde V)$ are objective-equivalent for $L=f\circ W$ when

\[ W(U,V)=W(\widetilde U,\widetilde V). \]

They then have the same scalar loss for every objective depending only on that represented matrix. Orthogonal gauge-related pairs are objective-equivalent by Lemma [ftip-00AY]; objective equivalence need not imply membership in the same orthogonal orbit.

Definition. Parameterization-stable training intervention [ftip-00BU]

Fix a factorized objective, an update algorithm, hyperparameter schedule, randomness coupling, stopping rule, and initial optimizer state. The resulting training intervention is parameterization-stable on an orthogonal orbit when every two gauge-related initial states have identical represented trajectories:

\[ \widetilde U_t\widetilde V_t^{\mathsf T}=U_tV_t^{\mathsf T} \qquad \hbox {at every compared step }t. \]

This is a local stability property of a fully declared intervention. It does not say that arbitrary objective-equivalent pairs outside the orbit must agree.

Proposition. Gauge equivariance gives parameterization stability [ftip-00BV]

Every gauge-equivariant optimizer is parameterization-stable on each orthogonal orbit, provided the remaining protocol coordinates are coupled as in Definition [ftip-00BU].

Proof.

Definition [ftip-00B3] gives $(\widetilde U_t,\widetilde V_t)=(U_tQ,V_tQ)$. Hence

\[ \widetilde U_t\widetilde V_t^{\mathsf T} =U_tQQ^{\mathsf T}V_t^{\mathsf T}=U_tV_t^{\mathsf T}. \]

This is a sufficient condition on a declared orbit, not a claim that equivariance is necessary for every possible equality of represented trajectories.

Proposition. A first-step defect falsifies a loss-only protocol specification [ftip-00BW]

Suppose two orthogonally gauge-related initializations have equal represented object and scalar objective, but a declared update produces different represented objects after one step. Then the training intervention is not determined by the represented initialization and scalar objective alone.

Proof.

If those two objects determined the intervention, equal inputs would give the same represented next state. The witnessed inequality contradicts that factorization.

Proposition Proposition [ftip-00BE] and Example [ftip-00BF] instantiate its premise for one factored quadratic and zero-state Adam. It proves underspecification, not a capability difference.

Example. The orthogonal gauge of an attention head [ftip-00BX]

For the row-vector attention weights of Definition [ftip-000P], let $W^Q,W^K\in \mathbb R^{d\times d_h}$ and $Q_h=XW^Q$, $K_h=XW^K$. Attention logits depend on these weights through

\[ Q_hK_h^{\mathsf T}=XW^Q(W^K)^{\mathsf T}X^{\mathsf T}. \]

For $A\in \mathrm O(d_h)$, the right action $(W^Q,W^K)\mapsto (W^QA,W^KA)$ preserves the middle product.

$\text{[math]}$

Section 6 of The loss does not see the basis, but Adam does[singh2026lossbasis] uses column-vector weights and the invariant $W_Q^{\mathsf T}W_K$. Transposition gives the row-vector invariant $W^Q(W^K)^{\mathsf T}$, with the action on the right as displayed.

Remark. What the attention-twin experiment observes [ftip-00BY]

Section 6 and Table 3 of The loss does not see the basis, but Adam does[singh2026lossbasis] compare two function-equivalent initializations of a two-layer, four-head transformer on modular addition. The reported relative logit distance after one Adam step is $3.6\times 10^{-3}$, versus $2.9\times 10^{-7}$ for a same-basis noise twin; the final per-head invariant distance is reported as 56 percent.

The exact first-step defect has an algebraic explanation, but later drift, scale robustness, and downstream behavior are empirical. The tested small transformers, precision, optimizer allocation, seeds, and task do not establish a frontier-wide law.

Example. One loss quotient and two optimizer trajectories [ftip-00BZ]

A loss-only description collapses a gauge orbit to one represented starting point. A basis-sensitive optimizer can split that quotient back into different represented trajectories.

$\text{[math]}$

The right-hand split is possible only because the centre box omits the factor basis and optimizer geometry. An FTIP admissible protocol in Definition [ftip-005J] must retain those intervention coordinates when they affect the executable result.

Remark. Product symmetry and the scope of optimizer conclusions [ftip-00C0]

The relevant optimizer results appear in version 1 of [⧉], submitted 5 August 2026: Sections 3--4 and 6, Appendices B.1--B.6 and B.9, with limitations in Section 11.

The full product symmetry acts on the factors by

\[ (U,V)\longmapsto (UA,VA^{-\mathsf T}). \]

It does not use $(UA,VA)$. For row-vector attention weights, the corresponding action uses right multiplication, as in Example [ftip-00BX].

The general rectangular preconditioner representation need not have the square-case uniqueness. A probability-one full-rank claim additionally needs a law for the gradient; a generic full-rank observation does not specify one.

The coordinatewise theorem is a zero-state, memoryless obstruction. The clock theorem is continuous-time and excludes scalar-Adam's momentum. Experimental optimizer and attention results remain observations reported by the source. Equivariance is not identified with low-rank recovery.

Remark. Continuous-flow conclusions and discrete optimization [ftip-00C1]

The factored-model results prove that update geometry can be a genuine intervention coordinate even when objective and represented initialization are fixed. They do not prove transformer or RLVR convergence, practical optimizer superiority, a scaling law, or any change in reliable capability. Different represented trajectories are not by themselves an acquisition witness of Definition [ftip-005X].

The memoryless theorem does not describe Adam's full state, and the continuous-flow result does not supply discrete rates or finite-step bounds. The source also notes that fixed-rank LoRA changes the learned-rank mechanism; no LoRA transfer is made here.

A dynamic-compute comparison also depends on active research state, persistent harness state, retained history, evaluation configuration, and descendant and retry costs. One realized chain and an archive-best envelope are different outcomes, even at the same declared budget.

Definition. optimizer update support and effective rank [ftip-00OX]

The source question is whether reinforcement-learning updates need the momentum and per-parameter adaptive rates supplied by AdamW. Mukherjee et al. compare SGD, SGD with momentum, RMSProp, and AdamW in GRPO and PPO settings across mathematical, coding, and adaptive-verifiable environments ([mukherjee2026adam, Sections 4-6 and conclusion]). In the reported runs, SGD matches or often exceeds AdamW, while its updates are substantially sparser and lower-rank; the source reports fewer than $0.02$ percent changed parameters in representative cases and a 15.7 GB peak-memory reduction for Qwen3-1.7B. These are observations about the stated models, data, precision, threshold, seeds, and budgets.

The relevant FTIP intervention is therefore a tuple, not an optimizer name: model and checkpoint, RL algorithm, prompt and environment laws, optimizer and hyperparameters, numerical precision, update-support rule, effective-rank rule, training and evaluation budgets, and the held-out capability interface. A reported reward or pass rate does not identify the update geometry that produced it.

Let $\theta _0$ and $\theta _1$ be the parameters before and after a declared finite intervention, with $n$ trainable scalar parameters. For threshold $\varepsilon >0$, define \[ S_{\varepsilon }(\theta _0,\theta _1)= \frac {1}{n}\#\{i:|\theta _{1,i}-\theta _{0,i}|>\varepsilon \}, \]

and, for every two-dimensional weight update $\Delta W$, let $\operatorname {rank}_{.99}(\Delta W)$ be the least number of singular values whose squared sum reaches 99 percent of the total squared singular-value sum. A study must report the threshold, precision, tensor selection, and aggregation rule alongside these numbers. The source uses bf16 and a $10^{-5}$ threshold for its sparsity calculation; that convention is a source field, not a universal definition.

A matched Phase B comparison holds model family, checkpoint, training data and order, token and environment steps, rollout limits, precision, gradient accumulation, seeds, evaluation prompts, and stopping rule fixed while changing SGD or AdamW. Learning-rate and optimizer tuning must use the same predeclared trial count, search ranges, selection data, and tuning seeds, with tuning tokens, steps, wall-clock, and energy charged to both arms. The protocol also declares an optimizer-state memory cap (or explicitly reports no cap) and records persistent-state bytes, peak memory, wall-clock and energy, update support, effective rank, verifier/evaluator cost, and a prespecified held-out transfer task. Capability change and support change are reported as separate outcomes; a lower-rank update is not itself an acquisition witness.

The source does not establish general SGD superiority, a universal low-rank plasticity law, or transfer to long-horizon agents. Deep Delta's learned read/compare/write mechanism changes a different coordinate and remains in the later recurrence/rewrite batch.

Remark. Fixed-weight adaptation and joint compute interventions [ftip-00D9]