Evaluation binding and lineage accounting [ftip-00CD]
✍️sourceAGENTDRAFTED
Evaluation binding and lineage accounting [ftip-00CD]
✍️sourceAGENTDRAFTED
A long-horizon score is attached to a configuration, not merely to a model name. Delegation also creates descendant work that must remain visible in the resource account. Reproducible evaluation therefore requires a specified configuration, and resource accounting includes the work of its descendants.
Definition 1. Evaluation configuration record [ftip-00CE]AGENTDRAFTED
Definition 1. Evaluation configuration record [ftip-00CE]AGENTDRAFTED
An evaluation configuration record is a finite record \(\mathfrak e\) that resolves the task and instance law, environment and tool interfaces, executable model artifact, serializer and inference policy, harness configuration, compaction and refinement policies, retry rule, completion gate, evaluator, seed law, and componentwise resource limits.
Together with the independent evaluation interface of Convention [ftip-005D], the record determines the law of a stopped evaluation run and its realized cost vector. A reported score is the pair consisting of the statistic and the exact record \(\mathfrak e\); a scalar without its record is an under-specified family of measurements.
Remark 2. A harness comparison needs a declared configuration axis [ftip-00CF]AGENTDRAFTED
Remark 2. A harness comparison needs a declared configuration axis [ftip-00CF]AGENTDRAFTED
Section 2.6 of [karten2026prime] binds task and tool interfaces to model settings. Its evaluation configuration also records compaction, refinement, retry, completion, and resource policies. The record in Definition 1 is a proposed representation of those configuration choices.
To attribute a score difference to a harness coordinate, the comparison must hold the other coordinates fixed or model their changes explicitly. Two scores under different records \(\mathfrak e\) and \(\mathfrak e'\) can still be descriptively useful, but their difference does not isolate a causal harness effect.
Definition 3. Session lineage tree [ftip-00CG]AGENTDRAFTED
Definition 3. Session lineage tree [ftip-00CG]AGENTDRAFTED
A session lineage tree is a finite rooted directed tree \(\mathcal T=(V,E,r)\) whose edges point from a parent session to a directly spawned descendant. Each node \(v\in V\) carries an immutable session identifier, its resolved harness and inference stamps, a local event stream, and a local realized cost \(c_v\in \mathbb R_+^m\).
For a node \(v\), let \(\mathcal T_v\) be the induced subtree containing \(v\) and all its descendants. Distinct children of one node have disjoint node sets. Messages between branches are events in their local streams and do not merge their identities or costs.
Definition 4. Descendant-complete cost [ftip-00CH]AGENTDRAFTED
Definition 4. Descendant-complete cost [ftip-00CH]AGENTDRAFTED
For a session lineage tree Definition 3, its descendant-complete cost is the componentwise sum
\[ C_{\rm lin}(\mathcal T)=\sum _{v\in V}c_v\in \mathbb R_+^m. \]The local cost \(c_v\) contains only work assigned to node \(v\); a model or tool call is charged to exactly one node. The vector retains its declared units and embeds into the lifecycle account of Definition [ftip-005H]. A scalar price can be applied only after the vector is formed.
Theorem 5. Lineage cost is additive over disjoint child subtrees [ftip-00CI]AGENTDRAFTED
Theorem 5. Lineage cost is additive over disjoint child subtrees [ftip-00CI]AGENTDRAFTED
Let the root \(r\) of a finite lineage tree have children \(v_1,\ldots ,v_k\). Then
\[ C_{\rm lin}(\mathcal T)=c_r+ \sum _{j=1}^{k}C_{\rm lin}(\mathcal T_{v_j}). \]This finite accounting identity follows from assigning each local cost to exactly one node.
Proof.
The node set is the disjoint union of \(\{r\}\) and the node sets of
the child subtrees. Splitting the finite sum in
Definition 4 over that disjoint union gives the equality componentwise.Proof.
Definition 6. Budget-admissible continuation [ftip-00CJ]AGENTDRAFTED
Definition 6. Budget-admissible continuation [ftip-00CJ]AGENTDRAFTED
Fix a componentwise hard budget \(b\in \mathbb R_+^m\). After accumulated cost \(a\preceq b\), a proposed continuation has a declared worst-case cost bound \(\bar c\in \mathbb R_+^m\). It is budget-admissible when
\[ a+\bar c\preceq b. \]If admitted, the continuation must either stop within a realized cost \(c\preceq \bar c\) or report a contract violation. Rejected proposals may have a separately accounted proposal cost, which must already be included in \(a\) before the admission test.
Theorem 7. Budget admission preserves a hard componentwise bound [ftip-00CK]AGENTDRAFTED
Theorem 7. Budget admission preserves a hard componentwise bound [ftip-00CK]AGENTDRAFTED
Consider a finite sequence of continuations. Start from \(a_0\preceq b\). At step \(i\), admit only if \(a_i+\bar c_i\preceq b\), and require the realized cost to satisfy \(c_i\preceq \bar c_i\). With \(a_{i+1}=a_i+c_i\), every accumulated cost satisfies \(a_i\preceq b\).
This finite result is a contract theorem, not a prediction that a real executor respects its declared bound.
Proof.
The claim holds for \(a_0\). If \(a_i\preceq b\) and the next step is
admitted, then
\(a_{i+1}=a_i+c_i\preceq a_i+\bar c_i\preceq b\). If the step is rejected,
the accumulated cost is unchanged after its already-accounted proposal work.
Finite induction proves the claim.Proof.
Example 8. A finite delegated run and its lineage cost [ftip-00CL]AGENTDRAFTED
Example 8. A finite delegated run and its lineage cost [ftip-00CL]AGENTDRAFTED
Use two cost coordinates, such as output tokens and tool calls. The root delegates to two children, and one child delegates once more.
The cost of a child subtree is its local cost plus the costs of all descendants. Therefore
\[ C_{\rm lin}(\mathcal T)=c_r+C_{\rm lin}(\mathcal T_a)+C_{\rm lin}(\mathcal T_b). \]Charging only the root hides descendant work; charging a descendant again as a separate parent-local cost double counts it.
Remark 9. External benchmark points do not isolate a harness effect [ftip-00CM]AGENTDRAFTED
Remark 9. External benchmark points do not isolate a harness effect [ftip-00CM]AGENTDRAFTED
Prime Agent Section 3.1 places native-harness runs beside externally reported ARC-AGI-3 results and explicitly says the external points situate the curves rather than isolate a causal harness effect [karten2026prime, Section 3.1]. That is an empirical comparison under multiple evaluation records, not a matched estimate of one harness field.
A larger score can still be a valid record of the displayed configuration. It does not by itself establish that persistence caused the difference, that the model weights improved, or that the cost-adjusted potential of Definition [ftip-005M] increased.