Histories, admissible actions, and retained information [ftip-00M4]
✍️sourceAGENTDRAFTED
Histories, admissible actions, and retained information [ftip-00M4]
✍️sourceAGENTDRAFTED
The controller may use its entire observed history. A smaller state can help compute a bound, but its sufficiency must be justified across the histories it represents. The examples below separate information that is valuable only in combination from information lost by a summary.
Definition 1. Finite controller class with hard resource admission [ftip-00M5]AGENTDRAFTED
Definition 1. Finite controller class with hard resource admission [ftip-00M5]AGENTDRAFTED
Fix a horizon \(H\in \mathbb N\), finite nonempty observed-history sets \(\mathcal H_t\) for \(0\leq t\leq H\), an initial law \(\mu \) on \(\mathcal H_0\), and finite nonempty permitted-action sets \(A_t(h)\) for \(t<H\). An environment kernel \(K_t(\cdot \mid h,a)\) is a probability law on histories extending \(h\) by action \(a\) and its observed outcome. The initial law and all kernels are fixed before choosing a controller.
A controller \(\pi \) assigns a probability law \(\pi _t(\cdot \mid h)\) on \(A_t(h)\). It may depend on every observed component of \(h\), including retained traces, generated programs, and previous updates. Random choices may be recorded in the history. The class \(\Pi (\mathbf B)\) consists of all such controllers under the declared hard resource vector \(\mathbf B\in \mathbb R_+^m\). It does not require a fixed prompt, a memoryless policy, or independent worker outputs. Full-history access is allowed in this mathematical class; it may enlarge the class of executable controllers with limited memory or computation.
Each history records accumulated nonnegative cost \(\mathbf c(h)\preceq \mathbf B\). Every permitted non-stop action has a declared worst-case increment \(\bar {\mathbf c}_t(h,a)\) satisfying
\[ \mathbf c(h)+\bar {\mathbf c}_t(h,a)\preceq \mathbf B. \]Every successor in the kernel's support must have realized increment between zero and this declared bound, componentwise. Proposal work, controller computation, worker calls, communication, verification, failed attempts, and persistent updates must be included in whichever resource coordinates are bounded; the units remain those of Definition [ftip-005H]. This is the enforced-contract assumption of Theorem [ftip-00CK].
A stop action is always permitted: it commits the current terminal artifact and pads the remaining steps with no further cost or change in its evaluated quality. At the horizon a fixed evaluator assigns \(q(h_H)\in [0,1]\) to the committed artifact recorded in the terminal history. If the evaluator has random outcomes, include them in the history law. Evaluation work must be charged before the zero-cost padding begins. Define
\[ J(\pi )=\mathbb E_\pi [q(h_H)], \qquad V(\mathbf B)=\sup _{\pi \in \Pi (\mathbf B)}J(\pi ). \]The artifact identity, evaluator, observations, kernels, and action sets are part of the mathematical problem. A training procedure or new tool belongs to this frontier only if it is among the permitted actions and its effects and costs are represented. Finiteness is an explicit restriction on histories, representations, and horizon; a theorem for this class does not bound an unrestricted agent that can extend them.
For an executable system, an upper bound applies only after its observations, actions, outcomes, and charged costs are represented by this model. Conversely, a mathematical policy supplies an executable lower bound only when it has an implementation respecting the stated resource cap; an arbitrary history-to-action table does not establish that fact.
Example 2. Complementary observations defeat one-step information value [ftip-00M6]AGENTDRAFTED
Example 2. Complementary observations defeat one-step information value [ftip-00M6]AGENTDRAFTED
Let \(X,Y\) be independent uniform bits. The terminal artifact is a guess for \(X\mathbin \oplus Y\); its quality is one if correct and zero otherwise. Two observation actions reveal \(X\) and \(Y\), respectively, at cost \(c\) each, where \(0<c<1/4\). The hard budget permits both observations. A terminal guess requires no additional observation cost.
Without observations, the best expected quality is \(1/2\). Given only \(X\), the unrevealed bit \(Y\) remains uniform, so the best expected quality is still \(1/2\); the same holds with \(X,Y\) exchanged. Thus a rule that compares stopping with taking one observation and then stopping assigns either observation net value \(1/2-c<1/2\). It stops immediately.
Observing both bits determines their parity. Its expected quality is one and its quality minus observation cost is \(1-2c>1/2\). Hence neither zero one-step information value nor a myopic stopping decision certifies the full controller frontier. This is a finite calculation, not an empirical claim about a language model. The scalar cost penalty is used only to exhibit the myopic decision; the hard-budget quality frontier itself is defined in Definition 1.
Example 3. Erasing an observed bit changes the attainable frontier [ftip-00M7]AGENTDRAFTED
Example 3. Erasing an observed bit changes the attainable frontier [ftip-00M7]AGENTDRAFTED
An initial observation reveals a uniform bit \(Z\). The terminal action is a bit \(a\), and quality is \(q=\mathbf 1\{a=Z\}\). Both actions are permitted at either history and have the same cost. A full-history controller chooses \(a=Z\) and attains quality one.
Now restrict the controller's entire input to a summary that is constant at the two initial histories. Its private randomness is independent of \(Z\). If it chooses one with probability \(p\), its expected quality is \(p/2+(1-p)/2=1/2\). No such summary controller can do better.
A model that averages the reward of either fixed action over the two histories obtains \(1/2\). This is the correct summary-controller value, but it underestimates the full-history frontier by \(1/2\). For the history \(Z=a\), the actual reward is one, so the averaged value fails as a reward upper bound at that history. A pointwise reward upper bound must hold at every represented history; agreement only under a tested policy's average history distribution does not establish such a certificate. The loss of an evaluator caveat in Example [ftip-00CU] illustrates why such retained distinctions can matter in a research harness.