Definition. One agentic post-training round [ftip-0073]
✍️source

    At round \(n\), fix a task \(\mathsf T_n\), its law \(\mu _{\mathsf T_n}\), and a versioned interactive realization \(\mathsf {Env}^{\upsilon _n}_{\mathsf T_n}\). Let \(\mathcal I_{\rm art}\) and \(\mathcal I_{\rm inf}\) be declared identifier spaces. Let \(a_n\in \mathcal I_{\rm art}\) be an immutable content identifier resolving the artifact \(M_n\), and let \(\iota _n\in \mathcal I_{\rm inf}\) record all serializer, decoding, tool, and stopping settings. The policy--inference stamp \(\nu _n=(a_n,\iota _n)\in \mathcal I_{\rm art}\times \mathcal I_{\rm inf}\) resolves a non-anticipating policy \(\pi _n\). Let \(\Omega _n^{\rm task}\), \(\Omega _n^{\rm roll}\), \(\Omega _n^{\rm feed}\), \(\Omega _n^{\rm credit}\), and \(\Omega _n^{\rm update}\) be declared seed spaces. The rollout seed contains both policy-decoding and environment randomness. Write \(\Omega _n=\Omega _n^{\rm task}\times \Omega _n^{\rm roll}\times \Omega _n^{\rm feed}\times \Omega _n^{\rm credit}\times \Omega _n^{\rm update}\), and let \(\omega _n=(\omega _n^{\rm task},\omega _n^{\rm roll}, \omega _n^{\rm feed},\omega _n^{\rm credit},\omega _n^{\rm update})\) be sampled from a declared joint law on \(\Omega _n\). Write \(\mathcal Z_{\mathsf T_n}\) for the stopped trajectories admitted by the versioned realization.

    For declared feedback and credit spaces \(\mathcal F_n\) and \(\mathcal G_n\), decision space \(\mathcal D_n=\{\mathsf {accept},\mathsf {reject}\}\), accounting dimension \(m\geq 1\), and exact implementation versions \(v_n^{\rm S}\), \(v_n^{\rm R}\), \(v_n^{\rm F}\), \(v_n^{\rm C}\), \(v_n^{\rm U}\), and \(v_n^{\rm A}\), the round specifies maps

    \[ \begin {aligned} \mathsf S_n^{v_n^{\rm S}}&: \Omega _n^{\rm task}\longrightarrow \mathcal X_{\mathsf T_n},\\ \mathsf R_n^{v_n^{\rm R}}&: \mathcal X_{\mathsf T_n}\times \Omega _n^{\rm roll} \longrightarrow \mathcal Z_{\mathsf T_n},\\ \mathsf F_n^{v_n^{\rm F}}&: \mathcal X_{\mathsf T_n}\times \mathcal Z_{\mathsf T_n} \times \Omega _n^{\rm feed}\longrightarrow \mathcal F_n,\\ \mathsf C_n^{v_n^{\rm C}}&: \mathcal X_{\mathsf T_n}\times \mathcal Z_{\mathsf T_n} \times \mathcal F_n\times \Omega _n^{\rm credit} \longrightarrow \mathcal G_n,\\ \mathsf U_n^{v_n^{\rm U}}&: \mathcal M_{\rm exec}\times \mathcal Z_{\rm opt}\times \mathcal G_n \times \Omega _n^{\rm update} \longrightarrow \mathcal D_n\times \mathcal M_{\rm exec}\times \mathcal Z_{\rm opt},\\ \mathsf A_n^{v_n^{\rm A}}&: \mathcal M_{\rm exec}\times \mathcal Z_{\rm opt} \times \mathcal X_{\mathsf T_n}\times \mathcal Z_{\mathsf T_n}\\ &\quad {}\times \mathcal F_n\times \mathcal G_n\times \mathcal D_n\\ &\quad {}\times \mathcal M_{\rm exec}\times \mathcal Z_{\rm opt} \times \Omega _n\longrightarrow \mathbb R_+^m. \end {aligned} \]

    The accounting map receives the starting state, proposed state, decision, and realized seeds, so update work and rejected proposals remain observable in the cost record.

    They produce

    \[ \begin {aligned} X_n&=\mathsf S_n^{v_n^{\rm S}}(\omega _n^{\rm task}),\\ Z_n&=\mathsf R_n^{v_n^{\rm R}}(X_n,\omega _n^{\rm roll}),\\ F_n&=\mathsf F_n^{v_n^{\rm F}}(X_n,Z_n,\omega _n^{\rm feed}),\\ \Gamma _n&=\mathsf C_n^{v_n^{\rm C}} (X_n,Z_n,F_n,\omega _n^{\rm credit}),\\ (d_n,\widetilde M_{n+1},\widetilde s_{n+1}) &=\mathsf U_n^{v_n^{\rm U}} (M_n,s_n,\Gamma _n,\omega _n^{\rm update}),\\ c_n&=\mathsf A_n^{v_n^{\rm A}}( M_n,s_n,X_n,Z_n,F_n,\Gamma _n,d_n, \widetilde M_{n+1},\widetilde s_{n+1},\omega _n). \end {aligned} \]

    The task sampler pushes its seed law forward to \(\mu _{\mathsf T_n}\). Conditional on \(X_n\), the rollout sampler pushes its seed law forward to the interaction law of Remark [ftip-0028] under \(\mathsf {Env}^{\upsilon _n}_{\mathsf T_n}\) and \(\pi _n\).

    If \(d_n=\mathsf {accept}\), set \((M_{n+1},s_{n+1})=(\widetilde M_{n+1},\widetilde s_{n+1})\); if \(d_n=\mathsf {reject}\), set \((M_{n+1},s_{n+1})=(M_n,s_n)\). The realized cost \(c_n\) includes proposal work even after rejection. With \(v_n^{\rm ops}=(v_n^{\rm S},v_n^{\rm R},v_n^{\rm F},v_n^{\rm C}, v_n^{\rm U},v_n^{\rm A})\), append

    \[ \begin {aligned} r_n={}&(\mathsf T_n,X_n,\nu _n,\upsilon _n,Z_n,F_n,\Gamma _n,d_n, \widetilde M_{n+1},\widetilde s_{n+1},M_{n+1},s_{n+1}, \omega _n,v_n^{\rm ops},c_n),\\ \mathcal L_{n+1}={}&\operatorname {append}(\mathcal L_n,r_n). \end {aligned} \]

    The record belongs to \(\mathcal R_{\rm ag}\) from Definition [ftip-004U] and determines \(\Sigma _{n+1}\) without rewriting an earlier record.