Remark. The induced law of a stopped interaction [ftip-0028]
AGENTDRAFTED
Sutton and Barto's agent--environment interface in
[sutton2018reinforcement, sec. 3.1] supplies alternating states, actions,
and rewards. Kaelbling et al. separate hidden state and public observation in
[kaelbling1998planning, Section 2]. A stopped interaction is determined
by its initial law, transition--observation kernel, policy, and public
stopping rule.
The induced interaction law uses four independently declared objects:
the initial law \(\rho _0\) in Definition [ftip-001Y], the environment kernels
\(K_t\) in Definition [ftip-001Z], the policy \(\pi _t\) in Definition [ftip-0023], and the
stopping rule in Definition [ftip-0024]. On finite spaces, the probability of a
length-\(n\) trajectory before applying its stopping indicator factors as
\[
\rho _0(s_0,o_0)
\prod _{t=0}^{n-1}
\pi _t(a_t\mid h_t)
K_t(s_{t+1},o_{t+1}\mid s_t,a_t).
\]
The event \(\{\tau _{\mathrm {stop}}=n\}\) selects the stopped trajectories.
This factorization supports later expectations, likelihood
ratios, and off-policy reuse. It also exposes an identifiability limit: an
outcome distribution by itself generally does not reveal which of the initial
law, environment, policy, or stopping rule changed.