Notation. State, observation, action, time, history, and stopping [ftip-001X]

✍️sourceAGENTDRAFTED

Let \(\mathcal S,\mathcal O,\mathcal A\) be declared measurable spaces of environment states, public observations, and agent actions. Time is discrete, \(t\in \{0,1,\ldots ,T_{\max }\}\), where \(T_{\max }<\infty \) is a hard horizon. Random states, observations, and actions are \(S_t,O_t,A_t\); lowercase \(s_t,o_t,a_t\) denote realized values. The same convention makes \(h_t\) a realization of \(H_t\) and \(z\) a realization of \(Z\).

The public history available before action \(A_t\) is \(H_t\); the full environment trajectory is \(Z\); and the stopping time is \(\tau _{\mathrm {stop}}\in \{0,1,\ldots ,T_{\max }\}\). Probability kernels are written \(K(dy\mid x)\); on finite spaces this means the probability law \(K(y\mid x)\) and reduces to the notation of Notation [ftip-000A].