Example. A finite agent--environment interaction loop [ftip-0026]
Example. A finite agent--environment interaction loop [ftip-0026]
A deterministic three-action episode records each observation before the next policy decision and ends with an explicit terminal observation.
Let \(H_0=(\text {door locked})\). After action \(A_0=\text {take key}\), the environment returns \(O_1=\text {key held}\), so \(H_1=(H_0,A_0,O_1)\). Repeating the same update with unlock and door open, then choosing \(A_2=\text {enter}\), produces \(O_3=\text {terminal success}\) and \(H_3=(H_2,A_2,O_3)\). No later observation is available to an earlier action choice.
The alternating action--observation interface is the finite-history form of the agent setup in [kaelbling1998planning, Section 2]. The trace checks chronology and termination; uncertain state, partial observability, and language-valued actions lie beyond this deterministic instance.