Definition. Response-level episode [ouyang2022training, Section 3.5] [ftip-002Y]
Definition. Response-level episode [ouyang2022training, Section 3.5] [ftip-002Y]
In the response-level specialization used for language-model RLHF, a prompt \(x\) is the initial context, a complete response \(y\sim \pi _\theta (\mathord \cdot \mid x)\) is the sampled action, and a scalar score supplies the terminal reward. The episode terminates after that response.
This contextual-bandit view is sufficient for sequence-level reward-model training. Token generation may still be exposed as multiple policy decisions when an optimizer assigns token-level likelihood ratios or advantages. A later agentic setting also permits environment observations and tool actions between model responses.