Definition. Process-supervised reward model [lightman2023verify, Section 2.6, ``Process-supervised reward models''] [ftip-003X]
Definition. Process-supervised reward model [lightman2023verify, Section 2.6, ``Process-supervised reward models''] [ftip-003X]
Suppose a response is segmented into reasoning steps \(y=(s_1,\ldots ,s_m)\). A process-supervised reward model produces a score for each declared prefix,
\[ r_{\mathrm {proc},\phi }(x,s_{\leq k})\in \mathbb R, \qquad 1\leq k\leq m, \]and is fitted from step-level labels. The segmentation rule and label semantics are part of the supervision: a score after step \(k\) need not be a decomposition of an outcome score for the whole response.