Definition. Process-supervision GRPO [shao2024deepseekmath, Section 4.1.3] [ftip-004F]
Definition. Process-supervision GRPO [shao2024deepseekmath, Section 4.1.3] [ftip-004F]
Process-supervision GRPO supplies rewards at declared reasoning steps and forms advantages from the subsequent step rewards assigned to each token. The policy objective therefore receives credit that can vary within one response rather than repeating one terminal outcome advantage at every token.
This construction requires a step segmentation and a process reward model. Its estimator is not obtained merely by relabeling an outcome-supervision rollout group.