Remark. Reward density is not cost-free smoothing [ftip-00AJ]
✍️source

A denser reward can distinguish more responses and supply more frequent update signal. It can also require information absent from a sparse verifier. In the controlled source, edit feedback assumes the full target and computes a string metric; process feedback uses a 32-billion-parameter judge, five problem-specific milestones, answer extraction, an enumerated near miss, and loop and format penalties [clay2026demystifying, Appendices C--D].

The richer feedback channel incurs specification, computation, and validation costs. Replacing sparse feedback by dense feedback can alter both the information available to training and the objective being optimized. The resulting comparison is an intervention on feedback resolution, not evidence that one fixed reward was smoothed at zero cost.

Remark. Reward density is not cost-free smoothing [ftip-00AJ]
✍️source

Reference. Demystifying Reinforcement Learning Post-Training of Language Models [clay2026demystifying]

Sparse, dense, and process feedback [ftip-00A8]

Definition. Sparse exact reward [ftip-00A9]

Remark. Main-text and Appendix-C sparse-reward discrepancy [ftip-00AA]

Definition. Edit-distance reward [ftip-00AB]

Remark. Dense reward imports target structure [ftip-00AC]

Definition. Milestone process reward [ftip-00AD]

Remark. Process feedback assumptions and milestone-weight discrepancy [ftip-00AE]

Definition. Reward-induced observational equivalence [ftip-00AF]

Lemma. Refining feedback separates at least as many responses [ftip-00AG]

Proof.

Example. Counterexample: equal sparse reward, unequal dense reward [ftip-00AH]

Example. Three feedback resolutions on one response set [ftip-00AI]

Remark. Reward density is not cost-free smoothing [ftip-00AJ]
✍️source