Remark. Dense reward imports target structure [ftip-00AC]
✍️source

The edit reward of Definition [ftip-00AB] is not obtained from the binary value in Definition [ftip-00A9] by a numerical smoothing operation. It also receives the target string, a character-level edit model, and the ordering of symbols. Two non-target responses that both receive sparse reward zero can therefore receive different edit rewards.

That extra resolution can improve credit assignment, but it changes the feedback channel and its assumptions. The source calls edit distance a dense proxy and reports controlled movie-quote experiments [clay2026demystifying, Sections 4.2 and 5.2]. Neither the formula nor those observations establish that edit proximity is an independent measure of general response quality.

Remark. Dense reward imports target structure [ftip-00AC]
✍️source

Reference. Demystifying Reinforcement Learning Post-Training of Language Models [clay2026demystifying]

Sparse, dense, and process feedback [ftip-00A8]

Definition. Sparse exact reward [ftip-00A9]

Remark. Main-text and Appendix-C sparse-reward discrepancy [ftip-00AA]

Definition. Edit-distance reward [ftip-00AB]

Remark. Dense reward imports target structure [ftip-00AC]
✍️source

Definition. Milestone process reward [ftip-00AD]

Remark. Process feedback assumptions and milestone-weight discrepancy [ftip-00AE]

Definition. Reward-induced observational equivalence [ftip-00AF]

Lemma. Refining feedback separates at least as many responses [ftip-00AG]

Proof.

Example. Counterexample: equal sparse reward, unequal dense reward [ftip-00AH]

Example. Three feedback resolutions on one response set [ftip-00AI]

Remark. Reward density is not cost-free smoothing [ftip-00AJ]

Definition. Edit-distance reward [ftip-00AB]

Definition. Sparse exact reward [ftip-00A9]