Remark. Preference acquisition and judge provenance [ftip-002N]
Remark. Preference acquisition and judge provenance [ftip-002N]
A preference observation depends on the query presented and the judge who evaluates it. Concrete elicitation procedures include [christiano2017deep, Sections 2.2.2 and 2.2.4] and the language-model comparison pipeline of [ouyang2022training, Sections 3.2 and 3.4].
The response generator, pair-selection rule, presentation order, judge population, and aggregation rule can each change the observed law. A later sample-complexity or population claim must condition on those choices rather than treating comparison labels as an unqualified source of ground truth.