Example. Cyclic preferences that no scalar ordering represents [ftip-002V]
Example. Cyclic preferences that no scalar ordering represents [ftip-002V]
Three strict comparisons arranged in a directed cycle obstruct representation by a single scalar ordering.
Declare \(A\succ B\), \(B\succ C\), and \(C\succ A\). If a scalar \(r\) represented all three comparisons by strict inequalities, then \[ r(A)>r(B)>r(C)>r(A), \] which implies \(r(A)>r(A)\), a contradiction. Thus no real-valued score can represent this cycle by ordinary greater-than.
Scalar pairwise reward modeling is used in [christiano2017deep, Section 2]; the finite cycle marks one assumption needed for that reduction. No empirical cycle is attributed to a particular dataset or population, and the obstruction concerns scalar orderings rather than all preference models.