Example. Cyclic preferences that no scalar ordering represents [ftip-002V]
AGENTDRAFTED
Three strict comparisons arranged in a directed cycle obstruct
representation by a single scalar ordering.
Declare \(A\succ B\), \(B\succ C\), and \(C\succ A\). If a scalar
\(r\) represented all three comparisons by strict inequalities, then
\[
r(A)>r(B)>r(C)>r(A),
\]
which implies \(r(A)>r(A)\), a contradiction. Thus no real-valued score can
represent this cycle by ordinary greater-than.
Scalar pairwise reward modeling is used in
[christiano2017deep, Section 2]; the finite cycle marks one assumption
needed for that reduction. No empirical cycle is attributed to a particular
dataset or population, and the obstruction concerns scalar orderings rather
than all preference models.