Notation. Comparison queries, judges, and response pairs [ftip-002K]
Notation. Comparison queries, judges, and response pairs [ftip-002K]
Use the prompt and response spaces from Notation [ftip-002A]. A comparison query is a displayed triple \(c=(x,y^0,y^1)\) with \(y^0,y^1\in \mathcal Y(x)\). Let \(\mathcal J\) be the declared judge population, and let \(j\in \mathcal J\) identify the human, model, or rule that returns a comparison label \(b_{\mathrm {pref}}\in \{0,1\}\).
When \(b_{\mathrm {pref}}=1\), write \(y^+=y^1\) and \(y^-=y^0\); when \(b_{\mathrm {pref}}=0\), reverse those names. Thus \(y^+\) means selected in this observation, not objectively or uniquely best. The acquisition mechanism may also record a tie, abstention, or invalid comparison, but those outcomes must use an enlarged label space rather than being silently forced into \(\{0,1\}\).