Theorem. Finite exponential tilting preserves support [ftip-007C]
✍️source

    Fix a prompt \(x\), a finite response set \(\mathcal Y\), a reference policy \(\pi _{\mathrm {ref}}\), a finite real reward \(r(x,y)\), and \(\beta >0\). Let \(\pi _r\) be the normalized exponential tilt of Definition [ftip-003O]. Then, for every \(y\in \mathcal Y\),

    \[ \pi _r(y\mid x)>0 \quad \Longleftrightarrow \quad \pi _{\mathrm {ref}}(y\mid x)>0. \]

    Proof. 

      The multiplier \(\exp (r(x,y)/\beta )\) is finite and strictly positive. The finite normalizer \(Z_r(x)\) is also strictly positive. Multiplication by the first quantity and division by the second therefore preserve whether the reference mass is zero or positive.

      The source policy form appears as equation (4) in [rafailov2023direct, Section 4]; the displayed result is the finite corollary derived here. Infinite rewards, non-normalizable response spaces, approximate optimization, and changes to the generation mechanism lie outside the statement.