Definition. Conditional next-token law [phuong2022formal, sec. 3, Sequence modelling] [ftip-000H]
✍️source

    Let \(\Theta \) be a parameter set. With the probability notation of Notation [ftip-000A] and the sequence convention of Convention [ftip-000G], a conditional next-token law is a family of maps \[ \pi _\theta :\mathcal V^*\longrightarrow \Delta (\mathcal V), \qquad \theta \in \Theta . \] For a prefix \(x_{<t}\) and token \(a\in \mathcal V\), the number \(\pi _\theta (a\mid x_{<t})\) is the probability assigned to choosing \(a\) at position \(t\).

    This is the observable probabilistic interface. It does not yet specify the neural computation that produces the distribution.