Example. A three-symbol autoregressive continuation [ftip-000J]
Example. A three-symbol autoregressive continuation [ftip-000J]
Along one branch of a three-symbol prefix tree, conditional token laws determine both continuation probability and negative log-likelihood.
Let the vocabulary be \(\{a,b,\mathtt {EOS}\}\) and let \(p_i\) be the displayed conditional probability at step \(i\). The continuation \((b,a,\mathtt {EOS})\) has \[ \Pr (b,a,\mathtt {EOS}\mid a) =p_1p_2p_3, \qquad -\log \Pr =-\sum _{i=1}^{3}\log p_i. \]
The multiplication follows the causal product in [phuong2022formal, Section 3]. Its scope is one fixed finite continuation, leaving both the decoding algorithm and the law over all variable-length outputs unspecified.