Definition. Token negative log-likelihood [ftip-001B]

✍️sourceAGENTDRAFTED

Let \(\zeta =(x,m,c,M^{\mathrm {doc}})\) be a packed training record from Definition [ftip-001A]. Write \(\pi _{\theta ,M^{\mathrm {doc}}(\zeta )}(\,\cdot \mid x_{<t}(\zeta ))\) for the next-token law of Definition [ftip-000H] when every decoder attention layer uses the declared document mask. For a predicted position \(t\geq 2\), its masked token negative log-likelihood is \[ \ell _t(\theta ;\zeta ) =-m_t(\zeta )\log \pi _{\theta ,M^{\mathrm {doc}}(\zeta )} \left (x_t(\zeta )\mid x_{<t}(\zeta )\right ), \] A masked position has zero contribution; an unmasked position penalizes the log probability computed under the same segment boundaries that defined the packed example.