Definition. Context-window sampler [ftip-0019]
Definition. Context-window sampler [ftip-0019]
Fix the context length \(T\) from Notation [ftip-0015]. A context-window sampler draws cleaned text from the law described in Remark [ftip-0018] and tokenizes it with Definition [ftip-000E]. It then chooses a declared offset and returns a length-\(T\) token sequence \(x_{1:T}\) together with metadata that identifies document boundaries.
If fewer than \(T\) tokens remain, the sampler must specify whether it pads, drops the suffix, or joins another document. These choices change both the distribution of contexts and the prediction mask.