Remark. Window sampling, document masks, and packing [ftip-006K]
AGENTDRAFTED
[kaplan2020scaling], Section 2.2, uses fixed-length sequence batches.
[phuong2022formal], Section 7, gives the causal language-model
objective.
The proposed data interface additionally specifies the sampler, segment
identifiers, document mask, and prediction mask. Those boundary choices are
not determined by the high-level training accounts.
These objects describe one admissible data interface. A concrete training
run must replace them with its offset, padding, joining, masking, and packing
rules.