Definition. Ordered pretraining data pipeline [ftip-0017]
Definition. Ordered pretraining data pipeline [ftip-0017]
Let \(\mathcal C_{\mathrm {raw}}\in \mathcal M_{\rm doc}\) be a finite corpus drawn from the source mixture in Definition [ftip-0016]. Using the declared document filter, deduplication operator, and evaluation decontamination operator, an ordered pretraining data pipeline returns \[ \mathcal C_{\mathrm {clean}} =C_{\mathcal E}\left (D\left (F_{\#}(\mathcal C_{\mathrm {raw}})\right )\right ). \] Changing the order can change the retained corpus. The pipeline therefore records the order and exact revisions in addition to the three operators.