Definition. Task law [ftip-001Q]
Definition. Task law [ftip-001Q]
For the instance set in Definition [ftip-001O], a task law is a probability law \(\mu _{\mathsf T}\) on \(\mathcal X_{\mathsf T}\). An evaluation sample of size \(N\) is a declared joint law of instances \((X_1,\ldots ,X_N)\) whose marginals are \(\mu _{\mathsf T}\); independent sampling is a further assumption, not part of the term ``task law.''
In an episodic control problem this law includes the distribution of starting situations. In a fixed benchmark it may instead be the uniform law on a held-out finite set. Reweighting instances changes the task law even when the instance files are unchanged.