Definition. Supervised fine-tuning objective [ftip-002F]
Definition. Supervised fine-tuning objective [ftip-002F]
For the language-model policy \(\pi _\theta \) and the notation of Notation [ftip-002A], a masked supervised fine-tuning (SFT) objective has the form
\[ L_{\mathrm {sft}}(\theta ) =-\sum _{z=(x,y,\omega )\in D_{\mathrm {sft}}} \sum _{t=1}^{|y|}w_t(z)m_t(z) \log \pi _\theta \left (y_t\mid s_{\mathrm {pr}}(x)\mathbin {\Vert }y_{<t}\right ), \]where the nonnegative weights satisfy \(\sum _{z\in D_{\mathrm {sft}}}\sum _{t=1}^{|y|}w_t(z)m_t(z)=1\). The response index \(t\) does not range over serialized prompt tokens. Minimizing this objective increases conditional likelihood on the selected target tokens. The weights and mask are local conventions and must be declared for each implementation.