Notation. Factored objective and represented matrix [ftip-00AW]
Notation. Factored objective and represented matrix [ftip-00AW]
Fix integers \(d_1,d_2,k\geq 1\). Let \(U\in \mathbb R^{d_1\times k}\) and \(V\in \mathbb R^{d_2\times k}\) be factors, let
\[ W(U,V)=UV^{\mathsf T}\in \mathbb R^{d_1\times d_2}, \]and let \(f:\mathbb R^{d_1\times d_2}\to \mathbb R\) be differentiable. The corresponding factored objective is \(L(U,V)=f(W(U,V))\). The pair \((U,V)\) is a parameterization; \(W(U,V)\) is the represented object on which \(f\) depends. This is the rectangular form noted in Section 3 of The loss does not see the basis, but Adam does[singh2026lossbasis].