Definition. Optimizer state action [ftip-00B2]
Definition. Optimizer state action [ftip-00B2]
Use the optimizer-state convention of Definition [ftip-001H]. For every \(Q\in \mathrm O(k)\), an optimizer state action is a map \(\sigma _Q:\mathcal S_{\rm opt}\to \mathcal S_{\rm opt}\) compatible with the shapes of the factor states and satisfying \(\sigma _Q(s_0)=s_0\) for the declared initial state.
For example, a momentum buffer shaped like \(U\) transforms by right multiplication by \(Q\); a shared scalar second moment is fixed. An entrywise second-moment array need not admit a state action compatible with every orthogonal \(Q\).