Corollary. Fixed nonlinear coordinatewise updates break the gauge [ftip-00BC]
Corollary. Fixed nonlinear coordinatewise updates break the gauge [ftip-00BC]
A nonlinear fixed entrywise update applied at a zero optimizer state is not gauge-equivariant when \(k\geq 2\). In particular, the source applies proposition [ftip-00BB] to the first-step maps of Adam, RMSProp, signSGD, and Lion; it checks Adafactor separately because its factored statistics are not an entrywise map.
This is a first-step obstruction. It suffices to disprove equivariance of the full run, but it does not classify every later state or say that such an optimizer cannot reach a particular solution.