proposition. Gradient descent and momentum preserve the orthogonal gauge [ftip-00B8]
✍️source

Gradient descent and Polyak or Nesterov momentum are gauge-equivariant when their factor gradients, momentum buffers, schedules, and look-ahead points are transformed consistently.

Proof.

By Lemma [ftip-00B1], \(G_t\) becomes \(G_tQ\). Gradient descent therefore sends \(U_tQ\) to \((U_t-\eta _tG_t)Q\). A momentum buffer initialized at zero and formed by linear combinations of covariant gradients transforms as \(M_tQ\); a Nesterov look-ahead point is likewise the reference look-ahead right-multiplied by \(Q\). Induction proves the claim.

Proposition 4.2(1) and its proof are in Appendix B.3 of The loss does not see the basis, but Adam does[singh2026lossbasis].

proposition. Gradient descent and momentum preserve the orthogonal gauge [ftip-00B8]
✍️source

Proof.

Reference. The loss does not see the basis, but Adam does [singh2026lossbasis]

Equivariant and basis-sensitive update rules [ftip-00B6]

Definition. Memoryless right-equivariant update [ftip-00B7]

proposition. Gradient descent and momentum preserve the orthogonal gauge [ftip-00B8]
✍️source

Proof.

proposition. A shared scalar second moment preserves the orthogonal gauge [ftip-00B9]

Proof.

proposition. Full-matrix Muon and Shampoo preserve the orthogonal gauge [ftip-00BA]

Proof.

proposition. Coordinatewise equivariance forces linearity [ftip-00BB]

Proof.

Corollary. Fixed nonlinear coordinatewise updates break the gauge [ftip-00BC]

Definition. First-step Adam map and gauge defect [ftip-00BD]

proposition. Exact represented-matrix defect after one Adam step [ftip-00BE]

Proof.

Example. A two-coordinate Adam defect witness [ftip-00BF]

Example. A rotated sign update fails to commute [ftip-00BG]

Theorem. Full-rank memoryless equivariant rules are Gram-determined [ftip-00BH]

Proof.

Corollary. The square invertible preconditioner representation [ftip-00BI]

Proof.

Remark. What the structure theorem leaves unresolved at rank deficiency [ftip-00BJ]

Lemma. Gradients transform covariantly along an orthogonal orbit [ftip-00B1]

Proof.