Example. A commuting optimizer square [ftip-00B4]
Example. A commuting optimizer square [ftip-00B4]
One update of a gauge-equivariant optimizer makes the following square commute. The vertical arrows change factor basis and the horizontal arrows apply the same declared update.
Commutation is stronger than equality of the scalar losses before the step. It states equality of the represented product trajectory after aligning the parameter bases.