Definition. optimizer update support and effective rank [ftip-00OX]
Definition. optimizer update support and effective rank [ftip-00OX]
The source question is whether reinforcement-learning updates need the momentum and per-parameter adaptive rates supplied by AdamW. Mukherjee et al. compare SGD, SGD with momentum, RMSProp, and AdamW in GRPO and PPO settings across mathematical, coding, and adaptive-verifiable environments ([mukherjee2026adam, Sections 4-6 and conclusion]). In the reported runs, SGD matches or often exceeds AdamW, while its updates are substantially sparser and lower-rank; the source reports fewer than \(0.02\) percent changed parameters in representative cases and a 15.7 GB peak-memory reduction for Qwen3-1.7B. These are observations about the stated models, data, precision, threshold, seeds, and budgets.
The relevant FTIP intervention is therefore a tuple, not an optimizer name: model and checkpoint, RL algorithm, prompt and environment laws, optimizer and hyperparameters, numerical precision, update-support rule, effective-rank rule, training and evaluation budgets, and the held-out capability interface. A reported reward or pass rate does not identify the update geometry that produced it.
Let \(\theta _0\) and \(\theta _1\) be the parameters before and after a declared finite intervention, with \(n\) trainable scalar parameters. For threshold \(\varepsilon >0\), define \[ S_{\varepsilon }(\theta _0,\theta _1)= \frac {1}{n}\#\{i:|\theta _{1,i}-\theta _{0,i}|>\varepsilon \}, \]and, for every two-dimensional weight update \(\Delta W\), let \(\operatorname {rank}_{.99}(\Delta W)\) be the least number of singular values whose squared sum reaches 99 percent of the total squared singular-value sum. A study must report the threshold, precision, tensor selection, and aggregation rule alongside these numbers. The source uses bf16 and a \(10^{-5}\) threshold for its sparsity calculation; that convention is a source field, not a universal definition.
A matched Phase B comparison holds model family, checkpoint, training data and order, token and environment steps, rollout limits, precision, gradient accumulation, seeds, evaluation prompts, and stopping rule fixed while changing SGD or AdamW. Learning-rate and optimizer tuning must use the same predeclared trial count, search ranges, selection data, and tuning seeds, with tuning tokens, steps, wall-clock, and energy charged to both arms. The protocol also declares an optimizer-state memory cap (or explicitly reports no cap) and records persistent-state bytes, peak memory, wall-clock and energy, update support, effective rank, verifier/evaluator cost, and a prespecified held-out transfer task. Capability change and support change are reported as separate outcomes; a lower-rank update is not itself an acquisition witness.
The source does not establish general SGD superiority, a universal low-rank plasticity law, or transfer to long-horizon agents. Deep Delta's learned read/compare/write mechanism changes a different coordinate and remains in the later recurrence/rewrite batch.