Both destabilized H100 training:
- Reward normalization (÷pos_frac): collapsed gradients at epoch 2-3
- Magnitude gradient restore (beta*MSE): Q-value explosion at epoch 25
(tested beta=0.10, 0.05, 0.02 — ALL cause Q explosion >50)
The magnitude MSE gradient is structurally unstable post-warmup.
ANY non-zero beta feeds a Q-overestimation loop that CQL can't counter.
The magnitude branch head learns during MSE warmup (epochs 1-5), then
the trunk continues improving via IQN (60% budget). This is the ONLY
stable configuration proven on H100 (RUN 4: stable through 9 epochs,
grad_norm=0.44, Q=2.03).
Reverts 76478559b (reward norm) and 22b7bc083 (gradient restore).
Keeps 1540c0287 (CUDA Graph fixes — the critical root cause).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
09fb80baa8