Four improvements to the training pipeline:
1. Asymmetric spread gradient: challenger action (adjacent to taken) gets
pushed UP at half strength instead of DOWN. Creates a two-horse race
instead of single-action monopoly.
2. Per-branch spread scaling: spread_grad *= branch_scale. Direction branch
(high impact) gets more spread than urgency (low impact).
3. Distributional variance position sizing (Layer 4): Var[Q] = E[Z²] - E[Z]²
computed in compute_expected_q. Position scaled by 1/(1+sqrt(Var[Q_taken])).
High uncertainty → smaller position. Kelly criterion from C51 atoms.
4. Reward std guard: skip rank normalization when observed_reward_std ≈ 0
(epoch 0). Prevents zeroing all rewards before std is observed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
b2eb1b7741