PopArt: normalize_rewards_popart_inplace was implemented but never
called from run_full_step. Without it, rank-normalized rewards [-1,+1]
spread across C51 atom range [-50,+50] — only 2% of atoms used,
near-zero C51 gradient. Now called before forward pass (Phase 0a).
Counterfactual: when do_flip=true AND cf_cycle==0 (directional mirror),
cf_reward = -reward double-negated (reward already flipped at line 1895).
Fix: cf_reward = do_flip ? reward : -reward. Affects ~1/6 of
counterfactual experiences that were teaching wrong directional signal.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
4f6b4540c5