Files
foxhunt/docs/superpowers
jgrusewski 0fe825a8c5 fix(rl): decouple C51 atom span from reward-clamp ceiling
Spec: docs/superpowers/specs/2026-05-30-c51-atom-span-decouple-from-clamp.md

Diagnosed in cluster v5 (alpha-rl-rjsjq SHA 7064c9269) at step 371:
c51_v_max ratcheted up 1 → 24 → 53 → 859 → 1938 while WIN clamp swung
4910 → 568 → 7540 → 44 (volatile from MARGIN saturation). The slow
EWMA tracking win_bound time-averaged the early spikes, locking
atom span at extreme-value magnitude. Δz reached ~184 per atom — Q
could no longer discriminate +$500 from +$5000 wins.

Root cause: atom-span target was anchored on `MARGIN × pos_max_ema`,
where pos_max is the EXTREME-VALUE statistic across b=1024 batch
elements. The clamp bound (rightly fat-tail) and the atom span (should
be typical-magnitude) were treating two distinct concerns as one.

Fix F: anchor atom span on `atom_margin × mean_abs_pnl_ema` — true
central-tendency signal — while leaving the reward clamp on
pos_max_ema. The clamp can still admit fat-tail wins ($42k → +245)
and V regression (scalar head) retains the magnitude for PPO advantage,
but Q's CATEGORICAL distributional support now sizes itself for the
trades where most learning happens.

New ISV slot 685 (RL_C51_ATOM_MARGIN_INDEX, bootstrap 10.0).
RL_SLOTS_END 685 → 686.

Validation (local b=128 1k smoke):
  metric          | pre-Fix-F      | with Fix F
  v_max final      | 1929           | 86.6        (22× tighter)
  Δz per atom      | 184            | 8.7         (resolution restored)
  qpa final        | +0.957         | +0.958      (Q-π alignment preserved)
  mean_active      | +$47k          | +$34k       (within smoke variance)

13/13 risk_stack_invariants + 20/20 trade_management_kernels +
integrated_trainer_smoke pass.

Tradeoff documented in spec: Q loses fat-tail categorical magnitude
(rewards beyond ±87 project to top/bottom atom), but V regression
preserves it. Acceptable for the lottery-ticket strategy where typical-
magnitude resolution matters more than fat-tail magnitude precision.
2026-05-30 23:52:43 +02:00
..