Files
foxhunt/docs
jgrusewski 207fce8778 docs(policy-quality): Track 2 reward audit (Phase 1) — 2 DELETE, 3 KEEP
V7 audit of reward terms R1-R8 against the baseline smoke run
(magnitude_distribution.rs, 3 folds x 20 epochs = 60 HEALTH_DIAG rows):

* R1 step_return: KEEP (base reward, denominator)
* R2 PopArt drift: PENDING (warmup-gated zero at smoke; re-check at L40S)
* R3 CF-flip: KEEP (49-74% contribution, dominates shaping)
* R4 trail_r: KEEP-WITH-CAVEAT (fold-3 dominance, trade-volume gated)
* R5 micro-reward: DELETE (micro_reward_scale=0 in smoke, intended)
* R6 loss-aversion: DELETE (sub-1% in 54/60 epochs, relocate protection)
* R7 segment-patience: ALREADY-REMOVED (stub deleted 83d524f86)
* R8 raw_returns_out: KEEP (load-bearing for Sharpe/MaxDD pipeline)

Cross-term finding: R3 is the dominant reward shaper at smoke scale;
every other shaping term is <=3% of reward magnitude except R4 in fold
3. Confirms Track 1's H4 gradient-starvation diagnosis is compounded
by thin shaping elsewhere in the reward landscape.

Follow-ups flagged for Phase 2:
- stale patience_mult docstring at experience_kernels.cu:1049
- R5 deletion gated on TD-propagation diagnostic (Task #8)
- R6 relocation to Q-target smoothing in C51 Bellman
2026-04-22 09:06:50 +02:00
..