jgrusewski
207fce8778
docs(policy-quality): Track 2 reward audit (Phase 1) — 2 DELETE, 3 KEEP
V7 audit of reward terms R1-R8 against the baseline smoke run
(magnitude_distribution.rs, 3 folds x 20 epochs = 60 HEALTH_DIAG rows):
* R1 step_return: KEEP (base reward, denominator)
* R2 PopArt drift: PENDING (warmup-gated zero at smoke; re-check at L40S)
* R3 CF-flip: KEEP (49-74% contribution, dominates shaping)
* R4 trail_r: KEEP-WITH-CAVEAT (fold-3 dominance, trade-volume gated)
* R5 micro-reward: DELETE (micro_reward_scale=0 in smoke, intended)
* R6 loss-aversion: DELETE (sub-1% in 54/60 epochs, relocate protection)
* R7 segment-patience: ALREADY-REMOVED (stub deleted 83d524f86)
* R8 raw_returns_out: KEEP (load-bearing for Sharpe/MaxDD pipeline)
Cross-term finding: R3 is the dominant reward shaper at smoke scale;
every other shaping term is <=3% of reward magnitude except R4 in fold
3. Confirms Track 1's H4 gradient-starvation diagnosis is compounded
by thin shaping elsewhere in the reward landscape.
Follow-ups flagged for Phase 2:
- stale patience_mult docstring at experience_kernels.cu:1049
- R5 deletion gated on TD-propagation diagnostic (Task #8)
- R6 relocation to Q-target smoothing in C51 Bellman
2026-04-22 09:06:50 +02:00
..
2026-04-01 21:36:40 +02:00
2026-04-04 11:49:13 +02:00
2026-04-04 11:07:47 +02:00
2026-04-04 09:28:42 +02:00
2026-04-04 00:57:49 +02:00
2026-04-04 23:00:16 +02:00
2026-04-04 17:30:18 +02:00
2026-04-05 21:25:05 +02:00
2026-04-06 22:48:07 +02:00
2026-04-06 11:15:24 +02:00
2026-04-07 13:58:11 +02:00
2026-04-06 11:15:24 +02:00
2026-04-07 13:43:04 +02:00
2026-04-08 01:26:09 +02:00
2026-04-07 23:14:40 +02:00
2026-04-07 22:21:11 +02:00
2026-04-08 20:05:22 +02:00
2026-04-08 01:49:01 +02:00
2026-04-08 08:48:36 +02:00
2026-04-08 12:04:23 +02:00
2026-04-08 22:23:19 +02:00
2026-04-08 20:26:40 +02:00
2026-04-10 20:26:55 +02:00
2026-04-10 22:16:24 +02:00
2026-04-11 11:23:23 +02:00
2026-04-11 13:30:12 +02:00
2026-04-13 09:26:14 +02:00
2026-04-13 14:51:30 +02:00
2026-04-13 20:13:39 +02:00
2026-04-13 11:29:05 +02:00
2026-04-14 21:40:03 +02:00
2026-04-14 23:46:27 +02:00
2026-04-14 22:14:49 +02:00
2026-04-15 20:46:05 +02:00
2026-04-15 18:44:18 +02:00
2026-04-15 08:52:17 +02:00
2026-04-16 01:24:03 +02:00
2026-04-16 23:51:45 +02:00
2026-04-16 01:29:11 +02:00
2026-04-16 17:30:01 +02:00
2026-04-16 23:43:55 +02:00
2026-04-16 01:24:03 +02:00
2026-04-17 09:01:53 +02:00
2026-04-16 21:44:35 +02:00
2026-04-18 14:59:57 +02:00
2026-04-18 16:03:45 +02:00
2026-04-18 23:59:35 +02:00
2026-04-18 22:04:41 +02:00
2026-04-19 12:10:49 +02:00
2026-04-19 23:47:04 +02:00
2026-04-20 18:58:35 +02:00
2026-04-20 08:51:42 +02:00
2026-04-20 08:17:34 +02:00
2026-04-20 22:03:14 +02:00
2026-04-20 14:29:53 +02:00
2026-04-21 02:05:28 +02:00
2026-04-22 08:42:50 +02:00
2026-04-21 21:06:33 +02:00
2026-04-22 08:51:51 +02:00
2026-04-22 09:06:50 +02:00
2026-04-22 09:06:50 +02:00
2026-04-21 11:04:51 +02:00