gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fix(rl): reward clamp bootstrap WIN=1.0 LOSS=3.0 (was 0.5/0.5)

Browse Source 
The clamp bounds were bootstrapped at ±0.5 (matching the old ±0.5 atom
span) instead of the intended asymmetric WIN=1.0, LOSS=3.0. This
caused: (1) apply_reward_scale clamped at ±0.5 instead of [-3,+1],
(2) the C51 atom span EWMA target = max(1.0, 0.5) = 1.0 → V_MIN
never moved because target min(-1.0, -0.5) = -1.0 = bootstrap.

Now: WIN=1.0 (positive reward ceiling), LOSS=3.0 (loss-aversion
asymmetry per pearl_audit_unboundedness). Atom span will EWMA from
[-1, +1] toward [-3, +1] over ~2100 steps (α=0.001).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-05-27 15:46:59 +02:00
parent 18e19b4733
commit dd049d9a4c
1 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
crates/ml-alpha/src/trainer/integrated.rs
View File

@@ -2686,8 +2686,8 @@ impl IntegratedTrainer {
// outliers." V_MAX/V_MIN seeded to the original C51
// [-1, +1] span as the canonical floor — the ratchet
// in `rl_reward_clamp_controller` only grows magnitude.
(crate::rl::isv_slots::RL_REWARD_CLAMP_WIN_INDEX, 0.5),
(crate::rl::isv_slots::RL_REWARD_CLAMP_LOSS_INDEX, 0.5),
(crate::rl::isv_slots::RL_REWARD_CLAMP_WIN_INDEX, 1.0),
(crate::rl::isv_slots::RL_REWARD_CLAMP_LOSS_INDEX, 3.0),
(crate::rl::isv_slots::RL_REWARD_CLAMP_MARGIN_INDEX, 1.5),
(crate::rl::isv_slots::RL_REWARD_CLAMP_RATIO_INDEX, 3.0),
(crate::rl::isv_slots::RL_REWARD_CLAMP_CLIP_RATE_TARGET_INDEX, 0.05),