gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fix(rl): LOSS bootstrap 3.0 → 1.5 — don't start in a hole

Browse Source 
LOSS=3.0 bootstrap means Q tolerates 3× losses during warmup before
the done-gated EMAs adapt. This accumulates -$2M before the controller
kicks in. Bootstrap 1.5 = moderate loss aversion from step 0.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-05-28 01:42:09 +02:00
parent 9355118e8b
commit a9c74bac33
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
crates/ml-alpha/src/trainer/integrated.rs
View File

@@ -2687,7 +2687,7 @@ impl IntegratedTrainer {
// [-1, +1] span as the canonical floor — the ratchet
// in `rl_reward_clamp_controller` only grows magnitude.
(crate::rl::isv_slots::RL_REWARD_CLAMP_WIN_INDEX, 1.0),
(crate::rl::isv_slots::RL_REWARD_CLAMP_LOSS_INDEX, 3.0),
(crate::rl::isv_slots::RL_REWARD_CLAMP_LOSS_INDEX, 1.5),
(crate::rl::isv_slots::RL_REWARD_CLAMP_MARGIN_INDEX, 1.5),
(crate::rl::isv_slots::RL_REWARD_CLAMP_RATIO_INDEX, 3.0),
(crate::rl::isv_slots::RL_REWARD_CLAMP_CLIP_RATE_TARGET_INDEX, 0.05),