feat(rl): reset_session_state regime boundary policy (F1.7)

Extends reset_session_state with 8 new regime_observer slot resets per
spec section "Regime observer reset policy at fold/eval boundaries":

- Transient state (FLAG/DURATION/TIMEOUT) → 0
- RECOVERY_FACTOR → 1.0
- TAIL_EVENT_RECENCY → 1e6 sentinel
- KELLY_EPS_RECOVERY_LIVE → 0.50 (= ε_max)
- PREV_WORST_PNL → 0 (CRITICAL — prevents spurious 3σ tail at boundary)
- POPART_MAX_ABS_REWARD_EMA → 0 (F4 envelope reset)

Welford state (M2/MEAN/COUNT) intentionally NOT reset — variance of
session_pnl_change is regime-invariant per pearl_adaptive_carryover_discipline.

Issue γ fix: without PREV_WORST_PNL reset, the first regime_observer call
after a fold boundary would see Δworst = 0 - (-\$15k_train) = +\$15k, fire
a spurious 3σ tail event, and peg ε_recovery at ε_min for 100 steps.
This commit is contained in:
jgrusewski
2026-05-31 12:15:34 +02:00
parent 67f862191f
commit 0a9ff73b1f

View File

@@ -3059,6 +3059,25 @@ impl IntegratedTrainer {
(crate::rl::isv_slots::RL_REWARD_CLAMP_CLIP_RATE_EMA_INDEX, 0.0_f32),
// v9 — arm the defensive warmup window.
(crate::rl::isv_slots::RL_EVAL_WARMUP_REMAINING_INDEX, warmup_steps),
// Regime observer boundary policy (F1.7 — spec 2026-05-31 v3,
// [[pearl_adaptive_carryover_discipline]]):
// - Transient state (FLAG/DURATION/TIMEOUT) → 0
// - RECOVERY_FACTOR → 1.0 (no recent tail; ε = ε_max)
// - TAIL_EVENT_RECENCY → 1e6 sentinel ("no tails seen yet")
// - KELLY_EPS_RECOVERY_LIVE → 0.50 (= ε_max default)
// - PREV_WORST_PNL → 0 (CRITICAL — must align with the just-reset
// session_pnl_worst above to prevent spurious 3σ tail event)
// - POPART_MAX_ABS_REWARD_EMA → 0 (F4 envelope reset)
// - Welford state (M2/MEAN/COUNT) intentionally NOT reset — the
// variance of session_pnl_change is regime-invariant.
(crate::rl::isv_slots::RL_REGIME_DEAD_ZONE_FLAG_INDEX, 0.0_f32),
(crate::rl::isv_slots::RL_REGIME_DEAD_ZONE_DURATION_INDEX, 0.0_f32),
(crate::rl::isv_slots::RL_REGIME_DEAD_ZONE_TIMEOUT_FLAG_INDEX, 0.0_f32),
(crate::rl::isv_slots::RL_REGIME_TAIL_EVENT_RECENCY_INDEX, 1.0e6_f32),
(crate::rl::isv_slots::RL_REGIME_RECOVERY_FACTOR_INDEX, 1.0_f32),
(crate::rl::isv_slots::RL_KELLY_EPS_RECOVERY_LIVE_INDEX, 0.50_f32),
(crate::rl::isv_slots::RL_REGIME_PREV_WORST_PNL_INDEX, 0.0_f32),
(crate::rl::isv_slots::RL_POPART_MAX_ABS_REWARD_EMA_INDEX, 0.0_f32),
] {
let mut args = RawArgs::new();
args.push_ptr(self.isv_dev_ptr);