# v9 — Defensive Eval-Boundary Calibration (Adaptive Carryover Discipline)

**Goal**: close the v8 train-eval gap by completing the adaptive-controller principle at the regime boundary. Reset *all* adaptive EMAs on train→eval transition, override the controllers that drive risk sizing with *defensive defaults* during a configurable warmup window, and use higher learning rates during the warmup so network weights adapt fast to new-regime statistics.

**Why now**: v8 (SHA `6d4a962e5`, `alpha-rl-vxbpq`) shipped the full risk-stack and achieved train +$135M / eval **−$507k** on fold-1 walk-forward. 18% better than OLD baseline (−$616k), but still deeply negative on held-out data. Diagnosis: Fix D (commit `7064c9269`) intentionally preserved Kelly + inventory EMAs across the fold boundary on the rationale that "training-acquired statistics should carry into eval." That was wrong. The new pearl [[pearl_adaptive_carryover_discipline]] proves this is a structural failure mode at every regime transition.

**Out of scope**: changes to the policy network architecture, encoder, atom span sizing (proven sound by `2026-05-31-c51-atom-span-math-validation`), reward shaping (deferred to v10), meta-learning across folds (deferred to v11+).

---

## 1. Diagnosis — why Fix D fails at the eval boundary

The risk-stack has 12+ adaptive controllers, all driven by EMAs of recent realized signals. Fix D (in `crates/ml-alpha/src/trainer/integrated.rs::reset_session_state`) resets *some* per-batch buffers but explicitly NOT the Kelly + inventory EMAs:

```rust
// Kelly + inventory EMAs are intentionally NOT reset: training-acquired
// statistics should carry into eval (matches "policy learned during
// train, applied in eval" intent).
```

This rationale violates the meta-principle of the whole risk-stack: behavior should derive from RECENT REALIZED signals. By preserving train EMAs into eval, we put the controllers in a *poisoned* state:

**Cascade at the v8 eval boundary**:

| EMA | Train value at step 20k | Eval reality (over 5000 eval steps) | Time to re-converge at α=0.05 |
|---|---|---|---|
| `Kelly.win_rate_ema` | 0.34 | 0.25 | ~14 trade events |
| `Kelly.avg_win_usd_ema` | $3000+ (scaled) | $11.7k (eval real) | ~14 events |
| `Kelly.avg_loss_usd_ema` | $200+ (scaled) | $6.6k (eval real) | ~14 events |
| `Kelly.cumulative_dones` | 261,478 | n/a (not reset → still huge) | — |
| `Inventory.penalty_beta` | 3.81 | (depends on new reward magnitude) | ~10 steps |

With `cumulative_dones >> min_trades=1000`, Kelly is "out of warmup" — its safety mechanisms (warmup gate, dead-signal guard) don't fire. Kelly computes `f = 0.5 × (0.34×14 − 0.66)/14 ≈ 0.146` from train EMAs and feeds it to position sizing. The agent enters eval with this aggressive sizing on what turns out to be a losing distribution.

By the time the EMAs re-converge (~200 cluster steps = 4% of the 5000-step eval phase), the damage is done. Worst-account session_pnl crosses dd_limit, Fix E force-closes, but the realized loss is locked in. The CMDP's downstream defenses (cooldown, IQN τ tightening) trigger AFTER the loss.

**Quantitative analysis**: v8 train had avg_win/avg_loss = $3k/$200 (post-scale). Train Kelly was sized for that distribution. Eval avg_win/avg_loss = $11.7k/$6.6k (raw, post-rescale). Position sizing optimized for tiny-loss-vs-big-win pattern fails when both sides are large in real terms.

---

## 2. Design — three-layer defensive boundary

### Layer 1: Reset *all* adaptive EMAs at boundary

Extend `reset_session_state()` to reset every signal-driven EMA, not just CMDP:

```rust
// Summary slots (existing — kept)
RL_SESSION_PNL_USD_INDEX                      → 0.0
RL_SESSION_PNL_WORST_INDEX                    → 0.0
RL_SESSION_DD_TRIGGERED_INDEX                 → 0.0
RL_CONSEC_LOSS_COUNT_INDEX                    → 0.0
RL_COOLDOWN_REMAINING_STEPS_INDEX             → 0.0

// Kelly EMAs (NEW — was preserved by Fix D, now reset)
RL_WIN_RATE_EMA_INDEX                         → 0.5 (neutral sentinel)
RL_AVG_WIN_USD_EMA_INDEX                      → 0.0 (sentinel)
RL_AVG_LOSS_USD_EMA_INDEX                     → 0.0 (sentinel)
RL_CUMULATIVE_DONES_INDEX                     → 0.0 (re-enter warmup)
RL_KELLY_FRACTION_INDEX                       → 1.0 (bootstrap = no sizing change until warmup releases)

// Inventory EMAs (NEW — was preserved by Fix D, now reset)
RL_INVENTORY_PENALTY_BETA_INDEX               → 0.0 (sentinel, bootstrap from first observation)
RL_INVENTORY_VARIANCE_EMA_INDEX               → 0.0 (sentinel)

// Reward-clamp EMAs (NEW — were not explicitly preserved but also not reset)
RL_POS_SCALED_REWARD_MAX_EMA_INDEX            → 0.0 (sentinel)
RL_NEG_SCALED_REWARD_MAX_EMA_INDEX            → 0.0 (sentinel)
RL_REWARD_CLAMP_CLIP_RATE_EMA_INDEX           → 0.0 (sentinel)

// Per-batch buffers (existing from Fix D — kept)
session_pnl_per_batch_d                       → memset 0
session_dd_triggered_per_batch_d              → memset 0
consec_loss_per_batch_d                       → memset 0
cooldown_remaining_per_batch_d                → memset 0
```

This implements: at the boundary, every adaptive signal restarts from neutral and re-learns from new-regime data via its normal bootstrap path.

### Layer 2: Defensive warmup window — risk controllers

For the first `RL_EVAL_WARMUP_STEPS = 500` steps after the boundary, override risk-controller outputs with *defensive defaults*:

| Slot | Normal value | Warmup override | Rationale |
|---|---|---|---|
| `RL_KELLY_SAFETY_FRAC_INDEX` | 0.5 (half-Kelly) | **0.25** (quarter-Kelly) | half position sizing during regime discovery |
| `RL_IQN_ACTION_TAU_MIN_INDEX` | 0.10 | **0.30** | more pessimistic action selection (lower quantile of IQN distribution) |
| `RL_ENTROPY_COEF_MIN_INDEX` | 0.01 | **0.05** | force more exploration / less commitment to learned strategy |
| `RL_PPO_CLIP_EPS_MIN_INDEX` | 0.05 | **0.10** | larger policy update steps allowed (faster adaptation) |

These are restored to normal values at the end of the warmup window via a `rl_eval_warmup_decay` kernel that linearly interpolates between defensive and normal values over an additional `RL_EVAL_WARMUP_DECAY_STEPS = 200` steps. Total transitional period: 700 steps (~14% of a 5000-step eval phase).

### Layer 3: Higher learning rate during warmup — network weights

During the warmup window, lift LR by 2× across all heads (Q, π, V, IQN):

```rust
// In reset_session_state, after resetting EMAs:
let warmup_remaining = RL_EVAL_WARMUP_STEPS;  // new ISV slot
isv[RL_EVAL_WARMUP_REMAINING_INDEX] = warmup_remaining;

// In step_with_lobsim, during warmup:
//   if warmup_remaining > 0:
//     lr_eff = lr_base × 2.0
//     warmup_remaining -= 1
```

Justification: network weights are the slowest-adapting layer of the agent. During regime discovery, fast SGD updates let the policy re-calibrate faster than its default LR. 2× is conservative — major literatures use 5-10× for transfer learning fine-tuning, but 2× preserves stability with the existing PER + PPO infrastructure.

---

## 3. ISV slots added

```rust
pub const RL_EVAL_WARMUP_REMAINING_INDEX:   usize = 686;  // counter, decremented each step in warmup
pub const RL_EVAL_WARMUP_STEPS_INDEX:       usize = 687;  // configured warmup duration (bootstrap 500)
pub const RL_EVAL_WARMUP_DECAY_STEPS_INDEX: usize = 688;  // decay duration (bootstrap 200)

// Defensive override values (read by controllers when warmup_remaining > 0)
pub const RL_EVAL_KELLY_SAFETY_FRAC_OVERRIDE_INDEX:  usize = 689;  // bootstrap 0.25
pub const RL_EVAL_IQN_TAU_MIN_OVERRIDE_INDEX:        usize = 690;  // bootstrap 0.30
pub const RL_EVAL_ENTROPY_COEF_MIN_OVERRIDE_INDEX:   usize = 691;  // bootstrap 0.05
pub const RL_EVAL_PPO_CLIP_EPS_MIN_OVERRIDE_INDEX:   usize = 692;  // bootstrap 0.10
pub const RL_EVAL_LR_MULTIPLIER_INDEX:               usize = 693;  // bootstrap 2.0

// RL_SLOTS_END: 686 → 694
```

---

## 4. Validation criteria

### Local smoke (b=128, single fold-0 with --n-eval-steps 1000)
- v8 trajectory parameters preserved through train phase (qpa > +0.9, l_v < 1.0, etc.)
- At eval boundary: Kelly EMAs reset to sentinels, warmup_remaining = 500
- During warmup: kelly_f < 0.10 typically (quarter-Kelly forcing), action_entropy > 1.8 (higher entropy)
- Eval-phase pnl > v8 fold-0 eval (whatever that turns out to be)

### Cluster validation (b=1024)
- **Run all 3 folds** (per [[pearl_single_window_oos_is_not_oos]]). v8 only ran fold-1. We need fold-0, fold-1, fold-2 with the same v8 baseline and again with v9 for clean comparison.
- Success criterion: **mean eval pnl across 3 folds > 0** (vs v8's known fold-1 = −$507k, unknown 0/2)
- Or weaker: **mean eval pnl > v8 mean** (improvement, even if still negative)

### Pearl invariant
After v9 ships, [[pearl_adaptive_carryover_discipline]] should be referenced in every PR that touches a `reset_*_state` or session-boundary handler.

---

## 5. Implementation plan

1. **isv_slots.rs**: add 8 new ISV slots (lines after current `RL_C51_ATOM_MARGIN_INDEX = 685` — note that one was deleted via the Fix F revert, so 685 is open). Bump `RL_SLOTS_END` to 694.

2. **integrated.rs `reset_session_state`**: extend to reset all Kelly/inventory/clamp EMAs listed in §2 Layer 1. Set `RL_EVAL_WARMUP_REMAINING_INDEX = RL_EVAL_WARMUP_STEPS_INDEX`.

3. **integrated.rs bootstrap**: add 8 new ISV bootstrap entries with values from §3. Bump bootstrap array length from `169` to `177`.

4. **New kernel `rl_eval_warmup_decay.cu`**: single-thread kernel, runs once per step. Reads `RL_EVAL_WARMUP_REMAINING_INDEX`. If > 0, applies defensive overrides to `RL_KELLY_SAFETY_FRAC_INDEX`, `RL_IQN_ACTION_TAU_MIN_INDEX`, `RL_ENTROPY_COEF_MIN_INDEX`, `RL_PPO_CLIP_EPS_MIN_INDEX`. During decay phase (warmup_remaining ∈ [-decay_steps, 0]), linearly interpolates back to normal. Decrements counter.

5. **integrated.rs step_with_lobsim**: launch `rl_eval_warmup_decay` after `reset_session_state` is called (i.e., on first eval step). LR multiplier applied via `RL_EVAL_LR_MULTIPLIER_INDEX` read in LR controllers.

6. **Diag emit**: add `risk_stack.eval_warmup.{remaining, kelly_safety_active, tau_min_active}` for visibility.

7. **Invariant test G14**: GPU-oracle test that confirms the override values are written during warmup and decay back to normal after.

8. **Local smoke**: b=128, 5000 train + 1000 eval. Verify reset + warmup behavior in JSONL.

9. **Atomic commit**, push, submit cluster.

---

## 6. Memory pearl to ship

**See `[[pearl_adaptive_carryover_discipline]]`** (newly created in this session). The pearl captures the structural principle this spec operationalizes. Future sessions auditing `reset_*_state` or designing boundary handlers should reference it.

---

## 7. Risks and rollback

**Risk 1: Resetting Kelly EMAs eliminates train signal that was actually useful**. If train EMAs DO encode useful priors for eval (e.g., R-multiple structure), resetting them loses information. Mitigation: warmup window is short (500 steps = 10% of eval phase); after warmup, EMAs re-converge to eval reality which is the correct prior anyway.

**Risk 2: Higher LR during warmup destabilizes V/Q**. With existing popart normalization + LR plateau controllers, this should be robust. But if l_v spikes during warmup, the LR multiplier could be tuned down (1.5× or 1.25×).

**Risk 3: Defensive sizing prevents agent from capturing eval-phase alpha that DOES generalize**. Possible. Defensive warmup is structured to give up SOME early-eval performance in exchange for not blowing up. The expected-value trade is favorable given v8's eval result was −$507k.

**Rollback path**: revert this commit. Cluster runs at v8 SHA `6d4a962e5` are reproducible.

---

## 8. Connection to broader system

This spec completes the adaptive-controller principle that the whole risk-stack was building toward. After v9, every behaviorally-relevant signal in the system will derive from recent realized data, including at regime transitions. The static special-case in Fix D becomes a closed-loop adaptive controller like everything else.

If v9 closes the train-eval gap substantially: confirms the eval problem was *transition mismanagement*, not fundamental overfitting.

If v9 only modestly improves: confirms remainder is genuine overfitting/non-generalization, justifying the deeper Level 2 (regime-feature conditioning) or Level 3 (meta-learning) interventions sketched in the eval-problem ultrathink.

Either outcome is *informative* in a way single-fold v8 results aren't.
