# Regime-Invariance — Four Interventions for the Train→Eval Gap

**Date**: 2026-06-02
**Status**: Spec draft
**Branch**: ml-alpha-regime-observer (current)
**Reference SHA**: 63fc16f17 (v5.2 adaptive scaffold)
**Linked pearls**: [[pearl_surfer_baseline_was_train_only_never_eval]], [[pearl_reward_signal_anti_aligned_with_pnl]], [[project_adaptive_scaffold_session_2026_06_02]]

## §0. Why this spec exists

The session of 2026-06-01/02 burned 4 cluster smokes iterating on the surfer-scaffold controller (v4 → v5 → v5.1 → v5.2). Each iteration revealed a controller bug introduced by the previous fix. None reached eval verdict before being killed; only v5.2 (alpha-rl-grwwh) was let to run to completion. Pattern matches `feedback_going_in_circles_pattern`.

**Diagnostic conclusion**: the controller iterations are tuning the wrong knob. The fundamental failure mode is that **the agent learns a trend-specialized policy from fold-1 train data that does not generalize to OOS regime shifts**. Phase 5 hold-bonus teaches "ride winners, cut losers" — works in trends, fails in chop. No fade timing fixes this — the inductive bias itself is regime-dependent.

Perplexity consultation 2026-06-02 confirmed this is a known class of problem in financial RL with a documented playbook:
- Domain randomization over market regimes
- Regime-conditional architectures
- Distributionally robust objectives
- Microstructure features tied to invariants, not trend structure

This spec captures the 4 actionable interventions ranked by leverage/risk ratio. Each intervention is self-contained — they can ship in series or in parallel.

## §1. Intervention #1 — R-multiple reward (HIGHEST LEVERAGE)

**Problem**: Phase 5 hold-bonus rewards `hold_bonus × √hold_time` for profitable holds. This is dollar-magnitude reward in market-dependent units. In trending markets, holding profitable positions accumulates more dollars per step. In choppy markets, holding profitable positions reverses → reward drops → policy un-learns the hold pattern. Reward magnitude scales with the trend regime.

**Fix**: replace Phase 5 hold-bonus + long-ride-bonus with **R-multiple reward** normalized by entry-time ATR:

```c
// At close event (done > 0.5):
const float atr_at_entry = unit_atr_at_entry[b];  // captured at flat→open transition
const float r_multiple   = realized_pnl_delta / fmaxf(atr_at_entry, EPS_ATR);
r = r_multiple;  // dimensionless, regime-invariant
```

**Why this generalizes**: a $1000 win in high-vol ATR=$2000 → R=0.5. A $300 win in low-vol ATR=$600 → R=0.5. Same reward signal. Policy learns "capture 0.5 ATR worth of move" not "make $X regardless of vol."

### §1.1 Implementation

- New CUDA buffer: `unit_atr_at_entry_d` (per-account f32, captured at flat→open transition in `extract_position`)
- Read ATR from existing risk-stack signals (already computed for Kelly position sizing)
- Replace Phase 5 steps 2-4 entirely with R-multiple computation at close events
- **Drop the surfer-scaffold controller** — no longer needed if reward is regime-invariant
- Keep Phase 5 step 1 (entry cost) — fees are regime-invariant in absolute terms; alternatively also divide by ATR

### §1.2 Why surfer-scaffold controller can retire

The 753 ISV slot (`RL_SURFER_SCAFFOLD_WEIGHT_INDEX`) and its 3 config slots (754/755/756) become obsolete. R-multiple reward is its own invariant — no scaffold needed because the gradient is already regime-correct. Slots delete back to `RL_SLOTS_END=753`. Per `feedback_no_feature_flags` + `feedback_single_source_of_truth_no_duplicates` the scaffold infrastructure (controller kernel + bootstrap entries + Phase 5 gating logic) gets deleted in the same commit.

### §1.3 Falsification

Cluster smoke fold-1 b=1024 20k+5k. Verdict criteria:
- **PASS**: train wr ≥ 0.30 by step 5k AND eval wr ≥ 0.28 (within 10% of train) AND eval pnl > 0
- **FAIL**: eval wr < 0.25 OR eval pnl < −$5M → R-multiple normalization insufficient; proceed to Intervention #2 (regime features) on top

**Scope**: ~30 LOC kernel + ~50 LOC trainer (ATR capture) + delete ~150 LOC scaffold infra. Net code REDUCTION.

## §2. Intervention #2 — Wire `regime_observer` features into policy network input

**Problem**: F1.3 already computes regime descriptors (Welford σ_pnl, dead_zone, recovery_factor, eps_live) but they're emitted to `diag.jsonl` only. The policy network has no input that tells it "we're in a chop regime, behave differently."

**Fix**: extend the encoder input vector with 4-6 regime features. Policy explicitly conditions on regime instead of baking trend-assumption into its core behavior.

### §2.1 Existing signals to wire

Read from ISV:
- `RL_REGIME_FLAT_COUNT_INDEX` — fraction of accounts that are flat (regime activity proxy)
- `RL_REGIME_WELFORD_SIGMA_INDEX` — pnl variance EMA (vol regime)
- `RL_REGIME_DEAD_ZONE_INDEX` — fraction of time in dead-zone (chop indicator)
- `RL_REGIME_RECOVERY_FACTOR_INDEX` — current/max DD ratio (drawdown regime)
- `RL_EDGE_PH_MEAN_INDEX` — Page-Hinkley edge-decay signal (per-trade EV regime)

### §2.2 Implementation

- Extend encoder input projection from current N dims to N+5 dims
- Bootstrap the new weight columns to small random or zero — agent learns to use them
- 5 new tokens broadcast to every account in the batch via existing ISV-read kernel

**Scope**: ~80 LOC (encoder projection weight matrix + 5 ISV reads in encoder forward) + checkpoint format bump

### §2.3 Caveat

This works ONLY when training data exposes multiple regimes (which #1 alone doesn't solve since we still train on fold 1). Therefore best deployed AFTER or WITH Intervention #3.

## §3. Intervention #3 — Cross-fold training (domain randomization)

**Problem**: Current walk-forward setup gives fixed fold 1 train + fold 1 eval. If fold 1 has a different regime mix than the deployment distribution, the agent over-fits to fold-1's regime. Per `pearl_fold_split_is_quarterly_regime_break` (SUPERSEDED for current PVC data but the principle stands), folds can differ substantially.

**Fix**: at episode start, sample a random fold from {0, 1, 2} for training; evaluate on a held-out fold (e.g., the latest quarter). Forces the policy to be **regime-mix robust**.

### §3.1 Implementation

- Modify walk-forward sampler in `alpha_rl_train.rs` (or trainer init) to randomly sample fold per batch element
- Need 3× the training data resident on PVC — already there (24 months ES MBP-10 per `pearl_mbp10_data_is_structurally_broken` correction)
- Eval stays on a held-out window (e.g., last 3 months never seen in train)

**Scope**: ~30 LOC trainer + verification that all 3 folds load cleanly

### §3.2 Risk

Trade rate may drop initially (more diverse regimes → more uncertainty → policy more conservative). Worth ~5k extra steps of training to compensate.

## §4. Intervention #4 — Mixture-of-experts gated by regime

**Problem**: A single policy network must serve all regimes. The optimal trading strategy is genuinely different across regimes (trend vs mean-revert). One policy averages them → mediocre everywhere.

**Fix**: 2-3 specialist policies + gating network. Gating reads regime features (from #2), outputs softmax weights over experts. Each expert specializes on its dominant regime.

**Now spec'd separately**: see `docs/superpowers/specs/2026-06-02-multi-head-policy-with-r-multiple.md` for the architecturally clean version (multi-head policy on shared encoder, no reward-shaping diversity, no invented ISV signals). That spec was written after sp-critical-reviewer BLOCKED the prior in-line v6 attempt for inventing a `trend_strength` signal that does not exist and re-introducing the anti-Pearson reward shaping pattern.

### §4.1 Architecture

- Expert A: trend-follower (current Phase 5 baseline behavior)
- Expert B: mean-reversion (different reward shaping — capture short oscillations)
- Optional Expert C: stay-flat (no positions when neither A nor B is confident)
- Gating net: 2-layer MLP on regime features → softmax(K) over experts
- Forward: `action = Σ_k gate_k × expert_k(state)`
- Backward: each expert's gradient weighted by `gate_k`; gate net learns from policy gradient

### §4.2 Implementation

- 3× the policy head parameters
- Each expert trained on the same data but the reward shaping differs (use a per-expert reward signal — trend-expert uses R-multiple of hold-duration; reversion-expert uses R-multiple of inverse-hold-duration)
- Most complex of the 4 interventions

**Scope**: ~250 LOC over 2 days. Defer until #1-#3 are validated.

## §5. Sequencing & decision points

### Order of operations

1. **Ship #1 (R-multiple reward)** — single commit, kills the surfer-scaffold infra in the same commit. Cluster smoke fold-1 b=1024 20k+5k. Plan at `docs/superpowers/plans/2026-06-02-regime-invariance-implementation.md`.
2. **Wait for #1 verdict**. If eval pnl > 0: #1 was sufficient.
3. **If #1 eval fails**: ship multi-head policy (#4 — separate spec at `docs/superpowers/specs/2026-06-02-multi-head-policy-with-r-multiple.md`). Architectural fix that does not violate Pearson alignment.
4. **If #1+#4 eval fails**: ship #3 (cross-fold training) and #2 (regime features) in sequence as remaining levers.

### Falsification criteria (each step)

Each intervention has the same eval-gate:
- Eval pnl > 0 (any positive amount) → PASS, ship
- Eval pnl in [−$2M, 0] → MARGINAL, continue to next intervention
- Eval pnl < −$2M → FAIL, deeper rethink (maybe the architecture / data pipeline itself)

### Termination clause

If ALL 4 interventions ship and eval still fails, the conclusion is that this neural architecture (mamba2 + DQN-IQN + PPO) cannot extract alpha from MBP-10 ES futures at b=1024 20k steps. We then either:
- Change data scale (much longer training, multi-instrument)
- Change architecture (attention-based, longer context window)
- Change approach (behavior cloning from a profitable rule-based bot, then fine-tune)

## §6. Reward-shaping discipline (cross-cutting)

All 4 interventions assume the reward signal must be:
1. **Regime-invariant in units** (R-multiple, not dollars)
2. **Aligned with realized pnl direction** (Pearson ≥ 0.7 per `pearl_reward_signal_anti_aligned_with_pnl`)
3. **Sparse at close-events only** (not per-step inflation)
4. **Bounded** (no exponential growth with hold time)

Phase 5 hold-bonus violates (1), (3), (4). The v5.2 adaptive scaffold partially restored (2) but not (1). R-multiple satisfies all four cleanly.

## §7. Open questions to resolve before implementation

1. **ATR source for R-multiple denominator**: use existing Kelly ATR (already computed) or compute fresh? Check Kelly controller for double-update risk.
2. **Bootstrap value for ATR** at first-trade: ATR EMA needs warmup. Use `max(ATR_ema, ATR_floor)` with floor ≈ 1 ES tick × tick-multiplier.
3. **Eval-boundary discipline**: R-multiple uses ATR EMA — must reset at eval boundary per `pearl_adaptive_carryover_discipline` (or NOT, depending on whether we want eval to start with train-end calibration).
4. **Fold sampling weight**: equal-weight all 3 folds, or weight by regime-novelty? Active domain randomization (ADR) literature suggests the latter; defer until #1 works.
