diff --git a/docs/superpowers/specs/2026-05-06-trader-discipline-and-recovery-design.md b/docs/superpowers/specs/2026-05-06-trader-discipline-and-recovery-design.md new file mode 100644 index 000000000..2d8d364d3 --- /dev/null +++ b/docs/superpowers/specs/2026-05-06-trader-discipline-and-recovery-design.md @@ -0,0 +1,684 @@ +# SP15: Trader Discipline and Recovery — Design + +> **Status**: spec +> **Date**: 2026-05-06 +> **Predecessors**: SP11/12/13/14 chain merged into main at `0275a25d9` (2026-05-06) +> **Implementation branch**: `sp15-trader-discipline-recovery` (forks from main post-spec-commit) +> **Authoritative path**: `docs/superpowers/specs/2026-05-06-trader-discipline-and-recovery-design.md` + +## TL;DR + +Take the system from "stable, gradient-flowing, but trades like a panicked toddler" to "disciplined trader with measurable competence on a sealed quarter." Six phases: + +1. **Phase 0** — Retune SP14 EGF (gate1=closed forever) so its anchors are ISV-driven, not hardcoded. +2. **Phase 1** — Honest numbers: unified sharpe kernel, cost-net sharpe (commission + spread + OFI-impact, dev/prod parity), drawdown reporting, 8 counterfactual baselines, `dd_pct` as foundational state input, `--holdout-quarters` + `--dev-quarters` CLI flags, consume abandoned walk-forward test slice. +3. **Phase 2** — 21-test behavioral suite on dev RTX 3050 Ti. Pre-commit hook blocks `argo-train.sh` if any test fails. CI gate before L40S. +4. **Phase 3** — 5 trader teachings (reward + action selection): r_quality/r_discipline split, explicit cost, quadratic DD penalty, regret signal, confidence-aware Hold floor. +5. **Phase 3.5** — 4 recovery mechanisms (3.5.1 already in Phase 1): asymmetric reward under DD, cooldown gate, plasticity injection, recovery curriculum in PER. +6. **Phase 4** — L40S walk-forward on Q1-Q7, Q8 final dev, sealed Q9 OOS — produces report card with deltas vs all 8 baselines. + +**Cross-cutting discipline rule**: no new pearl, controller, kernel, ISV slot, or reward term ships without a Phase 2 behavioral test that proves the intended trader behavior. Hard rule alongside `feedback_no_partial_refactor`. + +**Estimated scope**: ~4900 LOC, 13-21 days serial / 10-15 days with parallel dispatch (Approach B). + +--- + +## 1. Premise + +> "we know it's stable, we know it learns, but it has behavioural issues as well as mechanical issues. The model doesn't understand its role as trader. Drives itself into a downward spiral, very hard to recover from." +> +> — User, 2026-05-06, post-`train-dd4xl` termination + +The architecture works. Gradients flow. Smoke tests pass. But: + +- Each iteration we identify a gaming behaviour and add an adaptive controller. Controllers control controllers. We end up with a 50-controller toddler, not a trader. +- The model has no architectural mechanism to escape its own losses. Once it starts losing, it learns "trade less" — which reduces gradient signal — which slows recovery — which compounds. +- We measure on the same window we tune controllers against. "Out-of-sample" sharpe on val is a fantasy: val IS the selection set. + +This spec is the alternative path: fix measurement, validate behaviour locally, teach role explicitly. + +--- + +## 2. Failure evidence + +### 2.1 The spiral curve (`train-dd4xl`, 8 epochs, terminated 2026-05-06) + +Commit `c0fc28e45` (full SP14 Layer A+B chain, bus fix, warmup gate delete) on L40S walk-forward: + +``` + trades active sharpe pf dir_ent +ep1: 131,805 0.482 79.45 1.195 1.03 +ep2: 127,463 0.446 70.46 1.197 1.00 +ep3: 116,817 0.381 65.87 1.214 0.93 +ep4: 98,835 0.296 69.52 1.252 0.81 +ep5: 98,781 0.295 63.92 1.279 0.81 +ep6: 91,883 0.268 60.40 1.293 0.76 +ep8: 63,837 0.171 43.63 1.225 0.57 ← terminated +``` + +Monotone descent across `trade_count` (−51%), `active_frac` (−65%), `dir_entropy` (−45%). Profit factor briefly **rises** (1.195 → 1.293) — fake win, surviving trades are higher-confidence — then drops as the policy collapses to non-trading. + +### 2.2 HEALTH_DIAG[8] terminal state + +``` +val_dir_dist [short=0.07 hold=0.69 long=0.10 flat=0.14] ← 69% Hold +mag_stats wr q=0.429 h=0.412 f=0.414 ← all ~ random +mag_stats var q=4.6e-10 h=4.2e-9 f=1.1e-8 ← magnitudes flat +eval_dist eq=0.885 eh=0.115 ef=0.000 ← 88.5% Quarter only +c51_budget mag 0.0004 ← magnitude over-suppressed +aux_dir_acc short=0.518 long=0.487 ← 50/50 = no signal +aux_moe util one expert at 1.000, rest at 0 ← MoE collapse +improvement_z -0.850 ← controller says "worse" +gate1 closed ← EGF never opens +sp9_kelly floor=0.0000 ← Kelly literally cannot trade +``` + +The model isn't learning to trade. It's learning to look stable on the window it tunes against, while collapsing to "Hold most bars, pick Quarter mag, ignore direction." + +### 2.3 Walk-forward audit finding + +`crates/ml/src/cuda_pipeline/gpu_walk_forward.rs:122-165` defines proper expanding-window walk-forward with `train_start..train_end / val_start..val_end / test_start..test_end` per fold. + +`crates/ml/src/trainers/dqn/trainer/mod.rs:1155` consumes only `train` and `val`: +```rust +self.set_training_range(fold.train_start, fold.train_end, fold.val_start, fold.val_end); +self.set_val_data_from_slices(&features[fold.val_start..fold.val_end], ...) +``` + +`grep -rn "fold\.test_start"` outside the generator returns **zero hits**. The reported `sharpe_annualised=64-79` is on the **val window** — same window the early-stop callback, Kelly warmup cap, and controller anchors all read. + +What this means in practice: + +| Signal | Sees val window? | +|---|---| +| Parameter gradients | No (train only) ✅ | +| Early-stop callback | **Yes** ❌ | +| Kelly warmup cap | **Yes** ❌ | +| Controller anchors / EMAs | **Yes** ❌ | +| Reported "OOS" sharpe | **Yes** (it's the same window) ❌ | + +Val is temporally clean for parameter learning, but it's the selection set, not a sealed test set. Sharpe=80 reported means "best on selection set we tuned against," not "robust on unseen quarter." + +--- + +## 3. Decisions reference (clarifying questions, 2026-05-06) + +| # | Question | Decision | +|---|---|---| +| Q1 | DD-state architectural placement | **Foundational (Phase 1)**: ISV slot + producer kernel + trunk concat. Layout fingerprint break OK (greenfield). Eval-time policy SEES dd context. | +| Q2 | SP14 EGF cut vs retune | **Keep, retune to ISV-driven**. No hardcoded thresholds; Schmitt thresholds, variance gate refs, steepness anchors all become ISV-tracked. | +| Q3 | Cost model dev/prod parity | **commission + spread + OFI-impact, everywhere**. Dev synthetic markets generate synthetic spread + OFI. Prod reads from MBP10 fxcache. Same kernel. | +| Q4 | Counterfactual baselines | **All 8**: buyhold, hold-only, random_dir + same Kelly, naive momentum, aux-only, mag-Quarter-fixed, trail-only, naive reversion. | +| Q5 | Phase 2 test set | **All 21 tests**: 7 trading behavior + 10 state/arch grounding + 4 recovery dynamics. | +| Q6 | OOS framework | **Train/dev/test split**: Q1-Q7 walk-forward train, Q8 final dev, Q9 sealed final test. | +| Q7 | Phase 3.5.4 plasticity injection | **Land all 4 recovery mechanisms** including plasticity. No defer. | + +--- + +## 4. Architecture overview + +### 4.1 Phase chain + +``` +Phase 0: SP14 EGF ISV-driven retune ─┐ +Phase 1: Honest numbers + dd_pct foundation ─┼─ parallel sub-worktrees (Approach B) +Phase 2A: behavioral test scaffolding ─┘ + │ + ▼ (Phase 2B individual tests land as Phase 1 capabilities ship) + Phase 2B: Groups 1+2 (17 tests, gates Phase 3 entry) + │ + ▼ + Phase 3: 5 trader teachings (atomic per teaching, each gated on anchor 2.X test) + │ + ▼ + Phase 3.5: 4 recovery mechanisms (3.5.1 already in Phase 1.5) + │ ↑ Phase 2C (Group 3 tests) interleaved with each teaching + ▼ + Phase 4: L40S walk-forward (Q1-Q7) + Q8 final dev + sealed Q9 OOS +``` + +### 4.2 Branch and worktree structure + +- **Branch**: `sp15-trader-discipline-recovery` forks from main (`0275a25d9` post-SP11-merge) +- **Sub-worktrees** for parallel dispatch: + - `sp15-phase0-egf-retune` — SP14 EGF retune (Phase 0) + - `sp15-phase1-honest-numbers` — Phase 1 sub-fixes (parallel to Phase 0) + - `sp15-phase2a-test-scaffold` — synthetic market generators + harness (parallel to Phase 0+1) +- **Merge model**: each phase merges back to `sp15-trader-discipline-recovery` atomically. SP15 branch merges to main only after Phase 4 production-track gate passes. + +### 4.3 Cross-cutting discipline rule + +**No new pearl, controller, kernel, ISV slot, or reward term ships without:** +1. **Phase 2 behavioral test** that proves the intended trader behavior (the test commit and the implementation commit ship together). +2. **Wire-up audit entry** in `docs/dqn-wire-up-audit.md`. +3. **Memory pearl** in `~/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/` if the technique generalises beyond this spec. + +This adds to `feedback_no_partial_refactor`, `feedback_no_stubs`, `feedback_isv_for_adaptive_bounds`, `feedback_wire_everything_up` as a hard discipline rule. Violations mean the change is reverted, not merged-with-followup. + +--- + +## 5. Phase 0 — SP14 EGF ISV-Driven Retune (~1-2 days, atomic per fix) + +### 5.1 Goal + +Make EGF actually fire under target conditions. Eliminate every hardcoded threshold per `feedback_isv_for_adaptive_bounds`. + +### 5.2 Phase 0.A — Diagnose + +`HEALTH_DIAG[7]/[8]` from `train-dd4xl` showed: +- `var_aux=0.00021` against threshold `0.01` → 50× below trip point +- `var_q=0.00879` against threshold `0.05` → 6× below trip point +- `α_smoothed=0.0008`, `α_raw=0.0006` → mathematically consistent with closed gate + +Either the thresholds are wrong (signal magnitudes are inherently smaller than expected when calibrated on synthetic-distribution training) or the signals being read into `var_aux`/`var_q` aren't the right ones (e.g., reading post-clipped values that have been collapsed). Diagnostic step: + +1. Add temporary HEALTH_DIAG emission of the raw aux/Q variance signals **before** EGF reads them. +2. Run 5-epoch L40S smoke against post-Phase-0.A HEAD. +3. Compare emitted variances to the EGF-internal `var_aux`/`var_q` values. Mismatch → upstream signal corruption. Match → thresholds need ISV-driven dynamic anchoring. + +### 5.3 Phase 0.B — Retune + +Every hardcoded anchor becomes ISV-driven: + +| Anchor (current value) | New ISV slot | Producer | +|---|---|---| +| Schmitt thresholds (±0.03 hardcoded) | `EGF_SCHMITT_HI_INDEX`, `EGF_SCHMITT_LO_INDEX` | rolling p99/p1 of α_raw over fold | +| Variance gate ref aux (0.01) | `EGF_VAR_AUX_REF_INDEX` | running mean of var_aux over fold | +| Variance gate ref Q (0.05) | `EGF_VAR_Q_REF_INDEX` | running mean of var_q over fold | +| Steepness normaliser (k_aux=20/(1+var_aux/0.01) hardcoded `0.01`) | reuses `EGF_VAR_AUX_REF_INDEX` | (above) | +| Steepness normaliser k_q (0.05 hardcoded) | reuses `EGF_VAR_Q_REF_INDEX` | (above) | + +Total: 4 new ISV slots + 2 new producer kernels. EGF kernels read these instead of constants. + +### 5.4 Phase 0 anchor test (Phase 2.21 — `egf_gate_opens_under_disagreement`) + +Synthetic batch with deliberately disagreeing aux head (output = +1.0) vs Q head (output = −1.0) → gate1 must open within K=100 steps. If it doesn't, EGF is broken regardless of how clean the kernels look. + +This test is authored alongside Phase 0.B and gates the commit. EGF retune cannot land without test 2.21 going green. + +--- + +## 6. Phase 1 — Honest Numbers (~2-3 days, 7 atomic sub-fixes) + +### 6.1 Sub 1.1 — Unified sharpe kernel + +Single GPU kernel `compute_sharpe_per_bar_kernel.cu`. Inputs: `pnl_array[T]`. Output: `(mean_pnl, std_pnl, sharpe_per_bar)`. Train and val use identical formula and identical window definition. + +Eliminates `sharpe_ema` (per-batch EMA with smoothing constant divergence) vs `sharpe_annualised` (val × sqrt(525,600)) confusion. Reports both `sharpe_per_bar` and `sharpe_annualised = sharpe_per_bar × sqrt(N_bars_per_year)` from the same kernel. + +### 6.2 Sub 1.2 — Cost-net sharpe + +`cost_t = commission + spread_t × position_size + ofi_impact(spread_t, ofi_t)` + +- `commission`: $1.50/RT default, `--commission` CLI overridable +- `spread_t`: real bid-ask from MBP10 fxcache (prod) OR synthetic spread from market generator (dev) +- `ofi_impact()`: linear-impact model `λ × |position_size| × |ofi_t|` where λ is ISV-tracked impact coefficient (initial: empirical fit from MES historical) +- `position_size`: contracts traded (1 for Quarter, 2 for Half, 4 for Full at base Kelly cap) + +`sharpe_net = (mean_pnl − cost_per_bar_avg) / std_pnl_with_costs` + +The cost kernel reads `spread_t` and `ofi_t` as input arrays. Dev synthetic markets generate these as part of fixtures. This is the same kernel in both environments — `feedback_isv_for_adaptive_bounds` extends to costs. + +### 6.3 Sub 1.3 — Drawdown reporting + +New ISV slots: +- `DD_CURRENT_INDEX`: peak-to-trough percentage of running cumulative PnL +- `DD_MAX_INDEX`: maximum DD seen this fold +- `DD_RECOVERY_BARS_INDEX`: bars since last DD peak (0 = currently at ATH) +- `DD_PERSISTENCE_INDEX`: bars in current drawdown ≥ threshold (used by 3.5.4) + +HEALTH_DIAG emits `max_dd_pct`, `current_dd`, `dd_recovery_bars`, `Calmar = mean_pnl / max(max_dd_pct, 1e-4)`. Denominator floor (1e-4) replaces the current Calmar=100 saturation artifact. + +### 6.4 Sub 1.4 — 8 counterfactual baselines + +Each baseline is a pure CUDA kernel that replays the val/test window with a fixed action policy and computes sharpe_net + max_dd + trade_count. All 8 run in parallel kernels per fold per epoch: + +| Baseline | Definition | +|---|---| +| `buyhold` | always Long, mag = Quarter (smallest), no exits | +| `hold_only` | always Hold, never enter | +| `random_dir_kelly` | random direction at random bars (seeded), mag pinned to policy's Kelly cap output | +| `naive_momentum` | direction = sign(last bar return), mag = Quarter | +| `aux_only` | direction = sign(aux head's next-bar prediction), mag = Quarter | +| `mag_quarter_fixed` | policy's direction, mag pinned to Quarter | +| `trail_only` | random entries (seeded), exits via policy's trail-stop logic | +| `naive_reversion` | direction = -sign(last bar return), mag = Quarter | + +HEALTH_DIAG emits per-fold deltas: `vs_buyhold = sharpe_net − buyhold_sharpe_net`, etc. + +### 6.5 Sub 1.5 — `dd_pct` as foundational state input + +This is the layout fingerprint break. + +- New ISV slot: `DD_PCT_INDEX = current_dd / max(dd_budget, 1e-4)`. `dd_budget` is config (e.g., 0.20 = 20% drawdown limit). Producer kernel reads `DD_CURRENT_INDEX` from sub 1.3. +- **Trunk concat extension**: trunk forward input grows by 1 dim. Layout fingerprint seed extended with `DD_PCT_TRUNK_INPUT`. All trunk-input consumers migrate atomically (per `feedback_no_partial_refactor`): batched_forward.rs, batched_backward.rs, gpu_dqn_trainer scratch buffer sizing, eval-mode forward path. + +Eval-time policy now SEES dd_pct on every forward. Phase 3 teachings condition on it naturally. + +### 6.6 Sub 1.6 — `--holdout-quarters` and `--dev-quarters` CLI flags + +In `bin/fxt/` and `services/ml-training/`: +- `--holdout-quarters N` (default 1): slice last N quarters off `training_data` BEFORE walk-forward generation. This is the sealed final test set. +- `--dev-quarters M` (default 1): slice next-N-from-tail quarters off, after holdout. This is the final dev set (used for hyperparameter tuning, controller anchor calibration). +- Walk-forward generates folds on `Q1..Q(9-N-M)`. With defaults: walk-forward on Q1-Q7 (3 folds at 0.5/0.125/0.125/0.125), Q8 = final dev, Q9 = sealed test. + +### 6.7 Sub 1.7 — Consume the abandoned test slice + +`set_test_data_from_slices` method on trainer + post-fold `test_sharpe_net` emit. The 12.5% `test_start..test_end` slice generated by walk-forward is now actually used: per-fold test slice metric reported separately from val. Sharpe gap between val and per-fold-test surfaces the selection-vs-test difference quantitatively for each fold. + +### 6.8 Phase 1 sequencing + +Sub 1.1 → 1.2 → 1.3 → 1.4 → 1.5 → 1.6 → 1.7. Each is one atomic commit. 1.5 (dd_pct) is the layout-fingerprint-break commit; explicitly tagged in the commit message so checkpoints from before this point are known-incompatible. + +--- + +## 7. Phase 2 — Behavioral Test Suite (~3-5 days, ~2000 LOC) + +### 7.1 Goal + +Every behavior the policy must exhibit gets a deterministic test on dev RTX 3050 Ti. Suite runs <10 min total. Pre-commit hook blocks `argo-train.sh` if any test fails. CI gate before L40S. + +### 7.2 Phase 2A — Test infrastructure (parallel to Phase 0+1) + +- **Synthetic market generators** in `crates/ml/tests/behavioral/synthetic_markets.rs`: + - `flat_market(noise_σ)` — flat price + Gaussian noise + parameterized bid-ask spread + - `drift_market(μ, σ, spread_pattern)` — geometric drift + spread parameterization + - `OU_market(θ, σ_eq, μ)` — Ornstein-Uhlenbeck process for mean-reversion + - `regime_switch_market(N_regimes, transition_matrix)` — multi-regime Markov chain + - All generators produce `(price_t, spread_t, ofi_t)` triples — feed unified cost kernel from Phase 1.2 +- **Oracle policies** — known-correct action sequences per market type. Used both as reference for test assertions and as initial-weight seeds for "policy starts informed" tests. +- **Evaluator harness** in `crates/ml/tests/behavioral/harness.rs`: runs policy on synthetic for K bars, captures action distribution + sharpe_net + max_dd + behavioral metrics, compares against test assertions. +- **Pre-commit hook** in `.claude/helpers/pre_commit_behavioral_suite.sh`: runs `cargo test -p ml --test behavioral_suite -- --nocapture`. Fail → block commit. +- **CI integration**: `argo-train.sh` invokes the same hook before submitting workflow. Local fail = no L40S deploy. + +### 7.3 21 tests by group + +#### Group 1: Trading behavior (7 tests, lands as Phase 1 cost + market gen ships) + +| # | Test | Pass criterion | +|---|---|---| +| 2.1 | `flat_market_holds` | ≥80% Hold over last 5k bars on flat synthetic | +| 2.2 | `trend_market_directional` | drift>0 → ≥70% Long; drift<0 → ≥70% Short | +| 2.3 | `mean_revert_reverses` | OU process → reverses at ±2σ within 5 bars | +| 2.4 | `cost_sensitivity` | trade freq drops ≥40% at high spread vs low spread | +| 2.6 | `regime_silences` | high-noise no-edge regime → trade freq ≤20% | +| 2.7 | `stop_discipline` | adverse excursion past trail dist → close within 1 bar | +| 2.18 | `action_latency_test` | step-function +5σ signal → policy switches Hold→Long within 5 bars | + +#### Group 2: State/architecture grounding (10 tests, lands as Phase 1 state + Phase 0 EGF ships) + +| # | Test | Pass criterion | +|---|---|---| +| 2.8 | `hold_vs_flat_semantics` | Hold preserves pre_pos; Flat zeros it | +| 2.9 | `eval_fold_isolation` | KL(shuffled-future, unshuffled-future) < 0.01 — no temporal leak in policy state | +| 2.13 | `eval_train_consistency` | KL(train-mode, eval-mode action dists) < 0.5 | +| 2.14 | `magnitude_uses_all_buckets` | each mag bucket > 5% of trade events | +| 2.15 | `fold_transition_stability` | sharpe collapse at fold boundary < 50% | +| 2.16 | `q_value_bounded_under_replay` | 1k replay updates → Q-values stay in [V_MIN, V_MAX] | +| 2.17 | `gradient_flow_to_all_heads` | non-zero grad reaches every head (dir/mag/ord/urg/aux/CQL/IQN/C51/ens) | +| 2.19 | `kelly_calibration_test` | kelly_f monotonically decreasing in Q-variance | +| 2.20 | `state_input_grounding_test` | KL(uptrend-context, downtrend-context action dists) > 0.5 | +| 2.21 | `egf_gate_opens` (Phase 0 anchor) | aux/Q deliberate disagreement → gate1 opens within 100 steps | + +#### Group 3: Recovery dynamics (4 tests, anchors Phase 3.5 teachings) + +| # | Test | Pass criterion | +|---|---|---| +| 2.5 | `drawdown_de_risks` | inject 10-bar loss streak → Kelly cap ≤0.5×base over next 100 bars | +| 2.10 | `recovery_after_streak` | inject 10R loss streak → recover ≥5R over next 200 bars (NOT collapse to all-Hold) | +| 2.11 | `dd_state_aware_sizing` | KL(0% DD, 50% dd_budget action dists) > 0.1 — policy conditions sizing on DD | +| 2.12 | `cooldown_engagement` | K=5 losses → cooldown fires within 1 bar; disengages cleanly after M=20 bars | + +### 7.4 Sequencing within Phase 2 + +Three atomic commit batches: +- **2A: scaffolding** (synthetic gens + oracle + harness) — first +- **2B: Groups 1+2** (17 tests) — once Phase 0+1 ships; gates Phase 3 entry +- **2C: Group 3** (4 tests) — written alongside Phase 3.5 teachings; each test commit paired with its teaching commit per the discipline rule + +### 7.5 Discipline gate enforcement + +Every Phase 3 / 3.5 commit cites which Group 1/2/3 test it makes green. If teaching X lands but `2.X` doesn't go green, teaching reverts. No "we'll fix the test later" exceptions. + +--- + +## 8. Phase 3 — Trader Teaching (~3-5 days, atomic per teaching, sequential) + +### 8.1 Goal + +Reward shape and action selection teach the trader role, not just optimize a number. + +**SP12 v3 (per-trade event-driven reward) stays.** Phase 3 builds ON SP12, doesn't replace it. SP12's per-event asymmetric pos/neg cap (5.0 / -10.0) and zero per-bar dense shaping are preserved. + +### 8.2 The 5 teachings + +#### 3.1 — r_quality + r_discipline split + +Architectural framing, no separate code beyond the convex combination: + +``` +r_total = α × r_quality_per_event + (1 − α) × r_discipline_per_bar +``` + +- `r_quality_per_event` = SP12 per-trade event-driven reward (UNCHANGED) +- `r_discipline_per_bar` = 3.4 regret signal (NEW) +- α is a fixed config constant (default 0.7), NOT learned, NOT controller-modulated. Per `feedback_no_feature_flags`: no `enable_*` toggle. + +**Anchor tests**: 2.4 (cost_sensitivity), 2.6 (regime_silences). Both should be made simultaneously achievable by the split. + +#### 3.2 — Explicit cost in r_quality + +``` +r_quality_per_event −= cost_t × trade_attempt_indicator +``` + +`cost_t` is the same Phase 1.2 unified kernel (commission + spread + OFI-impact). Model SEES the bill in the gradient signal during training, not just in eval-time metrics. + +**Anchor test**: 2.4 (cost_sensitivity). Policy must trade less under high-spread regime. + +#### 3.3 — Quadratic drawdown penalty + +``` +r −= λ_dd × max(0, current_dd − dd_threshold)² +``` + +- Asymmetric quadratic: zero penalty until dd crosses threshold, then quadratic growth. Encodes loss aversion per `pearl_audit_unboundedness_for_implicit_asymmetry`. +- `dd_threshold`: ISV-driven from running variance of dd (initial: 0.05 = 5% of capital, retunes to per-fold scale). +- `λ_dd`: ISV-driven from grad-balance ratio between r_total and r_quality (so DD term doesn't dominate gradient). + +**Anchor test**: 2.5 (drawdown_de_risks). Policy must reduce position size after loss streak. + +#### 3.4 — Regret signal = r_discipline content + +When the policy holds and a trade WOULD have been profitable past `cost_t + trail_dist`, log `regret_bar = ideal_pnl_missed − cost_t`. + +``` +r_discipline_per_bar = −λ_regret × regret_ema +``` + +- `regret_ema`: rolling EMA of `regret_bar` (Wiener-α adaptive smoothing per `pearl_wiener_optimal_adaptive_alpha`). +- `λ_regret`: ISV-driven from a target ratio between regret-derived gradient and r_quality gradient. + +Teaches "Hold when no edge: fine. Hold when edge exists: bad." + +**Anchor test**: 2.6 (regime_silences). Policy must trade in high-edge regimes, NOT silence in them. + +#### 3.5 — Confidence-aware Hold floor + +Action-selection level (NOT reward shaping): + +``` +hold_floor = monotonic_inv_func(entropy(dir_Q_distribution)) +``` + +When entropy is high (low confidence) → bias toward Hold by adding `+hold_floor` to the Hold-Q before argmax/Thompson selection. Hold becomes uncertainty expression, not distributional default. + +`monotonic_inv_func` parameters (slope, anchor) are ISV-driven from running entropy distribution. + +**Anchor tests**: 2.1 (flat_market_holds), 2.6 (regime_silences). Hold must dominate when dir entropy is high. + +### 8.3 Phase 3 sequencing + +Sequential. Each teaching's commit asserts (a) its anchor test goes green AND (b) prior anchor tests still pass. If 3.3 breaks 2.4, 3.3 reverts. + +--- + +## 9. Phase 3.5 — Recovery Dynamics (~2-3 days, atomic per mechanism, sequential) + +### 9.1 Goal + +Model can escape its own losses — break the spiral observed in train-dd4xl. + +**3.5.1 (dd_pct as state input) already landed in Phase 1.5** per Q1. Phase 3.5 has 4 remaining mechanisms. + +### 9.2 The 4 mechanisms + +#### 3.5.2 — Asymmetric reward under DD + +``` +For gains: r_adjusted = r × (1 + λ × dd_pct) +For losses: r_adjusted = r (unchanged) +``` + +- `λ` ≈ 0.5 (ISV-tracked from observed DD distribution) +- Layered on top of SP12 asymmetric pos/neg caps. Composes cleanly because SP12 is event-level value cap; this is gain-side multiplier conditioned on DD state. + +Teaches "a winning trade out of DD is worth more than the same trade at ATH." Matches actual trading P&L psychology — recovering from drawdown is harder than maintaining ATH, so recovery gains deserve more reward signal. + +**Anchor test**: 2.10 (recovery_after_streak). + +#### 3.5.3 — Cooldown gate + +After K consecutive losing trades, force Hold for M bars. Both K and M are ISV-driven: +- K: tracked from running variance of trade outcomes (cooldown trips when streak exceeds typical loss-cluster length) +- M: tracked from time-to-mean-reversion of vol_normalizer signal (cooldown lasts until regime stabilises) + +**Cooldown counter is part of state** — model can reason about it, not work around it. New ISV slot `COOLDOWN_BARS_REMAINING_INDEX`. Producer decrements per bar; consumer (action selection) forces Hold while > 0. + +**Anchor test**: 2.12 (cooldown_engagement). + +#### 3.5.4 — Plasticity injection during persistent drawdown + +When `dd_persistence_bars > threshold` (Phase 1.3 ISV slot), reset last 10% of advantage-head weights to Xavier-initialised values to escape local minima. + +- Trigger fires at most once per fold (debounced via additional ISV slot `PLASTICITY_INJECTION_FIRED_INDEX`). +- Threshold is ISV-driven from running mean of `dd_persistence_bars` over training history. +- Reset target: last 10% of advantage-head weights (NOT shared trunk, NOT direction Q-head fully). Affects only the magnitude/ordinal/urgency advantage tails where local minima are most likely. + +Mimics real-trader "step away & restart" — the most aggressive recovery mechanism. Most research-y; gated strictly on 2.10 staying green. + +**Anchor test**: 2.10 (recovery_after_streak) AND must NOT regress 2.16 (q_value_bounded_under_replay) — plasticity reset cannot create unbounded Q values. + +#### 3.5.5 — Recovery curriculum in PER + +When sampling from PER, **oversample episodes where policy started in drawdown and recovered to flat** (`dd > 0.05` at start, `dd < 0.01` at end). Provides positive recovery examples for credit assignment. + +- Oversample factor: ISV-driven from current-DD context (more DD now → more weight on recovery episodes). +- Currently PER oversamples loss states (high TD-error), which compounds the spiral. This counter-balances by giving "I survived a similar drawdown" examples more weight. + +**Anchor test**: 2.10 (recovery_after_streak). + +### 9.3 Phase 3.5 sequencing + +Per Q7: **all 4 mechanisms land**. No defer of 3.5.4. + +``` +Phase 2B green → 3.5.2 → 3.5.3 → 3.5.4 → 3.5.5 + ↓ each gated on anchor test +``` + +3.5.4's gate is strict: must not regress 2.10 OR 2.16. If it does, reverts and we ship 3.5 with only 3.5.2 + 3.5.3 + 3.5.5. + +--- + +## 10. Phase 4 — Final Validation (~2-3 days) + +### 10.1 Goal + +Produce trustworthy "this trained system is/isn't a trader" signal. Phase 4 is the only phase that touches the sealed Q9. + +### 10.2 Sub 4.1 — Pre-flight (~½ day) + +- All 21 Phase 2 tests green on dev (RTX 3050 Ti). Pre-commit hook blocks if not. +- Per-pearl audit doc updated, MEMORY.md index extended for any new pearls landed. +- New worktree `sp15-final-validation` for the deployment artifacts. + +### 10.3 Sub 4.2 — Walk-forward training (~1 day wall) + +``` +argo-train.sh \ + --branch sp15-final-validation \ + --commit \ + --holdout-quarters 1 \ + --dev-quarters 1 \ + --seeds 3 \ + --epochs 30 +``` + +- 3 seeds × 3 walk-forward folds on Q1-Q7. Q8 used as final dev (early-stop, Kelly warmup, controller anchors legitimately read from Q8 — it IS the dev set now, not the test set). +- HEALTH_DIAG emits `sharpe_net` + `Calmar` + 8 baseline deltas per fold per epoch. +- **Hard kill criterion** (per `feedback_kill_runs_on_anomaly_quickly`): if any fold sharpe_net trends down 3 epochs straight AND below all 8 baselines, terminate workflow + diagnose which Group 1/2/3 test failed in the wild. +- Best checkpoint per seed/fold saved to `trained-models-pvc` (task #30 dependency — needs landing first or as Phase 4 sub-step). + +### 10.4 Sub 4.3 — Sealed Q9 OOS evaluation (~½ day) + +- **Separate Argo workflow** `scripts/argo-eval-final.sh` — model-loaded eval-only, never touches training. Cannot accidentally use Q9 during a misconfigured training run. +- 3 seeds × Q9 evaluation. Report mean ± std for each metric. +- **Q9 burn policy**: each Q9 run generates `docs/runs/YYYY-MM-DD-sp15-q9-.md` audit file. Q9 is "burned" when run; subsequent runs against meaningfully-different architecture are allowed but each is logged. Clearly NOT to be used for hyperparameter tuning. + +### 10.5 Sub 4.4 — Report card + +Generated automatically by Phase 4.3 workflow: + +``` +SP15 Q9 Final OOS Report — commit +───────────────────────────────────────── +Per-fold (Q1-Q7 walk-forward, Q8 final dev): + fold 0: sharpe_net=X Calmar=Y trades=N dd_max=Z + fold 1: ... + fold 2: ... + Q8 dev: ... + +Q9 Sealed OOS (3-seed mean ± std): + sharpe_net = X.X ± σ + Calmar = Y.Y ± σ + trade_count = N ± σ + max_dd_pct = Z.Z ± σ + recovery_bars = R ± σ + +Deltas vs 8 baselines (Q9): + vs buyhold : +X.X (need > 0) + vs hold-only : +X.X (need > 0) + vs random_dir : +X.X (need > 0) + vs naive_momentum : +X.X (need > 0) + vs aux-only : +X.X (need > 0) + vs mag-Q-fixed : +X.X (need > 0) + vs trail-only : +X.X (need > 0) + vs naive_revert : +X.X (need > 0) + +Behavioral test status (final, on dev): + 21/21 passing + +Pearls landed: +Architectural changes: dd_pct as state, cost-net unified, 8 baselines, Q9 sealed +``` + +### 10.6 Sub 4.5 — Production-track gate + +**Pass criteria**: +1. Q9 `sharpe_net > 1.0` +2. Q9 `Calmar > 1.0` +3. Q9 beats ALL 8 baselines on net sharpe (delta > 0 for each) +4. All 21 Phase 2 tests still pass post-deployment + +**Fail mode → behavioral test mapping**: + +| Failure | Targeted test | Action | +|---|---|---| +| sharpe_net low | 2.4, 2.6 | Cost-aware trading / regime appropriateness broken | +| Calmar low | 2.5, 2.10 | Drawdown management or recovery broken | +| loses to hold-only | 2.6 | Model overtrading garbage | +| loses to aux-only | architecture | Q-network not earning keep over aux head — consider simplification | +| loses to mag-Q-fixed | 2.14 | Magnitude head dead silicon | +| loses to trail-only | architecture | Entry logic not adding value over random+disciplined exit | + +NOT a generic re-train. Each failure axis has a specific behavioral test to target. + +--- + +## 11. Cumulative scope + +| Phase | LOC | Days serial | Key artifacts | +|---|---|---|---| +| 0 | ~400 | 1-2 | EGF retune kernels, 4 ISV slots, test 2.21 | +| 1 | ~700 | 2-3 | sharpe kernel, cost kernel, 8 baselines, dd_pct trunk concat, CLI flags | +| 2 | ~2000 | 3-5 | Synthetic markets, harness, 21 tests, pre-commit hook | +| 3 | ~800 | 3-5 | 5 teachings (reward + Hold floor) | +| 3.5 | ~700 | 2-3 | 4 recovery mechanisms | +| 4 | ~300 | 2-3 | argo-eval-final.sh, report card automation | +| **Total** | **~4900** | **13-21** | | + +With Approach B parallel dispatch on Phases 0+1+2A: ~10-15 days realistic. + +--- + +## 12. Risks and open questions + +### 12.1 3.5.4 plasticity injection is research-y + +No published results in trading RL for "reset weights mid-training to escape local minima." Risk: mistuned trigger destroys good learning. Mitigation: +- Strict gate on 2.10 staying green +- Strict gate on 2.16 (q_value_bounded) NOT regressing — plasticity must not create unbounded values +- Debounced fire (at most once per fold) + +### 12.2 8 baselines compute cost + +Each baseline runs as a separate kernel against the val/test window. 8× the eval cost per fold per epoch. Mitigation: +- Baselines are pure forward (no backprop), much cheaper than training step +- Run baselines only once per fold per epoch (not per step) +- Estimated overhead: ~5-10% of total eval cost + +### 12.3 Q9 burn policy + +"Each Q9 run generates audit file" relies on discipline, not enforcement. A motivated developer could run Q9 eval N times and only commit the best. Mitigation: +- Q9 run requires a tagged commit — `argo-eval-final.sh` rejects un-tagged or amended commits +- Audit log includes counter incrementing per architectural milestone — anomalous gaps surface gaming + +### 12.4 Synthetic LOB fidelity for Phase 2 + +Phase 2 synthetic markets generate synthetic spread + OFI to match the Phase 1.2 unified cost kernel's signature. Risk: synthetic LOB doesn't match real LOB statistics → Phase 2 tests pass but L40S behavior diverges. + +Mitigation: +- Calibrate synthetic LOB to match MES historical first-moments (mean spread, OFI distribution) on Q1-Q7 training data +- Add Phase 2 test "cost_consistency_check": run policy on (synthetic LOB, real LOB) — KL divergence < 0.3 +- This is the same kind of dev/prod parity discipline as the unified cost kernel itself + +### 12.5 Discipline rule enforcement + +"No new pearl without behavioral test" relies on commit-time review. Mitigation: +- Pre-commit hook checks new ISV slots against `docs/dqn-wire-up-audit.md` entries +- Plan reviews in PRs reference the discipline rule explicitly +- If violation slips through, it's reverted, not patched + +### 12.6 Phase 3.1 "architectural split" is fuzzy + +3.1 is described as "no separate code beyond the convex combination." Risk: it doesn't land as a distinguishable commit, gets merged into 3.4. Mitigation: +- 3.1 lands as a single explicit commit that introduces the `α × r_quality + (1-α) × r_discipline` combine point with `r_discipline = 0.0` default. 3.4 then populates `r_discipline` with regret content. +- If 3.1 ends up indistinguishable from 3.4 in implementation, accept that and merge into a single 3.1+3.4 atomic commit. The discipline rule still applies (anchor tests 2.4, 2.6 must go green). + +### 12.7 Layout fingerprint break in Phase 1.5 + +`dd_pct` trunk concat invalidates pre-SP15 checkpoints. Per Q1 ("greenfield, checkpoint compatibility doesn't matter") this is acceptable — but it does mean any in-flight L40S work prior to SP15 cannot resume from those checkpoints. + +Mitigation: +- Phase 1.5 commit message tagged with explicit warning +- `argo-train.sh` rejects loading pre-SP15 checkpoints (layout fingerprint mismatch hard error) + +--- + +## 13. Anticipated memory pearls + +To be authored alongside the implementation phases: + +| Pearl | Origin | +|---|---| +| `pearl_dd_pct_foundational_state` | Phase 1.5 — DD-context as eval-time state input vs reward-only modulation | +| `pearl_unified_cost_kernel_dev_prod_parity` | Phase 1.2 — same cost kernel reads spread/OFI from synthetic (dev) or real (prod) LOB | +| `pearl_behavioral_test_gates_l40s_deploy` | Phase 2 — pre-commit hook + dev-runtime test suite is the L40S filter | +| `pearl_recovery_curriculum_in_per` | Phase 3.5.5 — PER weighting can encode "recovery exists" signal explicitly | +| `pearl_plasticity_injection_for_rl_local_minima` | Phase 3.5.4 — weight-reset mechanism for breaking spiral attractors (if 2.10 verifies) | +| `pearl_8_counterfactual_baselines_arch_specific` | Phase 1.4 — domain baselines that test specific subsystems we built (aux-only, mag-Q-fixed, trail-only) | +| `pearl_egf_isv_driven_anchors` | Phase 0.B — Schmitt thresholds and variance refs as ISV slots, not constants | + +--- + +## 14. Discipline rule (re-stated for posterity) + +**No new pearl, controller, kernel, ISV slot, or reward term ships without:** +1. A Phase 2 behavioral test that proves the intended trader behavior +2. A wire-up audit entry +3. A memory pearl if the technique generalises + +Violations mean the change is reverted, not merged-with-followup. + +This rule applies to every commit on the `sp15-trader-discipline-recovery` branch and every commit that merges back to main. It supersedes prior tolerance for "land it now, behavioral validation later." There is no later.