diff --git a/docs/dqn-wire-up-audit.md b/docs/dqn-wire-up-audit.md index 22ce21a60..ee4a03442 100644 --- a/docs/dqn-wire-up-audit.md +++ b/docs/dqn-wire-up-audit.md @@ -16120,3 +16120,93 @@ larger separations as model concentrates wins. The PER alpha boost path (via `WINNER_CONCENTRATION_INDEX` → `per_update_pa`) finally has a non-zero signal to amplify on volume-bar policies regardless of profitability state. + +## 2026-05-12 — SP21 T2.2 close-out + v10 hypothesis-test findings + +### SP21 T2.2 status: COMPLETE (cascade wiring) + +The cascade plumbing is now feature-complete and validated: + +| Concern | Status | Validated by | +|---|---|---| +| Per-trade tape (Phase 1) | wired | v5+ (cycles emit) | +| Enrichment real-data wireup (Phase 2) | wired | v5+ | +| E1 q_correction → Bellman (Phase 3) | wired | v5+ (`q_corr` non-zero) | +| E4 branch_lr_scale → per-branch Adam (Phase 4) | wired | v5+ (`branch_lr` heterogeneous) | +| Pearl C engagement (Phase 4.5) | wired | v5+ (multi-range slot writes) | +| E6+E7 PER alpha scaling (Phase 5+6) | wired | v5+ (kernel reads) | +| Hindsight synthetic injection (Phase 6.5) | wired | v5+ | +| E8 curriculum → PER alpha (Phase 7) | wired | v5+ | +| E8 per-segment PER insert boost (Phase 7.5) | wired | v5+ | +| Signal-driven E2/E5 gains (Phase 8) | wired | v5+ | +| Signal-driven E5 clamp bounds (Phase 8.1) | wired | v5-v9 (`agree_thr` past 10.0) | +| `pnl_std` data-scale anchors (Phase 8.2) | wired | v6+ (1.08e-5) | +| `hindsight_mag` scientific display (Phase 8.4) | wired | v8+ | +| Curriculum z-score `exp(-sharpe)` (Phase 8.6) | wired | v9 (`curric_conc=0.31`) | +| Winner-conc z-score separation (Phase 8.4-fix) | wired | v9 (`win_conc=1.72`) | +| Return v3.1 unconditional CAGR | wired | v5+ | +| Eval pipeline (Phase 8.3+9, 8.5, 8.7) | wired + local-test verified | compute-sanitizer 0 errors | +| Default ISV buffer (Phase 8.7) | wired | local test pass | + +### What we tried to validate that we didn't + +- **v9** terminated early to free GPU for the v10 hypothesis test +- The full 3-fold eval phase Phase 8.7 validation is incomplete on + production data; the local compute-sanitizer test exercises the same + code path with 0 errors, but full-fold-on-real-data run is deferred + +### v10 hypothesis test: "does longer training fix WR=50%?" + +Dispatched a 10-epoch baseline run on HEAD `d482efc41` (full SP21 +cascade) to test whether train Sharpe / WR / PF recover past the +E2-E3 overfit dip with more training time. Killed at E8 (clear answer). + +**Fold 0 trajectory** (val_Sharpe per epoch): + +``` +E1: 161 | E2: 161 | E3: 174 ← peak | E4: 153 | E5: 124 +E6: 104 | E7: 77 | E8: 66 +``` + +**Findings:** + +1. **val_Sharpe peaks at E3, monotonically degrades thereafter** + (174 → 66, -62% across 5 epochs). The 3-epoch baseline structure + is correct — that's exactly where the model is best. + +2. **WR / PF are structurally pinned at ~50.1% / 1.00 across ALL + 8 epochs**, regardless of training time, regardless of cascade + controller activity. This is NOT an undertraining problem. + +3. **Policy degrades monotonically in trade volume** (492K → 402K) + and the "top decile" dissolves (winners: 20000 → 3051). The + cascade signals (win_conc=1.66-1.75, curric_conc=0.28-0.41) stay + stable throughout — wiring is fine, model just has nothing + meaningful to amplify. + +4. **The Q-function converges to non-informative outputs**: train + Sharpe sign-flipping wildly (+70 → -45), Return clamp-pinning at + `+4.852e10%` through E7 (max-overfit), then dropping to `+5.7e8%` + at E8 as the policy contracts. + +**Verdict:** SP21 T2.2 cascade wiring is correct and signals are +alive, but cannot move WR past 50.1-50.2% on its own. The plateau +is upstream of the cascade — at the model's ability to learn +directional signal at all. + +### Implications for SP20+ goals (`project_goal_wr_55_pf_2`) + +Cannot be reached by further cascade tuning. The next problem to +solve is the WR=50% plateau, which is binding regardless of cascade +configuration. New SP arc needed: + +- **Plan**: `docs/plans/2026-05-12-sp22-wr-plateau-investigation.md` +- **Hypotheses to test** (in priority order): + 1. Label horizon mismatch (`project_label_horizon_is_bottleneck`) + 2. Action space pathology (`project_hold_hides_directional_signal_hypothesis`) + 3. Reward shape — current reward doesn't produce directional gradient + 4. Feature representation — bar-resolution feature set missing + timing/directional info +- **Validation method**: each hypothesis gets its own atomic smoke + with one variable changed; compare against v9 baseline (174 peak val, + WR=50.1%). diff --git a/docs/plans/2026-05-12-sp22-wr-plateau-investigation.md b/docs/plans/2026-05-12-sp22-wr-plateau-investigation.md new file mode 100644 index 000000000..55f475ab1 --- /dev/null +++ b/docs/plans/2026-05-12-sp22-wr-plateau-investigation.md @@ -0,0 +1,168 @@ +# SP22 — WR Plateau Investigation + +**Author**: jgrusewski + assistant pair +**Date**: 2026-05-12 +**Branch**: TBD (off `sp20-aux-h-fixed` after merge) +**Predecessor**: SP21 T2.2 (cascade wiring; closed `docs/dqn-wire-up-audit.md` 2026-05-12) +**Goal**: Move WR from the current ~50.1% plateau toward `project_goal_wr_55_pf_2`'s WR≥55% / PF≥2.0. + +## State at session start + +SP21 T2.2 cascade is feature-complete: + +``` +d482efc41 fix(sp21): T2.2 Phase 8.4-fix — winner_conc z-score separation +9f2d0fffb fix(sp21): T2.2 Phase 8.7 — default ISV buffer +518670198+ ... Phase 8.x stack ... +2937da889 fix(sp21): Return v3 (later v3.1) +``` + +All E1-E8 enrichment producers wired to consumers. Local compute-sanitizer +clean on eval-baseline closure path. v9 cycle 1 confirms cascade live: +`win_conc=1.72`, `curric_conc=0.31`, `hindsight_mag=1.47e-5`. + +**Binding constraint** (v10 hypothesis test, 8 epochs, fold 0): +- val_Sharpe peaks at E3 (174), monotonically degrades to E8 (66, -62%) +- WR pinned at 50.1-50.2% across ALL 8 epochs +- PF pinned at 1.00-1.01 +- Trade volume contracts as policy degrades (492K → 402K) +- Cascade controllers stable but cannot move the policy off the plateau + +## Hypotheses + +Each hypothesis gets its own atomic experiment. Compare against the v9 +baseline (3-epoch, full cascade, fold 0 peak val=174, WR=50.1%). + +### H1: Label horizon mismatch (`project_label_horizon_is_bottleneck`) + +**Theory**: The model predicts next-bar return but the actual directional +signal lives at a different time scale (e.g., 5-bar, 30-bar lookback in +volume bars ≈ 4-25 seconds). With volume bars firing every ~0.78s at +34_496 bars/day, single-bar prediction is dominated by micro-noise. + +**Test**: Modify the training reward (and aux-head label if applicable) +to use N-bar forward return for N ∈ {5, 20, 60} bars. Compare WR/PF +peak across the three settings. + +**Predicted outcome if H1 is true**: WR jumps above 50.5% at N=20 or N=60. + +**Predicted outcome if H1 is false**: WR stays at 50.1-50.2% across all N. + +**Cost**: 3 smokes × ~30 min each on L40S = 90 min wall-clock. + +### H2: Action-space pathology (`project_hold_hides_directional_signal_hypothesis`) + +**Theory**: The 4-direction action space (Short/Hold/Long/Flat) collapses +directional signal. Hold (keep position) and Flat (close position) both +register as "no-op" in metrics but differ structurally — the model may +be learning to predict Hold/Flat as a coward strategy that breaks even +without committing direction. + +**Test**: Run with Hold action **disabled** (force the model to choose +between Short, Long, Flat each bar). If WR ≥ 50.1% with no Hold, +directional learning is structurally suppressed by Hold. If WR drops +significantly below 50% without Hold, Hold is the right thing and the +plateau is elsewhere. + +**Cost**: One smoke + one comparison run. ~60 min. + +**Caveat**: requires action-space change in training config + state-layout +fields. Larger code change than H1. + +### H3: Reward shape doesn't produce directional gradient + +**Theory**: Current reward is some combination of per-bar return, +trade-cost penalty, hold-time bonus etc. (per `pearl_reward_shape_drives_quality_over_quantity`). +With WR=50% and PF=1.0, the reward gradient may be too noisy to teach +direction — model finds break-even policies more easily than directional +ones. + +**Test**: Two variants: +1. **Pure directional reward**: `reward = sign(next_return) * |action_dir|` + (binary correct/incorrect signal, no magnitude). +2. **Asymmetric loss-aversion** (per `pearl_audit_unboundedness_for_implicit_asymmetry`): + amplify losing trades 1.5× to penalize indecisive HOLD-as-default. + +**Predicted outcome if H3 is true**: variant 1 either rises (model learns +direction without magnitude confounders) or collapses (magnitude noise +was actually helping); variant 2 forces model out of indecision. + +**Cost**: 2 smokes × 30 min each = 60 min. + +### H4: Feature representation gap + +**Theory**: At 34_496 bars/day on volume bars, the feature set +(42-dim market features from `extract_ml_features`) may not capture +multi-bar context needed for directional prediction. MTF features +exist in state layout (16 dims at indices 90-106) but eval-baseline +feeds zeros for these. + +**Test**: Train with **real MTF features computed** (not zero-padded). +If WR rises ≥ 0.5%, MTF carries the directional signal. If not, MTF +is also noise at this resolution. + +**Cost**: Feature pipeline change + one smoke. Higher engineering cost +than H1-H3. + +### H5 (longshot): Bar-resolution itself is too noisy + +**Theory**: 34_496 bars/day = 0.78s/bar is below the timescale where +directional signal exists in ES.FUT order flow. Per +`project_bar_resolution_is_actual_architecture`, the "hybrid tick→bar" +architecture is aspirational not built. Maybe the model needs longer +bars to find signal. + +**Test**: Switch to time bars (e.g., 1-min or 5-min) and re-train. +Compare WR/PF against volume-bar baseline. + +**Cost**: Significant — different data ingestion path, retrain from +scratch, may need hyperparam re-tuning. Longest experiment. + +## Experiment order + +Run in this priority order, killing early on each as soon as the +hypothesis is conclusively resolved: + +1. **H1 (label horizon)** — cheapest, addresses the most likely root cause +2. **H3 (reward shape)** — cheap, complementary to H1 +3. **H2 (Hold disable)** — moderate cost +4. **H4 (MTF features)** — higher engineering cost +5. **H5 (bar resolution)** — last resort, biggest cost + +Stop after the first hypothesis that produces WR ≥ 51% as a clear signal. + +## Success criteria + +**Phase 1 milestone**: WR ≥ 51% on val window for ≥ 1 fold (any hypothesis). +**Phase 2 milestone**: WR ≥ 53% sustained across 3 folds. +**Phase 3 milestone**: WR ≥ 55% AND PF ≥ 2.0 (the SP20+ goal). + +Each milestone gates the next phase. Phase 1's hypothesis becomes the +core architectural change going into Phase 2. + +## Non-goals (out of scope for SP22) + +- Cascade controller tuning (SP21 T2.2 is closed) +- Eval pipeline architecture (Phase 8.3-8.7 wraps it; OFI/TLOB/MTF + fidelity is a separate deferred concern from SP21) +- Hyperopt — premature until the WR plateau is broken +- Multi-symbol generalization — single-symbol (ES.FUT) until plateau + resolves + +## Risk: confounded signal from multiple changes + +The cascade is **active** — its controllers will react to whatever +training-side change we make. If WR jumps, distinguishing "cascade +amplification of new signal" from "the new signal alone" requires +the cascade-off comparison. + +**Mitigation**: For any hypothesis that produces WR ≥ 51%, immediately +run a comparison smoke with cascade disabled (set all enrichment +producers to constant defaults). If WR holds → signal is real. If WR +drops back to 50% → cascade was amplifying the signal (expected and +desirable, but flag for the writeup). + +## Audit pointer + +Findings + commit log: `docs/dqn-wire-up-audit.md` (continue appending +under `## 2026-05-12 — SP22 ...` headers).