docs(sp21): close-out + v10 findings + SP22 WR plateau plan
SP21 T2.2 status: cascade wiring COMPLETE.
- All E1-E8 enrichment producers wired to consumers
- Local compute-sanitizer clean on eval-baseline closure path
- v9 cycle 1 confirms cascade live: win_conc=1.72, curric_conc=0.31,
hindsight_mag=1.47e-5
- Eval pipeline now hard-fails on GPU error, no CPU fallback,
factored-action branch sizes + default ISV all wired
v10 hypothesis test (10 epochs, killed at E8 with clear answer):
- val_Sharpe peaks at E3 (174), monotonically degrades to E8 (66, -62%)
- WR pinned at 50.1-50.2% across ALL 8 epochs (no improvement)
- PF pinned at 1.00-1.01
- Cascade controllers stable but cannot move policy off the plateau
- Verdict: 3-epoch baseline structure IS optimal; longer training
monotonically degrades. WR plateau is upstream of the cascade.
Implication: SP21 T2.2 cascade is necessary but insufficient for
project_goal_wr_55_pf_2 (WR≥55%, PF≥2.0). Need new SP arc to address
the WR=50% plateau at its source.
New plan: docs/plans/2026-05-12-sp22-wr-plateau-investigation.md
Hypotheses (priority order):
H1: Label horizon mismatch (cheapest, most likely)
H2: Action-space pathology (Hold hiding directional signal)
H3: Reward shape (no directional gradient)
H4: Feature representation gap (MTF features zero-padded)
H5: Bar resolution itself is too noisy (longshot)
Each hypothesis tested as atomic smoke with one variable changed
vs v9 baseline (peak val=174, WR=50.1%).
Phase 1 milestone: WR ≥ 51% on val for ≥ 1 fold.
Phase 2: WR ≥ 53% across 3 folds.
Phase 3: WR ≥ 55% AND PF ≥ 2.0 (SP20+ goal achieved).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
@@ -16120,3 +16120,93 @@ larger separations as model concentrates wins.
|
||||
The PER alpha boost path (via `WINNER_CONCENTRATION_INDEX` →
|
||||
`per_update_pa`) finally has a non-zero signal to amplify on
|
||||
volume-bar policies regardless of profitability state.
|
||||
|
||||
## 2026-05-12 — SP21 T2.2 close-out + v10 hypothesis-test findings
|
||||
|
||||
### SP21 T2.2 status: COMPLETE (cascade wiring)
|
||||
|
||||
The cascade plumbing is now feature-complete and validated:
|
||||
|
||||
| Concern | Status | Validated by |
|
||||
|---|---|---|
|
||||
| Per-trade tape (Phase 1) | wired | v5+ (cycles emit) |
|
||||
| Enrichment real-data wireup (Phase 2) | wired | v5+ |
|
||||
| E1 q_correction → Bellman (Phase 3) | wired | v5+ (`q_corr` non-zero) |
|
||||
| E4 branch_lr_scale → per-branch Adam (Phase 4) | wired | v5+ (`branch_lr` heterogeneous) |
|
||||
| Pearl C engagement (Phase 4.5) | wired | v5+ (multi-range slot writes) |
|
||||
| E6+E7 PER alpha scaling (Phase 5+6) | wired | v5+ (kernel reads) |
|
||||
| Hindsight synthetic injection (Phase 6.5) | wired | v5+ |
|
||||
| E8 curriculum → PER alpha (Phase 7) | wired | v5+ |
|
||||
| E8 per-segment PER insert boost (Phase 7.5) | wired | v5+ |
|
||||
| Signal-driven E2/E5 gains (Phase 8) | wired | v5+ |
|
||||
| Signal-driven E5 clamp bounds (Phase 8.1) | wired | v5-v9 (`agree_thr` past 10.0) |
|
||||
| `pnl_std` data-scale anchors (Phase 8.2) | wired | v6+ (1.08e-5) |
|
||||
| `hindsight_mag` scientific display (Phase 8.4) | wired | v8+ |
|
||||
| Curriculum z-score `exp(-sharpe)` (Phase 8.6) | wired | v9 (`curric_conc=0.31`) |
|
||||
| Winner-conc z-score separation (Phase 8.4-fix) | wired | v9 (`win_conc=1.72`) |
|
||||
| Return v3.1 unconditional CAGR | wired | v5+ |
|
||||
| Eval pipeline (Phase 8.3+9, 8.5, 8.7) | wired + local-test verified | compute-sanitizer 0 errors |
|
||||
| Default ISV buffer (Phase 8.7) | wired | local test pass |
|
||||
|
||||
### What we tried to validate that we didn't
|
||||
|
||||
- **v9** terminated early to free GPU for the v10 hypothesis test
|
||||
- The full 3-fold eval phase Phase 8.7 validation is incomplete on
|
||||
production data; the local compute-sanitizer test exercises the same
|
||||
code path with 0 errors, but full-fold-on-real-data run is deferred
|
||||
|
||||
### v10 hypothesis test: "does longer training fix WR=50%?"
|
||||
|
||||
Dispatched a 10-epoch baseline run on HEAD `d482efc41` (full SP21
|
||||
cascade) to test whether train Sharpe / WR / PF recover past the
|
||||
E2-E3 overfit dip with more training time. Killed at E8 (clear answer).
|
||||
|
||||
**Fold 0 trajectory** (val_Sharpe per epoch):
|
||||
|
||||
```
|
||||
E1: 161 | E2: 161 | E3: 174 ← peak | E4: 153 | E5: 124
|
||||
E6: 104 | E7: 77 | E8: 66
|
||||
```
|
||||
|
||||
**Findings:**
|
||||
|
||||
1. **val_Sharpe peaks at E3, monotonically degrades thereafter**
|
||||
(174 → 66, -62% across 5 epochs). The 3-epoch baseline structure
|
||||
is correct — that's exactly where the model is best.
|
||||
|
||||
2. **WR / PF are structurally pinned at ~50.1% / 1.00 across ALL
|
||||
8 epochs**, regardless of training time, regardless of cascade
|
||||
controller activity. This is NOT an undertraining problem.
|
||||
|
||||
3. **Policy degrades monotonically in trade volume** (492K → 402K)
|
||||
and the "top decile" dissolves (winners: 20000 → 3051). The
|
||||
cascade signals (win_conc=1.66-1.75, curric_conc=0.28-0.41) stay
|
||||
stable throughout — wiring is fine, model just has nothing
|
||||
meaningful to amplify.
|
||||
|
||||
4. **The Q-function converges to non-informative outputs**: train
|
||||
Sharpe sign-flipping wildly (+70 → -45), Return clamp-pinning at
|
||||
`+4.852e10%` through E7 (max-overfit), then dropping to `+5.7e8%`
|
||||
at E8 as the policy contracts.
|
||||
|
||||
**Verdict:** SP21 T2.2 cascade wiring is correct and signals are
|
||||
alive, but cannot move WR past 50.1-50.2% on its own. The plateau
|
||||
is upstream of the cascade — at the model's ability to learn
|
||||
directional signal at all.
|
||||
|
||||
### Implications for SP20+ goals (`project_goal_wr_55_pf_2`)
|
||||
|
||||
Cannot be reached by further cascade tuning. The next problem to
|
||||
solve is the WR=50% plateau, which is binding regardless of cascade
|
||||
configuration. New SP arc needed:
|
||||
|
||||
- **Plan**: `docs/plans/2026-05-12-sp22-wr-plateau-investigation.md`
|
||||
- **Hypotheses to test** (in priority order):
|
||||
1. Label horizon mismatch (`project_label_horizon_is_bottleneck`)
|
||||
2. Action space pathology (`project_hold_hides_directional_signal_hypothesis`)
|
||||
3. Reward shape — current reward doesn't produce directional gradient
|
||||
4. Feature representation — bar-resolution feature set missing
|
||||
timing/directional info
|
||||
- **Validation method**: each hypothesis gets its own atomic smoke
|
||||
with one variable changed; compare against v9 baseline (174 peak val,
|
||||
WR=50.1%).
|
||||
|
||||
168
docs/plans/2026-05-12-sp22-wr-plateau-investigation.md
Normal file
168
docs/plans/2026-05-12-sp22-wr-plateau-investigation.md
Normal file
@@ -0,0 +1,168 @@
|
||||
# SP22 — WR Plateau Investigation
|
||||
|
||||
**Author**: jgrusewski + assistant pair
|
||||
**Date**: 2026-05-12
|
||||
**Branch**: TBD (off `sp20-aux-h-fixed` after merge)
|
||||
**Predecessor**: SP21 T2.2 (cascade wiring; closed `docs/dqn-wire-up-audit.md` 2026-05-12)
|
||||
**Goal**: Move WR from the current ~50.1% plateau toward `project_goal_wr_55_pf_2`'s WR≥55% / PF≥2.0.
|
||||
|
||||
## State at session start
|
||||
|
||||
SP21 T2.2 cascade is feature-complete:
|
||||
|
||||
```
|
||||
d482efc41 fix(sp21): T2.2 Phase 8.4-fix — winner_conc z-score separation
|
||||
9f2d0fffb fix(sp21): T2.2 Phase 8.7 — default ISV buffer
|
||||
518670198+ ... Phase 8.x stack ...
|
||||
2937da889 fix(sp21): Return v3 (later v3.1)
|
||||
```
|
||||
|
||||
All E1-E8 enrichment producers wired to consumers. Local compute-sanitizer
|
||||
clean on eval-baseline closure path. v9 cycle 1 confirms cascade live:
|
||||
`win_conc=1.72`, `curric_conc=0.31`, `hindsight_mag=1.47e-5`.
|
||||
|
||||
**Binding constraint** (v10 hypothesis test, 8 epochs, fold 0):
|
||||
- val_Sharpe peaks at E3 (174), monotonically degrades to E8 (66, -62%)
|
||||
- WR pinned at 50.1-50.2% across ALL 8 epochs
|
||||
- PF pinned at 1.00-1.01
|
||||
- Trade volume contracts as policy degrades (492K → 402K)
|
||||
- Cascade controllers stable but cannot move the policy off the plateau
|
||||
|
||||
## Hypotheses
|
||||
|
||||
Each hypothesis gets its own atomic experiment. Compare against the v9
|
||||
baseline (3-epoch, full cascade, fold 0 peak val=174, WR=50.1%).
|
||||
|
||||
### H1: Label horizon mismatch (`project_label_horizon_is_bottleneck`)
|
||||
|
||||
**Theory**: The model predicts next-bar return but the actual directional
|
||||
signal lives at a different time scale (e.g., 5-bar, 30-bar lookback in
|
||||
volume bars ≈ 4-25 seconds). With volume bars firing every ~0.78s at
|
||||
34_496 bars/day, single-bar prediction is dominated by micro-noise.
|
||||
|
||||
**Test**: Modify the training reward (and aux-head label if applicable)
|
||||
to use N-bar forward return for N ∈ {5, 20, 60} bars. Compare WR/PF
|
||||
peak across the three settings.
|
||||
|
||||
**Predicted outcome if H1 is true**: WR jumps above 50.5% at N=20 or N=60.
|
||||
|
||||
**Predicted outcome if H1 is false**: WR stays at 50.1-50.2% across all N.
|
||||
|
||||
**Cost**: 3 smokes × ~30 min each on L40S = 90 min wall-clock.
|
||||
|
||||
### H2: Action-space pathology (`project_hold_hides_directional_signal_hypothesis`)
|
||||
|
||||
**Theory**: The 4-direction action space (Short/Hold/Long/Flat) collapses
|
||||
directional signal. Hold (keep position) and Flat (close position) both
|
||||
register as "no-op" in metrics but differ structurally — the model may
|
||||
be learning to predict Hold/Flat as a coward strategy that breaks even
|
||||
without committing direction.
|
||||
|
||||
**Test**: Run with Hold action **disabled** (force the model to choose
|
||||
between Short, Long, Flat each bar). If WR ≥ 50.1% with no Hold,
|
||||
directional learning is structurally suppressed by Hold. If WR drops
|
||||
significantly below 50% without Hold, Hold is the right thing and the
|
||||
plateau is elsewhere.
|
||||
|
||||
**Cost**: One smoke + one comparison run. ~60 min.
|
||||
|
||||
**Caveat**: requires action-space change in training config + state-layout
|
||||
fields. Larger code change than H1.
|
||||
|
||||
### H3: Reward shape doesn't produce directional gradient
|
||||
|
||||
**Theory**: Current reward is some combination of per-bar return,
|
||||
trade-cost penalty, hold-time bonus etc. (per `pearl_reward_shape_drives_quality_over_quantity`).
|
||||
With WR=50% and PF=1.0, the reward gradient may be too noisy to teach
|
||||
direction — model finds break-even policies more easily than directional
|
||||
ones.
|
||||
|
||||
**Test**: Two variants:
|
||||
1. **Pure directional reward**: `reward = sign(next_return) * |action_dir|`
|
||||
(binary correct/incorrect signal, no magnitude).
|
||||
2. **Asymmetric loss-aversion** (per `pearl_audit_unboundedness_for_implicit_asymmetry`):
|
||||
amplify losing trades 1.5× to penalize indecisive HOLD-as-default.
|
||||
|
||||
**Predicted outcome if H3 is true**: variant 1 either rises (model learns
|
||||
direction without magnitude confounders) or collapses (magnitude noise
|
||||
was actually helping); variant 2 forces model out of indecision.
|
||||
|
||||
**Cost**: 2 smokes × 30 min each = 60 min.
|
||||
|
||||
### H4: Feature representation gap
|
||||
|
||||
**Theory**: At 34_496 bars/day on volume bars, the feature set
|
||||
(42-dim market features from `extract_ml_features`) may not capture
|
||||
multi-bar context needed for directional prediction. MTF features
|
||||
exist in state layout (16 dims at indices 90-106) but eval-baseline
|
||||
feeds zeros for these.
|
||||
|
||||
**Test**: Train with **real MTF features computed** (not zero-padded).
|
||||
If WR rises ≥ 0.5%, MTF carries the directional signal. If not, MTF
|
||||
is also noise at this resolution.
|
||||
|
||||
**Cost**: Feature pipeline change + one smoke. Higher engineering cost
|
||||
than H1-H3.
|
||||
|
||||
### H5 (longshot): Bar-resolution itself is too noisy
|
||||
|
||||
**Theory**: 34_496 bars/day = 0.78s/bar is below the timescale where
|
||||
directional signal exists in ES.FUT order flow. Per
|
||||
`project_bar_resolution_is_actual_architecture`, the "hybrid tick→bar"
|
||||
architecture is aspirational not built. Maybe the model needs longer
|
||||
bars to find signal.
|
||||
|
||||
**Test**: Switch to time bars (e.g., 1-min or 5-min) and re-train.
|
||||
Compare WR/PF against volume-bar baseline.
|
||||
|
||||
**Cost**: Significant — different data ingestion path, retrain from
|
||||
scratch, may need hyperparam re-tuning. Longest experiment.
|
||||
|
||||
## Experiment order
|
||||
|
||||
Run in this priority order, killing early on each as soon as the
|
||||
hypothesis is conclusively resolved:
|
||||
|
||||
1. **H1 (label horizon)** — cheapest, addresses the most likely root cause
|
||||
2. **H3 (reward shape)** — cheap, complementary to H1
|
||||
3. **H2 (Hold disable)** — moderate cost
|
||||
4. **H4 (MTF features)** — higher engineering cost
|
||||
5. **H5 (bar resolution)** — last resort, biggest cost
|
||||
|
||||
Stop after the first hypothesis that produces WR ≥ 51% as a clear signal.
|
||||
|
||||
## Success criteria
|
||||
|
||||
**Phase 1 milestone**: WR ≥ 51% on val window for ≥ 1 fold (any hypothesis).
|
||||
**Phase 2 milestone**: WR ≥ 53% sustained across 3 folds.
|
||||
**Phase 3 milestone**: WR ≥ 55% AND PF ≥ 2.0 (the SP20+ goal).
|
||||
|
||||
Each milestone gates the next phase. Phase 1's hypothesis becomes the
|
||||
core architectural change going into Phase 2.
|
||||
|
||||
## Non-goals (out of scope for SP22)
|
||||
|
||||
- Cascade controller tuning (SP21 T2.2 is closed)
|
||||
- Eval pipeline architecture (Phase 8.3-8.7 wraps it; OFI/TLOB/MTF
|
||||
fidelity is a separate deferred concern from SP21)
|
||||
- Hyperopt — premature until the WR plateau is broken
|
||||
- Multi-symbol generalization — single-symbol (ES.FUT) until plateau
|
||||
resolves
|
||||
|
||||
## Risk: confounded signal from multiple changes
|
||||
|
||||
The cascade is **active** — its controllers will react to whatever
|
||||
training-side change we make. If WR jumps, distinguishing "cascade
|
||||
amplification of new signal" from "the new signal alone" requires
|
||||
the cascade-off comparison.
|
||||
|
||||
**Mitigation**: For any hypothesis that produces WR ≥ 51%, immediately
|
||||
run a comparison smoke with cascade disabled (set all enrichment
|
||||
producers to constant defaults). If WR holds → signal is real. If WR
|
||||
drops back to 50% → cascade was amplifying the signal (expected and
|
||||
desirable, but flag for the writeup).
|
||||
|
||||
## Audit pointer
|
||||
|
||||
Findings + commit log: `docs/dqn-wire-up-audit.md` (continue appending
|
||||
under `## 2026-05-12 — SP22 ...` headers).
|
||||
Reference in New Issue
Block a user