From d482efc4164d393d366b3c9549160d9a9ee18a84 Mon Sep 17 00:00:00 2001 From: jgrusewski Date: Tue, 12 May 2026 17:14:08 +0200 Subject: [PATCH] =?UTF-8?q?fix(sp21):=20T2.2=20Phase=208.4-fix=20=E2=80=94?= =?UTF-8?q?=20winner-concentration=20z-score=20separation=20(atomic)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit v8 smoke (train-96wfk) confirmed win_conc=0.0000 across all 9 cycles even when PF crossed 1.0. The Phase 8.2/8.4 threshold tweaks couldn't fix it because the underlying formula `top_mean / all_mean` is sign-unstable around `all_mean=0`. PER alpha boost path was dark for any cold-start policy below PF=1 — exactly the regime where the boost matters most. Old: if all_mean ≤ (0.01 * pnl_std).max(0.0) { return 0.0; } top_mean / all_mean → guard trips at PF≤1 → 0.0 (every v8 cycle) New: ((top_mean - all_mean) / pnl_std).max(0.0) → z-score separation: how many pnl_stds above the overall mean does the top decile sit? By construction top_mean ≥ all_mean, so separation is non-negative even when all_mean is negative. → bounded [0, ∞), scale-invariant, profitability-agnostic. Expected v9 magnitudes (v8 cycle-1-like inputs): top_mean ≈ 5e-6, all_mean ≈ 1e-7, pnl_std ≈ 1.08e-5 separation = (5e-6 - 1e-7) / 1.08e-5 ≈ 0.45 Healthy PF>1 cycles likely 0.2..1.5 range. Pearls honoured: - pearl_controller_anchors_isv_driven: pnl_std is the signal-driven scale anchor (replaces hardcoded all_mean denominator) - feedback_isv_for_adaptive_bounds: z-score formulation removes the hardcoded multiplier dependency that motivated 8.2 and 8.4's threshold adjustments - feedback_no_quickfixes: structural reformulation, not a threshold tweak (8.2 and 8.4 already showed threshold tweaks couldn't fix the sign-instability) Verification: cargo check -p ml --features cuda # clean Co-Authored-By: Claude Opus 4.7 (1M context) --- .../ml/src/trainers/dqn/trainer/enrichment.rs | 56 +++++-------- docs/dqn-wire-up-audit.md | 82 +++++++++++++++++++ 2 files changed, 102 insertions(+), 36 deletions(-) diff --git a/crates/ml/src/trainers/dqn/trainer/enrichment.rs b/crates/ml/src/trainers/dqn/trainer/enrichment.rs index 051dc70c2..94d45d130 100644 --- a/crates/ml/src/trainers/dqn/trainer/enrichment.rs +++ b/crates/ml/src/trainers/dqn/trainer/enrichment.rs @@ -413,29 +413,31 @@ fn compute_agreement_threshold( new.clamp(lo, hi) } -/// SP21 T2.2 Phase 5+6 (2026-05-11) — winner P&L concentration. +/// SP21 T2.2 Phase 5+6 (2026-05-11) — winner-decile P&L separation. /// -/// `top_decile_mean_pnl / max(all_trade_mean_pnl, EPS)`. Captures -/// how concentrated the policy's profitability is: > 1.0 means top- -/// decile trades carry disproportionate profit (a few big wins); -/// ≈ 1.0 means uniform distribution; < 1.0 indicates the top decile -/// underperforms the average (rare — typically signals top-decile -/// includes near-zero-pnl trades while the bulk of trades skew -/// negative). +/// **Phase 8.4-fix (2026-05-12)** — rewrote from +/// `top_decile_mean / all_mean` (with losing-strategy guard) to +/// `(top_decile_mean - all_mean) / pnl_std` (z-score separation). +/// The old formula short-circuited to `0.0` whenever the policy was +/// losing (all_mean ≤ 0): v8 smoke (PF=0.94..1.01) had `win_conc=0` +/// across all 9 cycles even though the top decile clearly outperformed. +/// PER alpha boost path was dark for any policy that hadn't crossed +/// PF=1 — exactly the cold-start regime where the boost matters most. +/// +/// New semantic: "how many `pnl_std`s above the overall mean does the +/// top decile sit?" Bounded `[0, ∞)`, scale-invariant via `pnl_std`, +/// works for both winning and losing policies (top decile is by +/// construction above the mean, even when the mean is negative). /// /// Used by `per_update_pa` (PER alpha scaling) — see /// `WINNER_CONCENTRATION_INDEX` in `cuda_pipeline::sp21_isv_slots`. /// -/// **Phase 8.2 (2026-05-11)** — losing-strategy guard threshold is -/// signal-driven via `pnl_std` instead of a hardcoded `1e-6`. On -/// volume bars, per-trade `pnl` (realized/equity) lands in -/// `1e-7..1e-5` range, so the original constant tripped on every -/// profitable cycle and the signal stayed pinned at 0. The new -/// guard `all_mean ≤ 0.1 × pnl_std` is bar-resolution-invariant per -/// `pearl_controller_anchors_isv_driven` and -/// `feedback_isv_for_adaptive_bounds`. +/// Cold-start sentinel: empty trades OR `pnl_std=0` → `0.0` (no +/// signal, no boost). The `.max(0.0)` floor is defensive — by +/// construction `top_decile_mean ≥ all_mean`, so the raw separation +/// is non-negative anyway. fn compute_winner_concentration(trades: &[EvalTrade], pnl_std: f32) -> f32 { - if trades.is_empty() { + if trades.is_empty() || pnl_std <= 0.0 { return 0.0; } let mut sorted: Vec = trades.iter().map(|t| t.pnl).collect(); @@ -443,25 +445,7 @@ fn compute_winner_concentration(trades: &[EvalTrade], pnl_std: f32) -> f32 { let top_n = (sorted.len() / 10).max(1); let top_mean: f32 = sorted[..top_n].iter().sum::() / top_n as f32; let all_mean: f32 = sorted.iter().sum::() / sorted.len() as f32; - // Guard: losing or break-even strategy → 0.0 sentinel so the - // kernel short-circuits to no boost. Threshold is data-scale- - // adaptive: 0.1×pnl_std means "mean is below a tenth of a single - // trade's typical magnitude" (effectively zero). With `pnl_std=0` - // (degenerate input), the OR clause traps on all_mean ≤ 0.0. - // - // **Phase 8.4 (2026-05-12)**: relaxed multiplier from 0.1 → 0.01. - // v6 smoke (train-x4m96) showed `pnl_std ≈ 1e-5` on volume bars, - // and the val-trade `all_mean` is roughly 1e-7..1e-6 (per-trade - // mean × sqrt(N) ≈ val_Sharpe). With multiplier 0.1, threshold - // 1e-6 was still above typical `all_mean` → win_conc pinned at - // 0.0000 across all 9 cycles. Multiplier 0.01 keeps the - // degenerate-strategy guard intact (mean must be at least 1% of - // a single trade's typical magnitude — clearly non-zero) while - // letting healthy small-but-positive policies fire the signal. - if all_mean <= (0.01 * pnl_std).max(0.0) { - return 0.0; - } - top_mean / all_mean + ((top_mean - all_mean) / pnl_std).max(0.0) } /// SP21 T2.2 Phase 5+6 (2026-05-11) — hindsight magnitude. diff --git a/docs/dqn-wire-up-audit.md b/docs/dqn-wire-up-audit.md index fdb983c23..22ce21a60 100644 --- a/docs/dqn-wire-up-audit.md +++ b/docs/dqn-wire-up-audit.md @@ -16038,3 +16038,85 @@ eval phase completes for all 3 folds with: If v9 still fails, compute-sanitizer next time on the larger production data — but Phase 8.7's local repro caught the canonical bug and the test now exercises the same code path. + +## 2026-05-12 — SP21 T2.2 Phase 8.4-fix: winner-concentration → z-score separation (atomic) + +### Scope (atomic single commit) + +Rewrote `compute_winner_concentration` from `top_decile_mean / all_mean` +(with losing-strategy guard) to `(top_decile_mean - all_mean) / pnl_std` +(z-score separation). Documented in the v8 smoke audit as "honest +report" but flagged as a real signal-deadness issue: PER alpha boost +path is dark for any policy that hasn't crossed PF=1 — exactly the +cold-start regime where the boost matters most. + +### v8 smoke evidence + +``` +Cycle 1: WR=50.2% PF=0.94 → win_conc=0.0000 (guard trips: all_mean<0) +Cycle 2: WR=50.1% PF=1.00 → win_conc=0.0000 +Cycle 5: WR=50.1% PF=1.01 → win_conc=0.0000 (still tripping near PF=1) +Cycle 6: WR=50.1% PF=1.01 → win_conc=0.0000 +... all 9 cycles: win_conc=0.0000 +``` + +The top decile of trades CLEARLY outperformed the overall mean +(otherwise the policy would have negative Sharpe instead of slightly +positive). The old formula `top_mean / all_mean` produces a negative +or huge-positive when `all_mean ≈ 0` — undefined for a multiplicative +PER alpha boost factor in `[0, ∞)`. The Phase 8.2/8.4 threshold +adjustments only delayed the short-circuit; they didn't fix the +fundamental sign issue. + +### Fix + +New formula: `((top_mean - all_mean) / pnl_std).max(0.0)` + +Semantics: "How many `pnl_std`s above the overall mean does the top +decile sit?" By construction `top_decile_mean ≥ all_mean` (top decile +is a subset of the sorted-descending sequence), so the raw separation +is non-negative; the `.max(0.0)` is defensive. Bounded `[0, ∞)`, +scale-invariant via `pnl_std`, works regardless of policy profitability. + +### Pearls + invariants honoured + +- **pearl_controller_anchors_isv_driven**: `pnl_std` is the + signal-driven scale anchor (replaces the old hardcoded `all_mean` + scale that broke at PF<1) +- **feedback_isv_for_adaptive_bounds**: z-score formulation removes + the hardcoded multiplier dependency that motivated Phase 8.2 and + Phase 8.4's threshold adjustments +- **pearl_first_observation_bootstrap**: empty trades / `pnl_std=0` + → `0.0` sentinel preserved +- **feedback_no_quickfixes**: structural reformulation (ratio → z-score) + not a threshold tweak (which Phase 8.2 + 8.4 already showed couldn't + fix the underlying sign-instability) + +### Files changed + +- `crates/ml/src/trainers/dqn/trainer/enrichment.rs`: + `compute_winner_concentration` rewritten; threshold guard removed + (replaced by the structurally-non-negative separation formula). +- `docs/dqn-wire-up-audit.md`: this entry. + +### Verification + +``` +cargo check -p ml --features cuda # clean +``` + +### Expected v9 smoke + +For v8-cycle-1-like inputs (top_mean ≈ 5e-6, all_mean ≈ 1e-7, +pnl_std ≈ 1.08e-5): + +- old formula: `5e-6 / 1e-7 = 50.0` (or `0.0` via guard) — broken +- new formula: `(5e-6 - 1e-7) / 1.08e-5 ≈ 0.45` — non-zero, sensible + +Healthy PF>1 cycles: separation likely `0.2 .. 1.5` (top decile +sits 0.2-1.5 σ above mean). Strong-overfit cycles (cycle 3, 5): +larger separations as model concentrates wins. + +The PER alpha boost path (via `WINNER_CONCENTRATION_INDEX` → +`per_update_pa`) finally has a non-zero signal to amplify on +volume-bar policies regardless of profitability state.