From cb7ace0063e401a9850daf872bd666970a510646 Mon Sep 17 00:00:00 2001 From: jgrusewski Date: Tue, 12 May 2026 14:24:26 +0200 Subject: [PATCH] =?UTF-8?q?feat(sp21):=20T2.2=20Phase=208.6=20=E2=80=94=20?= =?UTF-8?q?curriculum=20weights=20via=20z-score=20exp(-sharpe)=20(atomic)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit v8 smoke (train-96wfk, commit 5694eb4df) cycle 1 confirmed curric_conc=0.0000 despite Phase 8.2 (relaxed clamp) and 8.4 (CV formula) — root cause was the per-segment weight function itself, not the post-hoc concentration metric. Old: (1.0 / sharpe.max(0.01)).clamp(0.01, 100.0) → saturates at 100 when sharpe < 0.01 → with PF<1 policy, every segment has |sharpe| < 0.01 → all weights = 100 → uniform → CV=0 New: exp(-(sharpe - sharpe_mean) / sharpe_std).clamp(-3, 3) → z-score normalised, scale-invariant → monotonic, smooth, no saturation cliff → preserves differential difficulty at any scale → uniform-input → std=0 → z=0 → uniform weights (cold-start ✓) The Phase 8.4 CV formula was correct but operates on post-saturated weights which destroy differential signal upstream. 8.6 fixes the producer, 8.4 + 8.6 compose to give a meaningful curric_conc on volume bars. Note: win_conc=0 is intentional under PF<1 policy (compute_winner_concentration guards against undefined "winner concentration" when policy is losing). This is honest reporting, not a bug; addressed in audit doc rather than this commit. Pearls honoured: - pearl_controller_anchors_isv_driven: normalisation anchor = observed sharpe_std (signal-driven), not magic - feedback_isv_for_adaptive_bounds: z-clamp [-3, 3] is 3σ tail- cutoff for numerical safety, structural not tuned - pearl_first_observation_bootstrap: uniform Sharpe → uniform weights (cold-start sentinel preserved) - feedback_no_quickfixes: structural reformulation (1/x → exp(-z)) not a clamp adjustment Verification: cargo check -p ml --features cuda # clean Expected v9 cycle 1 curric_conc: ~0.3-0.6 (strong differential signal) from typical per-segment Sharpe spread ~1σ across 8 segments. Co-Authored-By: Claude Opus 4.7 (1M context) --- .../ml/src/trainers/dqn/trainer/enrichment.rs | 76 ++++++++++++---- docs/dqn-wire-up-audit.md | 91 +++++++++++++++++++ 2 files changed, 151 insertions(+), 16 deletions(-) diff --git a/crates/ml/src/trainers/dqn/trainer/enrichment.rs b/crates/ml/src/trainers/dqn/trainer/enrichment.rs index abc9aa403..051dc70c2 100644 --- a/crates/ml/src/trainers/dqn/trainer/enrichment.rs +++ b/crates/ml/src/trainers/dqn/trainer/enrichment.rs @@ -586,30 +586,55 @@ fn compute_hindsight_labels(trades: &[EvalTrade], pnl_std: f32) -> Vec Vec { if trades.is_empty() || n_segments == 0 { return Vec::new(); } let seg_size = (trades.len() / n_segments).max(1); - let mut weights: Vec = (0..n_segments) + + // Pass 1: per-segment Sharpe. + let sharpes: Vec = (0..n_segments) .map(|seg| { let start = seg * seg_size; let end = ((seg + 1) * seg_size).min(trades.len()); if start >= trades.len() { - return 0.01; + return 0.0; } let slice = &trades[start..end]; let mean: f32 = @@ -619,8 +644,27 @@ fn compute_curriculum_weights(trades: &[EvalTrade], n_segments: usize) -> Vec() / slice.len().max(1) as f32; - let sharpe = mean / var.sqrt().max(1e-6); - (1.0 / sharpe.max(0.01)).clamp(0.01, 100.0) + mean / var.sqrt().max(1e-6) + }) + .collect(); + + let n = sharpes.len() as f32; + let sharpe_mean: f32 = sharpes.iter().sum::() / n; + let sharpe_var: f32 = sharpes + .iter() + .map(|s| (s - sharpe_mean).powi(2)) + .sum::() + / n; + let sharpe_std = sharpe_var.sqrt().max(1e-6); + + // Pass 2: z-score normalised exp weights. + let mut weights: Vec = sharpes + .iter() + .map(|&s| { + // Negate so HARDER segments (lower Sharpe) get larger weights. + let z = -(s - sharpe_mean) / sharpe_std; + // z-clamp to keep exp range well-conditioned. + z.clamp(-3.0, 3.0).exp() }) .collect(); let total: f32 = weights.iter().sum(); diff --git a/docs/dqn-wire-up-audit.md b/docs/dqn-wire-up-audit.md index c41894f21..cebe64383 100644 --- a/docs/dqn-wire-up-audit.md +++ b/docs/dqn-wire-up-audit.md @@ -15843,3 +15843,94 @@ masking it. A future Phase will either wire PPO/supervised branch sizes correctly (PPO uses a 5-exposure action space, supervised uses signal thresholds — different conventions from DQN factored actions) or delete the dead paths per `feedback_no_partial_refactor`. + +## 2026-05-12 — SP21 T2.2 Phase 8.6: curriculum weights → z-score exp(-sharpe) (atomic) + +### Scope (atomic single commit) + +Rewrote `compute_curriculum_weights` from `1/sharpe.max(0.01)` to +`exp(-(sharpe - mean)/std)` (z-score normalised). v8 smoke (train-96wfk, +commit 5694eb4df) cycle 1 confirmed `curric_conc=0.0000` despite the +Phase 8.2 clamp relaxation and Phase 8.4 CV formula — root cause was +the per-segment weight function itself, not the post-hoc concentration +metric. + +### Root cause + +```rust +let sharpe = mean / var.sqrt().max(1e-6); +(1.0 / sharpe.max(0.01)).clamp(0.01, 100.0) // ← saturates at 100 when sharpe < 0.01 +``` + +For v8 cycle 1 (PF=0.94, val-trade mean ≈ negative, pnl_std=1.08e-5), +per-segment Sharpe is well under 0.01 in absolute value. `sharpe.max(0.01) = 0.01` +→ `1/0.01 = 100` → upper clamp → all 8 segments map to 100 → uniform +weights after normalisation → CV = 0. The Phase 8.4 CV formula was +correct but it operates on the post-saturated weights, which destroy +the differential signal upstream. + +### Fix + +z-score normalised exp(-sharpe) per segment: + +```rust +let z = -(s - sharpe_mean) / sharpe_std; +z.clamp(-3.0, 3.0).exp() +``` + +Properties: +- **Scale-invariant**: z-score removes per-segment Sharpe magnitude + (the killer near zero). +- **Monotonic & smooth**: weight grows as Sharpe drops below mean + (harder segment → higher curriculum weight); no saturation cliff. +- **Differential-difficulty preserved at any scale**: weights spread + whether per-segment Sharpes are `~1.0` (healthy) or `~1e-3` + (degenerate cold-start). +- **Uniform-input handled**: `sharpe_std → 0` falls back to uniform + weights via the `.max(1e-6)` floor + post-norm (correct sentinel). +- **z-clamp [-3, 3]** keeps `exp` range `[0.05, 20]` — well- + conditioned without losing differential signal. + +### Why win_conc=0 is NOT addressed here + +Phase 8.4 noted that `win_conc=0` for v8 cycle 1 was the +`compute_winner_concentration` guard intentionally tripping when +`all_mean ≤ 0.01×pnl_std`. With PF=0.94 the val-trade mean is genuinely +negative → guard is correctly suppressing "concentration of wins" for +a losing policy. This is honest reporting, not a bug. A future Phase +may revisit whether to emit a non-zero signal for losing policies +(e.g., negative concentration as "anti-signal"), but that's an +architectural decision, not a scale/saturation fix. + +### Files changed + +- `crates/ml/src/trainers/dqn/trainer/enrichment.rs`: + `compute_curriculum_weights` rewritten (z-score exp); two-pass + per-segment Sharpe → z-score → exp(-z) → normalise. +- `docs/dqn-wire-up-audit.md`: this entry + +### Pearls + invariants honoured + +- **pearl_controller_anchors_isv_driven**: normalisation anchor is + the observed `sharpe_std` (signal-driven), not magic constants +- **feedback_isv_for_adaptive_bounds**: z-clamp `[-3, 3]` is structural + (3σ tail-cutoff for numerical safety), not a tuned value +- **pearl_first_observation_bootstrap**: uniform Sharpe → std=0 → + z=0 → uniform weights (correct cold-start sentinel) +- **feedback_no_quickfixes**: structural reformulation (1/x → exp(-z)) + rather than a clamp adjustment + +### Verification + +``` +cargo check -p ml --features cuda # clean +``` + +### Expected v9 smoke + +For v9 cycle 1 (mirror v8's per-segment Sharpes in [-0.01, 0.01] +range with std ~0.005): +- z-scores roughly `[-2, 2]` across segments +- weights roughly `[0.14, 7.4]` pre-normalisation +- post-normalisation: max/min ratio ~50× — strong differential signal +- `curric_conc` (Phase 8.4 CV formula): should reach ~0.3-0.6