feat(sp21): T2.2 Phase 8.6 — curriculum weights via z-score exp(-sharpe) (atomic)

v8 smoke (train-96wfk, commit 5694eb4df) cycle 1 confirmed
curric_conc=0.0000 despite Phase 8.2 (relaxed clamp) and 8.4 (CV
formula) — root cause was the per-segment weight function itself,
not the post-hoc concentration metric.

Old:  (1.0 / sharpe.max(0.01)).clamp(0.01, 100.0)
      → saturates at 100 when sharpe < 0.01
      → with PF<1 policy, every segment has |sharpe| < 0.01
      → all weights = 100 → uniform → CV=0

New:  exp(-(sharpe - sharpe_mean) / sharpe_std).clamp(-3, 3)
      → z-score normalised, scale-invariant
      → monotonic, smooth, no saturation cliff
      → preserves differential difficulty at any scale
      → uniform-input → std=0 → z=0 → uniform weights (cold-start ✓)

The Phase 8.4 CV formula was correct but operates on post-saturated
weights which destroy differential signal upstream. 8.6 fixes the
producer, 8.4 + 8.6 compose to give a meaningful curric_conc on
volume bars.

Note: win_conc=0 is intentional under PF<1 policy (compute_winner_concentration
guards against undefined "winner concentration" when policy is losing).
This is honest reporting, not a bug; addressed in audit doc rather
than this commit.

Pearls honoured:
  - pearl_controller_anchors_isv_driven: normalisation anchor =
    observed sharpe_std (signal-driven), not magic
  - feedback_isv_for_adaptive_bounds: z-clamp [-3, 3] is 3σ tail-
    cutoff for numerical safety, structural not tuned
  - pearl_first_observation_bootstrap: uniform Sharpe → uniform
    weights (cold-start sentinel preserved)
  - feedback_no_quickfixes: structural reformulation (1/x → exp(-z))
    not a clamp adjustment

Verification:
  cargo check -p ml --features cuda  # clean

Expected v9 cycle 1 curric_conc: ~0.3-0.6 (strong differential signal)
from typical per-segment Sharpe spread ~1σ across 8 segments.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-05-12 14:24:26 +02:00
parent 5694eb4df2
commit cb7ace0063
2 changed files with 151 additions and 16 deletions

View File

@@ -586,30 +586,55 @@ fn compute_hindsight_labels(trades: &[EvalTrade], pnl_std: f32) -> Vec<Hindsight
.collect()
}
/// E8: Curriculum weights — inverse-Sharpe per segment.
/// E8: Curriculum weights — z-score-normalised exp(-sharpe) per segment.
///
/// **Phase 8.2 (2026-05-11)** — relaxed inverse-Sharpe clamp from
/// `(0.1, 10.0)` to `(0.01, 100.0)` so close-but-distinct small
/// per-segment Sharpes (e.g. 0.05 vs 0.09 → `1/sharpe` of 20 vs 11)
/// no longer both saturate the upper bound. Under the tight clamp,
/// every cycle produced near-uniform weights → `curric_conc=0`. The
/// wider range preserves segment-rank fidelity while still keeping
/// the post-normalisation weights well-conditioned (max/min ratio
/// of 1e4 instead of 1e2). Per `pearl_controller_anchors_isv_driven`
/// these bounds should eventually be ISV-driven too — left as a
/// follow-up because the immediate Phase 8.2 goal is just to unblock
/// the dark-code path; full adaptive bounds are Phase 9 scope.
/// **Phase 8.6 (2026-05-12)** — rewrote from `1/sharpe.max(0.01)` to
/// `exp(-(sharpe - mean_sharpe) / std_sharpe)`. The old formula
/// saturated at the upper clamp (100.0) whenever per-segment Sharpe
/// was near zero or negative — and with `pnl_mean ≈ 1e-7` for a
/// PF<1 policy, EVERY segment had `|sharpe| < 0.01` → all saturated
/// to 100 → uniform weights → CV=0. v8 cycle 1 (commit 5694eb4df)
/// confirmed `curric_conc=0.0000` despite the wider Phase 8.2 clamp
/// and the more sensitive Phase 8.4 CV formula.
///
/// The new formula is z-score-normalised on per-segment Sharpe:
///
/// w_i = exp(-(sharpe_i - sharpe_mean) / sharpe_std.max(eps))
///
/// Properties:
/// - **Scale-invariant**: z-score normalisation removes the
/// per-segment Sharpe magnitude (the killer of the old formula
/// near zero).
/// - **Monotonic & smooth**: weight grows as Sharpe drops below
/// the segment-mean (harder segment → higher curriculum weight),
/// no saturation cliff.
/// - **Differential-difficulty preserved at any scale**: weights
/// spread regardless of whether per-segment Sharpes are
/// `~1.0` (healthy) or `~1e-3` (degenerate cold-start).
/// - **Uniform-input handled**: `sharpe_std → 0` falls back to
/// uniform weights (correct cold-start sentinel).
///
/// z-clamp at [-3, 3] before exp keeps weights in `[exp(-3), exp(3)] =
/// [0.05, 20]` — well-conditioned for normalisation without losing
/// the differential signal.
///
/// Per `pearl_controller_anchors_isv_driven` and
/// `feedback_isv_for_adaptive_bounds`, the normalisation anchor is
/// the observed cycle-scoped `sharpe_std` (signal-driven), not a
/// hardcoded constant.
fn compute_curriculum_weights(trades: &[EvalTrade], n_segments: usize) -> Vec<f32> {
if trades.is_empty() || n_segments == 0 {
return Vec::new();
}
let seg_size = (trades.len() / n_segments).max(1);
let mut weights: Vec<f32> = (0..n_segments)
// Pass 1: per-segment Sharpe.
let sharpes: Vec<f32> = (0..n_segments)
.map(|seg| {
let start = seg * seg_size;
let end = ((seg + 1) * seg_size).min(trades.len());
if start >= trades.len() {
return 0.01;
return 0.0;
}
let slice = &trades[start..end];
let mean: f32 =
@@ -619,8 +644,27 @@ fn compute_curriculum_weights(trades: &[EvalTrade], n_segments: usize) -> Vec<f3
.map(|t| (t.pnl - mean).powi(2))
.sum::<f32>()
/ slice.len().max(1) as f32;
let sharpe = mean / var.sqrt().max(1e-6);
(1.0 / sharpe.max(0.01)).clamp(0.01, 100.0)
mean / var.sqrt().max(1e-6)
})
.collect();
let n = sharpes.len() as f32;
let sharpe_mean: f32 = sharpes.iter().sum::<f32>() / n;
let sharpe_var: f32 = sharpes
.iter()
.map(|s| (s - sharpe_mean).powi(2))
.sum::<f32>()
/ n;
let sharpe_std = sharpe_var.sqrt().max(1e-6);
// Pass 2: z-score normalised exp weights.
let mut weights: Vec<f32> = sharpes
.iter()
.map(|&s| {
// Negate so HARDER segments (lower Sharpe) get larger weights.
let z = -(s - sharpe_mean) / sharpe_std;
// z-clamp to keep exp range well-conditioned.
z.clamp(-3.0, 3.0).exp()
})
.collect();
let total: f32 = weights.iter().sum();

View File

@@ -15843,3 +15843,94 @@ masking it. A future Phase will either wire PPO/supervised branch
sizes correctly (PPO uses a 5-exposure action space, supervised uses
signal thresholds — different conventions from DQN factored actions)
or delete the dead paths per `feedback_no_partial_refactor`.
## 2026-05-12 — SP21 T2.2 Phase 8.6: curriculum weights → z-score exp(-sharpe) (atomic)
### Scope (atomic single commit)
Rewrote `compute_curriculum_weights` from `1/sharpe.max(0.01)` to
`exp(-(sharpe - mean)/std)` (z-score normalised). v8 smoke (train-96wfk,
commit 5694eb4df) cycle 1 confirmed `curric_conc=0.0000` despite the
Phase 8.2 clamp relaxation and Phase 8.4 CV formula — root cause was
the per-segment weight function itself, not the post-hoc concentration
metric.
### Root cause
```rust
let sharpe = mean / var.sqrt().max(1e-6);
(1.0 / sharpe.max(0.01)).clamp(0.01, 100.0) // ← saturates at 100 when sharpe < 0.01
```
For v8 cycle 1 (PF=0.94, val-trade mean ≈ negative, pnl_std=1.08e-5),
per-segment Sharpe is well under 0.01 in absolute value. `sharpe.max(0.01) = 0.01`
`1/0.01 = 100` → upper clamp → all 8 segments map to 100 → uniform
weights after normalisation → CV = 0. The Phase 8.4 CV formula was
correct but it operates on the post-saturated weights, which destroy
the differential signal upstream.
### Fix
z-score normalised exp(-sharpe) per segment:
```rust
let z = -(s - sharpe_mean) / sharpe_std;
z.clamp(-3.0, 3.0).exp()
```
Properties:
- **Scale-invariant**: z-score removes per-segment Sharpe magnitude
(the killer near zero).
- **Monotonic & smooth**: weight grows as Sharpe drops below mean
(harder segment → higher curriculum weight); no saturation cliff.
- **Differential-difficulty preserved at any scale**: weights spread
whether per-segment Sharpes are `~1.0` (healthy) or `~1e-3`
(degenerate cold-start).
- **Uniform-input handled**: `sharpe_std → 0` falls back to uniform
weights via the `.max(1e-6)` floor + post-norm (correct sentinel).
- **z-clamp [-3, 3]** keeps `exp` range `[0.05, 20]` — well-
conditioned without losing differential signal.
### Why win_conc=0 is NOT addressed here
Phase 8.4 noted that `win_conc=0` for v8 cycle 1 was the
`compute_winner_concentration` guard intentionally tripping when
`all_mean ≤ 0.01×pnl_std`. With PF=0.94 the val-trade mean is genuinely
negative → guard is correctly suppressing "concentration of wins" for
a losing policy. This is honest reporting, not a bug. A future Phase
may revisit whether to emit a non-zero signal for losing policies
(e.g., negative concentration as "anti-signal"), but that's an
architectural decision, not a scale/saturation fix.
### Files changed
- `crates/ml/src/trainers/dqn/trainer/enrichment.rs`:
`compute_curriculum_weights` rewritten (z-score exp); two-pass
per-segment Sharpe → z-score → exp(-z) → normalise.
- `docs/dqn-wire-up-audit.md`: this entry
### Pearls + invariants honoured
- **pearl_controller_anchors_isv_driven**: normalisation anchor is
the observed `sharpe_std` (signal-driven), not magic constants
- **feedback_isv_for_adaptive_bounds**: z-clamp `[-3, 3]` is structural
(3σ tail-cutoff for numerical safety), not a tuned value
- **pearl_first_observation_bootstrap**: uniform Sharpe → std=0 →
z=0 → uniform weights (correct cold-start sentinel)
- **feedback_no_quickfixes**: structural reformulation (1/x → exp(-z))
rather than a clamp adjustment
### Verification
```
cargo check -p ml --features cuda # clean
```
### Expected v9 smoke
For v9 cycle 1 (mirror v8's per-segment Sharpes in [-0.01, 0.01]
range with std ~0.005):
- z-scores roughly `[-2, 2]` across segments
- weights roughly `[0.14, 7.4]` pre-normalisation
- post-normalisation: max/min ratio ~50× — strong differential signal
- `curric_conc` (Phase 8.4 CV formula): should reach ~0.3-0.6