feat(sp21): T2.2 Phase 8.6 — curriculum weights via z-score exp(-sharpe) (atomic)
v8 smoke (train-96wfk, commit 5694eb4df) cycle 1 confirmed
curric_conc=0.0000 despite Phase 8.2 (relaxed clamp) and 8.4 (CV
formula) — root cause was the per-segment weight function itself,
not the post-hoc concentration metric.
Old: (1.0 / sharpe.max(0.01)).clamp(0.01, 100.0)
→ saturates at 100 when sharpe < 0.01
→ with PF<1 policy, every segment has |sharpe| < 0.01
→ all weights = 100 → uniform → CV=0
New: exp(-(sharpe - sharpe_mean) / sharpe_std).clamp(-3, 3)
→ z-score normalised, scale-invariant
→ monotonic, smooth, no saturation cliff
→ preserves differential difficulty at any scale
→ uniform-input → std=0 → z=0 → uniform weights (cold-start ✓)
The Phase 8.4 CV formula was correct but operates on post-saturated
weights which destroy differential signal upstream. 8.6 fixes the
producer, 8.4 + 8.6 compose to give a meaningful curric_conc on
volume bars.
Note: win_conc=0 is intentional under PF<1 policy (compute_winner_concentration
guards against undefined "winner concentration" when policy is losing).
This is honest reporting, not a bug; addressed in audit doc rather
than this commit.
Pearls honoured:
- pearl_controller_anchors_isv_driven: normalisation anchor =
observed sharpe_std (signal-driven), not magic
- feedback_isv_for_adaptive_bounds: z-clamp [-3, 3] is 3σ tail-
cutoff for numerical safety, structural not tuned
- pearl_first_observation_bootstrap: uniform Sharpe → uniform
weights (cold-start sentinel preserved)
- feedback_no_quickfixes: structural reformulation (1/x → exp(-z))
not a clamp adjustment
Verification:
cargo check -p ml --features cuda # clean
Expected v9 cycle 1 curric_conc: ~0.3-0.6 (strong differential signal)
from typical per-segment Sharpe spread ~1σ across 8 segments.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -586,30 +586,55 @@ fn compute_hindsight_labels(trades: &[EvalTrade], pnl_std: f32) -> Vec<Hindsight
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// E8: Curriculum weights — inverse-Sharpe per segment.
|
||||
/// E8: Curriculum weights — z-score-normalised exp(-sharpe) per segment.
|
||||
///
|
||||
/// **Phase 8.2 (2026-05-11)** — relaxed inverse-Sharpe clamp from
|
||||
/// `(0.1, 10.0)` to `(0.01, 100.0)` so close-but-distinct small
|
||||
/// per-segment Sharpes (e.g. 0.05 vs 0.09 → `1/sharpe` of 20 vs 11)
|
||||
/// no longer both saturate the upper bound. Under the tight clamp,
|
||||
/// every cycle produced near-uniform weights → `curric_conc=0`. The
|
||||
/// wider range preserves segment-rank fidelity while still keeping
|
||||
/// the post-normalisation weights well-conditioned (max/min ratio
|
||||
/// of 1e4 instead of 1e2). Per `pearl_controller_anchors_isv_driven`
|
||||
/// these bounds should eventually be ISV-driven too — left as a
|
||||
/// follow-up because the immediate Phase 8.2 goal is just to unblock
|
||||
/// the dark-code path; full adaptive bounds are Phase 9 scope.
|
||||
/// **Phase 8.6 (2026-05-12)** — rewrote from `1/sharpe.max(0.01)` to
|
||||
/// `exp(-(sharpe - mean_sharpe) / std_sharpe)`. The old formula
|
||||
/// saturated at the upper clamp (100.0) whenever per-segment Sharpe
|
||||
/// was near zero or negative — and with `pnl_mean ≈ 1e-7` for a
|
||||
/// PF<1 policy, EVERY segment had `|sharpe| < 0.01` → all saturated
|
||||
/// to 100 → uniform weights → CV=0. v8 cycle 1 (commit 5694eb4df)
|
||||
/// confirmed `curric_conc=0.0000` despite the wider Phase 8.2 clamp
|
||||
/// and the more sensitive Phase 8.4 CV formula.
|
||||
///
|
||||
/// The new formula is z-score-normalised on per-segment Sharpe:
|
||||
///
|
||||
/// w_i = exp(-(sharpe_i - sharpe_mean) / sharpe_std.max(eps))
|
||||
///
|
||||
/// Properties:
|
||||
/// - **Scale-invariant**: z-score normalisation removes the
|
||||
/// per-segment Sharpe magnitude (the killer of the old formula
|
||||
/// near zero).
|
||||
/// - **Monotonic & smooth**: weight grows as Sharpe drops below
|
||||
/// the segment-mean (harder segment → higher curriculum weight),
|
||||
/// no saturation cliff.
|
||||
/// - **Differential-difficulty preserved at any scale**: weights
|
||||
/// spread regardless of whether per-segment Sharpes are
|
||||
/// `~1.0` (healthy) or `~1e-3` (degenerate cold-start).
|
||||
/// - **Uniform-input handled**: `sharpe_std → 0` falls back to
|
||||
/// uniform weights (correct cold-start sentinel).
|
||||
///
|
||||
/// z-clamp at [-3, 3] before exp keeps weights in `[exp(-3), exp(3)] =
|
||||
/// [0.05, 20]` — well-conditioned for normalisation without losing
|
||||
/// the differential signal.
|
||||
///
|
||||
/// Per `pearl_controller_anchors_isv_driven` and
|
||||
/// `feedback_isv_for_adaptive_bounds`, the normalisation anchor is
|
||||
/// the observed cycle-scoped `sharpe_std` (signal-driven), not a
|
||||
/// hardcoded constant.
|
||||
fn compute_curriculum_weights(trades: &[EvalTrade], n_segments: usize) -> Vec<f32> {
|
||||
if trades.is_empty() || n_segments == 0 {
|
||||
return Vec::new();
|
||||
}
|
||||
let seg_size = (trades.len() / n_segments).max(1);
|
||||
let mut weights: Vec<f32> = (0..n_segments)
|
||||
|
||||
// Pass 1: per-segment Sharpe.
|
||||
let sharpes: Vec<f32> = (0..n_segments)
|
||||
.map(|seg| {
|
||||
let start = seg * seg_size;
|
||||
let end = ((seg + 1) * seg_size).min(trades.len());
|
||||
if start >= trades.len() {
|
||||
return 0.01;
|
||||
return 0.0;
|
||||
}
|
||||
let slice = &trades[start..end];
|
||||
let mean: f32 =
|
||||
@@ -619,8 +644,27 @@ fn compute_curriculum_weights(trades: &[EvalTrade], n_segments: usize) -> Vec<f3
|
||||
.map(|t| (t.pnl - mean).powi(2))
|
||||
.sum::<f32>()
|
||||
/ slice.len().max(1) as f32;
|
||||
let sharpe = mean / var.sqrt().max(1e-6);
|
||||
(1.0 / sharpe.max(0.01)).clamp(0.01, 100.0)
|
||||
mean / var.sqrt().max(1e-6)
|
||||
})
|
||||
.collect();
|
||||
|
||||
let n = sharpes.len() as f32;
|
||||
let sharpe_mean: f32 = sharpes.iter().sum::<f32>() / n;
|
||||
let sharpe_var: f32 = sharpes
|
||||
.iter()
|
||||
.map(|s| (s - sharpe_mean).powi(2))
|
||||
.sum::<f32>()
|
||||
/ n;
|
||||
let sharpe_std = sharpe_var.sqrt().max(1e-6);
|
||||
|
||||
// Pass 2: z-score normalised exp weights.
|
||||
let mut weights: Vec<f32> = sharpes
|
||||
.iter()
|
||||
.map(|&s| {
|
||||
// Negate so HARDER segments (lower Sharpe) get larger weights.
|
||||
let z = -(s - sharpe_mean) / sharpe_std;
|
||||
// z-clamp to keep exp range well-conditioned.
|
||||
z.clamp(-3.0, 3.0).exp()
|
||||
})
|
||||
.collect();
|
||||
let total: f32 = weights.iter().sum();
|
||||
|
||||
@@ -15843,3 +15843,94 @@ masking it. A future Phase will either wire PPO/supervised branch
|
||||
sizes correctly (PPO uses a 5-exposure action space, supervised uses
|
||||
signal thresholds — different conventions from DQN factored actions)
|
||||
or delete the dead paths per `feedback_no_partial_refactor`.
|
||||
|
||||
## 2026-05-12 — SP21 T2.2 Phase 8.6: curriculum weights → z-score exp(-sharpe) (atomic)
|
||||
|
||||
### Scope (atomic single commit)
|
||||
|
||||
Rewrote `compute_curriculum_weights` from `1/sharpe.max(0.01)` to
|
||||
`exp(-(sharpe - mean)/std)` (z-score normalised). v8 smoke (train-96wfk,
|
||||
commit 5694eb4df) cycle 1 confirmed `curric_conc=0.0000` despite the
|
||||
Phase 8.2 clamp relaxation and Phase 8.4 CV formula — root cause was
|
||||
the per-segment weight function itself, not the post-hoc concentration
|
||||
metric.
|
||||
|
||||
### Root cause
|
||||
|
||||
```rust
|
||||
let sharpe = mean / var.sqrt().max(1e-6);
|
||||
(1.0 / sharpe.max(0.01)).clamp(0.01, 100.0) // ← saturates at 100 when sharpe < 0.01
|
||||
```
|
||||
|
||||
For v8 cycle 1 (PF=0.94, val-trade mean ≈ negative, pnl_std=1.08e-5),
|
||||
per-segment Sharpe is well under 0.01 in absolute value. `sharpe.max(0.01) = 0.01`
|
||||
→ `1/0.01 = 100` → upper clamp → all 8 segments map to 100 → uniform
|
||||
weights after normalisation → CV = 0. The Phase 8.4 CV formula was
|
||||
correct but it operates on the post-saturated weights, which destroy
|
||||
the differential signal upstream.
|
||||
|
||||
### Fix
|
||||
|
||||
z-score normalised exp(-sharpe) per segment:
|
||||
|
||||
```rust
|
||||
let z = -(s - sharpe_mean) / sharpe_std;
|
||||
z.clamp(-3.0, 3.0).exp()
|
||||
```
|
||||
|
||||
Properties:
|
||||
- **Scale-invariant**: z-score removes per-segment Sharpe magnitude
|
||||
(the killer near zero).
|
||||
- **Monotonic & smooth**: weight grows as Sharpe drops below mean
|
||||
(harder segment → higher curriculum weight); no saturation cliff.
|
||||
- **Differential-difficulty preserved at any scale**: weights spread
|
||||
whether per-segment Sharpes are `~1.0` (healthy) or `~1e-3`
|
||||
(degenerate cold-start).
|
||||
- **Uniform-input handled**: `sharpe_std → 0` falls back to uniform
|
||||
weights via the `.max(1e-6)` floor + post-norm (correct sentinel).
|
||||
- **z-clamp [-3, 3]** keeps `exp` range `[0.05, 20]` — well-
|
||||
conditioned without losing differential signal.
|
||||
|
||||
### Why win_conc=0 is NOT addressed here
|
||||
|
||||
Phase 8.4 noted that `win_conc=0` for v8 cycle 1 was the
|
||||
`compute_winner_concentration` guard intentionally tripping when
|
||||
`all_mean ≤ 0.01×pnl_std`. With PF=0.94 the val-trade mean is genuinely
|
||||
negative → guard is correctly suppressing "concentration of wins" for
|
||||
a losing policy. This is honest reporting, not a bug. A future Phase
|
||||
may revisit whether to emit a non-zero signal for losing policies
|
||||
(e.g., negative concentration as "anti-signal"), but that's an
|
||||
architectural decision, not a scale/saturation fix.
|
||||
|
||||
### Files changed
|
||||
|
||||
- `crates/ml/src/trainers/dqn/trainer/enrichment.rs`:
|
||||
`compute_curriculum_weights` rewritten (z-score exp); two-pass
|
||||
per-segment Sharpe → z-score → exp(-z) → normalise.
|
||||
- `docs/dqn-wire-up-audit.md`: this entry
|
||||
|
||||
### Pearls + invariants honoured
|
||||
|
||||
- **pearl_controller_anchors_isv_driven**: normalisation anchor is
|
||||
the observed `sharpe_std` (signal-driven), not magic constants
|
||||
- **feedback_isv_for_adaptive_bounds**: z-clamp `[-3, 3]` is structural
|
||||
(3σ tail-cutoff for numerical safety), not a tuned value
|
||||
- **pearl_first_observation_bootstrap**: uniform Sharpe → std=0 →
|
||||
z=0 → uniform weights (correct cold-start sentinel)
|
||||
- **feedback_no_quickfixes**: structural reformulation (1/x → exp(-z))
|
||||
rather than a clamp adjustment
|
||||
|
||||
### Verification
|
||||
|
||||
```
|
||||
cargo check -p ml --features cuda # clean
|
||||
```
|
||||
|
||||
### Expected v9 smoke
|
||||
|
||||
For v9 cycle 1 (mirror v8's per-segment Sharpes in [-0.01, 0.01]
|
||||
range with std ~0.005):
|
||||
- z-scores roughly `[-2, 2]` across segments
|
||||
- weights roughly `[0.14, 7.4]` pre-normalisation
|
||||
- post-normalisation: max/min ratio ~50× — strong differential signal
|
||||
- `curric_conc` (Phase 8.4 CV formula): should reach ~0.3-0.6
|
||||
|
||||
Reference in New Issue
Block a user