design(policy-quality): revision 4 — functional gaps

Addressed functional concerns from third review:

1. Surrogate-noise gate was statistically weak — "random Sharpe ≤ 0.5×
   trained Sharpe on same val data" could be satisfied by sample
   noise. Replaced with percentile-based test:
   * Pool all 6 val folds (~600 trades) for statistical power.
   * Run N=30 surrogate random-action rollouts (matched marginal
     action distribution — so difference is state-action mapping,
     not action frequency).
   * Assert trained pooled Sharpe > 95th-percentile of surrogate
     distribution. Explicit false-positive control.

2. Escalation path ("scope exceeds sub-project A v2") was hand-wavy.
   Now concrete: failure after 3 iterations → paused, not closed →
   triage spec opened with evidence summary + diagnosis (policy/
   reward/state/architecture/data bottleneck) + explicit decision
   (continue on feat/policy-quality or fork feat/policy-quality-v2).
   Triage itself is a full brainstorming cycle.

3. HEALTH_DIAG §4.2 Track 1 fields only covered H1–H5 — H6–H10
   needed their own detection signals. Added:
   - trail_fire_rate_{quarter,half,full}        (H6)
   - hold_time_at_exit_{quarter,half,full}      (H6)
   - vsn_mask_{magnitude,direction}             (H7)
   - noisy_sigma_{mag_head,dir_head}            (H7)
   - target_drift_{mag_head,dir_head}           (H8)
   - action_dist_eval_{quarter,half,full}       (H10)

docs/superpowers/specs/2026-04-21-policy-quality-design.md

@@ -40,7 +40,7 @@ The spec closes when a single L40S validation run (6-fold × 50-epoch, the exist
 
 - **B-1 Multi-fold Sharpe:** Best val Sharpe (annualized) `> 10` on at least **5/6 folds**
 - **B-2 Multi-fold WinRate:** Val WinRate `> 55%` on at least **5/6 folds**
-- **SN Surrogate-noise:** Random-action Sharpe `≤ 0.5×` trained-model Sharpe on same val data (`surrogate_noise_check.rs`)
+- **SN Surrogate-noise:** Trained-model pooled Sharpe `>` 95th-percentile of N=30 random-action surrogate runs (`surrogate_noise_check.rs`). Uses pooled-across-6-folds val returns (~600 trades) so the comparison has statistical power.
 
 ### 2.2 Soft gates (ship with documented follow-up if missed)
 
@@ -53,7 +53,11 @@ The spec closes when a single L40S validation run (6-fold × 50-epoch, the exist
 
 - **All mandatory + all soft pass** → merge to main, close spec clean.
 - **All mandatory pass, some soft miss** → merge to main, open `docs/superpowers/specs/YYYY-MM-DD-policy-quality-followup.md` documenting deltas.
-- **Any mandatory miss** → `feat/policy-quality` stays unmerged. Up to 2 more iterations on the feature branch. If still failing, scope exceeds this spec — escalate to sub-project A v2.
+- **Any mandatory miss** → `feat/policy-quality` stays unmerged. Up to 2 more iterations on the feature branch (total 3 Phase-3 validation runs). If all 3 iterations still miss a mandatory gate, the spec is **paused, not closed**. A dedicated triage spec is opened at `docs/superpowers/specs/YYYY-MM-DD-policy-quality-triage.md` with:
+  - Summary of the 3 failed runs' evidence (which gates, by how much, which hypotheses correlate with failure)
+  - Explicit diagnosis: policy/reward/state bottleneck? architecture bottleneck? data bottleneck?
+  - Decision: continue on `feat/policy-quality` with new hypotheses, OR fork a new branch `feat/policy-quality-v2` with a different angle (e.g., reward redesign, architecture change, alternative algorithm PPO/SAC)
+  - Triage spec is a full brainstorming cycle (not a follow-up note) — restarts the skill flow.
 
 ---
 
@@ -149,9 +153,11 @@ All live under `crates/ml/src/trainers/dqn/smoke_tests/`. Each is `#[ignore]`-ga
 
 **`surrogate_noise_check.rs`** — false-positive detector
 - Use the trained model from Phase 2 or Phase 3.
-- On validation data, REPLACE model actions with uniformly random actions (same distribution shape as trained model).
-- Run backtest with random actions.
-- Assert: random-action Sharpe ≤ 0.5× trained-model Sharpe. If random actions produce comparable Sharpe, we have measurement/leakage bugs (look-ahead, data re-use, stale features) — not a real edge.
+- **Pool all 6 validation folds** into a single val-return stream (~600 trades aggregated) — per-fold Sharpe is sample-size-noisy, so the per-fold comparison could be satisfied by luck. Pooled Sharpe has the statistical power needed for this gate.
+- Generate **N=30 surrogate runs**: for each, REPLACE model actions with uniformly random actions (matched marginal action distribution to the trained model's, so the comparison isn't trivially won by action-frequency differences alone). Run backtest per surrogate, collect 30 random-action pooled Sharpes.
+- Compute the 95th percentile of the surrogate Sharpe distribution.
+- Assert: **trained-model pooled Sharpe > surrogate 95th-percentile Sharpe** (i.e., the trained model is outside the top 5% of what random actions produce by chance).
+- Rationale for statistical gate: "0.5× ratio" in a single comparison is vulnerable to sample noise; a percentile-based test gives genuine statistical power with explicit false-positive control.
 - Critical production-readiness gate. Permanent artifact — guards against future measurement regressions.
 
 ### 4.2 HEALTH_DIAG extensions
@@ -159,11 +165,21 @@ All live under `crates/ml/src/trainers/dqn/smoke_tests/`. Each is `#[ignore]`-ga
 Add to the existing HEALTH_DIAG log line in `training_loop.rs`. All forward-only, no backward gradient, cost is one scalar readback per epoch per field.
 
 Track 1 fields (magnitude diagnosis):
+
+Covering H1–H5 (common causes):
 - `q_mag_full`, `q_mag_half`, `q_mag_quarter` — mean Q-value per magnitude bin
-- `var_scale_mean` — average Var[Q]-derived conviction shrink
-- `kelly_f_mean` — average Kelly fraction applied
-- `grad_ratio_mag_dir` — magnitude-head grad norm ÷ direction-head grad norm
-- `action_dist_quarter`, `action_dist_half`, `action_dist_full` — epoch-level usage fractions
+- `var_scale_mean` — average Var[Q]-derived conviction shrink (H2)
+- `kelly_f_mean`, `avg_win_ratio` — Kelly fraction + `avg_win / avg_loss` ratio (H3)
+- `grad_ratio_mag_dir` — magnitude-head grad norm ÷ direction-head grad norm (H4)
+- `action_dist_quarter`, `action_dist_half`, `action_dist_full` — epoch-level usage fractions (H5 indirect)
+
+Covering H6–H10 (codebase-specific):
+- `trail_fire_rate_{quarter, half, full}` — trailing-stop firing rate per magnitude bin (H6)
+- `hold_time_at_exit_{quarter, half, full}` — mean bars held until exit, per magnitude (H6)
+- `vsn_mask_{magnitude, direction}` — VSN mask mean per branch (H7)
+- `noisy_sigma_{mag_head, dir_head}` — NoisyNets σ mean per branch's weights (H7)
+- `target_drift_{mag_head, dir_head}` — cumulative ‖target − online‖ per branch (H8)
+- `action_dist_eval_{quarter, half, full}` — magnitude usage during evaluation rollouts (distinct from training-mode `action_dist_*`) (H10)
 
 Track 2 fields (reward V7 audit):
 - `reward_contrib_popart`, `reward_contrib_cf_flip`, `reward_contrib_trail`, `reward_contrib_micro`, `reward_contrib_loss_aversion`, `reward_contrib_segment_patience` — per-term magnitude as fraction of step_return (see class-specific measurement in §4.1)
@@ -412,7 +428,7 @@ Existing walk-forward config produces 6 folds on `test_data/futures-baseline`.
 | A — Trade count (soft) | ≥100 trades per val window on ≥5/6 folds (per-fold, not averaged) | Metrics kernel |
 | **B — Multi-fold Sharpe (MANDATORY)** | Best val Sharpe annualized > 10 on ≥5/6 folds | `training_loop` Best Sharpe log |
 | **B — Multi-fold WinRate (MANDATORY)** | Val WinRate > 55% on ≥5/6 folds | Metrics kernel |
-| **Surrogate-noise (MANDATORY)** | Random-action Sharpe ≤ 0.5× trained-model Sharpe | `surrogate_noise_check.rs` smoke |
+| **Surrogate-noise (MANDATORY)** | Trained-model pooled Sharpe > 95th-percentile of N=30 surrogate random-action runs | `surrogate_noise_check.rs` smoke |
 | Controller-activity (soft) | No controller fires in > 50% of epochs | HEALTH_DIAG `controller_fire_*` fields |
 
 Criterion B gates AND surrogate-noise check MUST pass for the spec to close successfully and merge to main (Phase 4). Criterion A gates are soft — if B and surrogate-noise pass but some A gate misses, the spec closes with a documented A follow-up. Controller-activity is soft — high firing rates go into a tech-debt follow-up note (sub-project B concern).