# Policy Quality V7 Audit Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Execute the 4-phase V7 audit specified in `docs/superpowers/specs/2026-04-21-policy-quality-design.md` — build measurement substrate, run parallel investigation tracks, land all confirmed fixes in a single convergence commit, validate on 6-fold × 50-epoch L40S, merge to main.

**Architecture:** All commits land on `main`. Tagged `policy-quality-baseline` after Phase 0 as rollback anchor. Phase 2 fixes commit incrementally — each leaves main working. Closing tag `policy-quality-v1` on the final validated commit.

**Tech Stack:** Rust (workspace crates: ml, ml-dqn, ml-core), CUDA (cuBLAS, custom kernels via nvcc), Argo Workflows on Scaleway Kapsule, L40S GPU (sm_89), fxcache binary feature cache, `#[ignore]` smoke tests gated on `FOXHUNT_TEST_DATA`.

**Budget:** ~5.5–6.5 hrs L40S GPU, ~4–6 days elapsed.

**Preconditions (verify before starting):**
- Branch `main` at HEAD, all today's stability fixes landed (commit `ad677466c` or later)
- Local GPU (RTX 3050) working — `cargo test -p ml --release --lib -- td_propagation --ignored` passes
- `argo` CLI configured against Scaleway cluster
- sccache reachable on `cargo-target-cuda` PVC (ensure-binary will verify)

---

## Plan Coverage Notes

This plan is **asymmetrically detailed** by design:

- **Phase 0 (measurement substrate)**, **Phase 3 (validation)**, **Phase 4 (merge)** — TDD detail, exact code, exact commands.
- **Phase 1 (investigation)** — process tasks (run instrumentation, record findings). Each sub-task has concrete command + what to record; no "code to write" because Phase 1 produces data, not code.
- **Phase 2 (convergence fix)** — template only. The spec's §5.1–5.4 decision matrices dictate fixes; the exact code depends on Phase 1 CONFIRMED/REJECTED outcomes. Phase 2 tasks list "for each CONFIRMED finding in Phase 1 triage, implement per the fix sketch; write per-finding smoke test; commit once all findings done." A follow-up plan (`docs/superpowers/plans/YYYY-MM-DD-policy-quality-phase2.md`) with concrete Phase 2 TDD tasks is produced AFTER Phase 1 triage lands.

Rationale: the spec is investigative. Forcing TDD detail on Phase 2 before Phase 1 runs would require guessing which of 30+ hypotheses/audit items confirm — guesswork contradicts V7 discipline.

---

## File Structure

### New files (Phase 0)

| Path | Responsibility |
|---|---|
| `crates/ml/src/trainers/dqn/smoke_tests/magnitude_distribution.rs` | Track 1 smoke: assert F_Half/F_Full ≥ 5% per-bin after 20-epoch train |
| `crates/ml/src/trainers/dqn/smoke_tests/reward_component_audit.rs` | Track 2 smoke: audit 8 reward terms via 3 measurement classes |
| `crates/ml/src/trainers/dqn/smoke_tests/controller_activity.rs` | Track 3 smoke: assert no controller fires in >50% of epochs |
| `crates/ml/src/trainers/dqn/smoke_tests/exploration_coverage.rs` | Track 4 smoke: assert magnitude-branch entropy ≥ 0.5 @ ep5, ≥ 0.3 @ ep20 |
| `crates/ml/src/trainers/dqn/smoke_tests/multi_fold_convergence.rs` | Fast local variant of Phase 3 gate: 3 folds × 20 epochs, ≥2/3 Sharpe > 0 |
| `crates/ml/src/trainers/dqn/smoke_tests/surrogate_noise_check.rs` | False-positive detector: pooled Sharpe vs N=30 random-action 95th %ile |
| `docs/superpowers/specs/2026-04-21-policy-quality-baseline-metrics.md` | Phase 0 baseline reference — HEALTH_DIAG field values from pre-fix code |

### Modified files (Phase 0 — measurement substrate)

| Path | Change scope |
|---|---|
| `crates/ml/src/trainers/dqn/smoke_tests/mod.rs` | Register 6 new test modules |
| `crates/ml/src/trainers/dqn/trainer/training_loop.rs:2160` | Extend HEALTH_DIAG format string with 20+ new fields |
| `crates/ml/src/trainers/dqn/trainer/mod.rs` | Add public getters for new diag fields if needed |
| `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` | Per-magnitude Q reduction, per-branch grad norm, VSN mask readback, NoisyNets σ readback, target drift tracking |
| `crates/ml/src/cuda_pipeline/experience_kernels.cu` | Per-magnitude trail fire counter, hold-time-at-exit tracking, reward-term contribution tracking |

### Modified files (Phase 2 — TBD after Phase 1 triage)

Per-finding. Recorded during Phase 2 planning commit.

### Files untouched in scope

- `infra/k8s/argo/` — workflows already exist, no changes
- `crates/ml-paper-trading/`, `crates/ml/src/deployment/` — out of scope (sub-projects C, D)
- `proto/`, `migrations/`, `config/` — no schema changes expected

---

## Phase 0 — Measurement Substrate

**Goal:** Build smoke tests + HEALTH_DIAG extensions. No behavioral change in training. Baseline metrics captured for Phase 1 reference.

**Exit criteria:** All 6 new smoke tests compile, run, and produce readable output. HEALTH_DIAG fields appear in training logs with non-zero values.

**Phase 0 commits (in order):**
1. Verify starting state
2. HEALTH_DIAG field scaffolding (host-side, zero-returning stubs)
3. Track 1 HEALTH_DIAG wiring (GPU-side readbacks for H1–H10 metrics)
4. Track 2 HEALTH_DIAG wiring (reward-term contribution)
5. Track 3 HEALTH_DIAG wiring (controller fire-rates)
6. Track 4 HEALTH_DIAG wiring (exploration metrics)
7. Smoke test files × 6
8. Baseline metrics document

---

### Task 0.1: Verify starting state

**Files:**
- git only

- [ ] **Step 1: Verify on main, up to date**

Run:
```bash
git checkout main
git pull --ff-only origin main
git log -1 --oneline
```
Expected: clean working tree, at latest main (spec/plan commits present).

- [ ] **Step 2: Note the baseline SHA**

Run: `git rev-parse HEAD | tee /tmp/policy-quality-pre-phase0-sha`
Expected: one SHA. Kept for reference — we'll tag the Phase-0-final commit (not this SHA) as `policy-quality-baseline` at Task 0.17.

---

### Task 0.2: HEALTH_DIAG scaffolding — zero-valued stubs

**Files:**
- Modify: `crates/ml/src/trainers/dqn/trainer/training_loop.rs` — extend HEALTH_DIAG format string + pass zero values

Why this task: landing the format-string and plumbing BEFORE the GPU readbacks means each later task is independent and we always have compiling code.

- [ ] **Step 1: Read the existing HEALTH_DIAG line**

Run: `grep -n "HEALTH_DIAG" crates/ml/src/trainers/dqn/trainer/training_loop.rs | head`
Expected: line 2160 is the format string.

- [ ] **Step 2: Extend format string with new field placeholders**

In `crates/ml/src/trainers/dqn/trainer/training_loop.rs`, find the `info!("HEALTH_DIAG[{}]:..."` block starting at line 2160. Append new field groups BEFORE the closing `);`:

```rust
// Inside the same info! block, append these fields:
// ... existing fields ...
" mag [q_full={:.3} q_half={:.3} q_quarter={:.3} var_scale={:.3} kelly_f={:.3} avg_win_ratio={:.3} grad_ratio_mag_dir={:.4} dist_q={:.3} dist_h={:.3} dist_f={:.3}]"
" trail [fire_q={:.3} fire_h={:.3} fire_f={:.3} hold_q={:.2} hold_h={:.2} hold_f={:.2}]"
" noisy [vsn_mag={:.3} vsn_dir={:.3} sigma_mag={:.4} sigma_dir={:.4} drift_mag={:.3} drift_dir={:.3}]"
" eval_dist [eq={:.3} eh={:.3} ef={:.3}]"
" reward_contrib [popart={:.3} cf={:.3} trail_r={:.3} micro={:.3} la={:.3} seg={:.3}]"
" controller [anti_lr={} tau={} gamma={} clip={} cql={} cost={} fire_frac={:.2}]"
" explore [ent_mag={:.2} ent_dir={:.2} sigma_mean={:.4}]"
```

Then pass zero/false values for each to start. Full edit looks like:

```rust
info!(
    "HEALTH_DIAG[{}]: ... existing format ... \
     mag [q_full={:.3} q_half={:.3} q_quarter={:.3} var_scale={:.3} kelly_f={:.3} avg_win_ratio={:.3} grad_ratio_mag_dir={:.4} dist_q={:.3} dist_h={:.3} dist_f={:.3}] \
     trail [fire_q={:.3} fire_h={:.3} fire_f={:.3} hold_q={:.2} hold_h={:.2} hold_f={:.2}] \
     noisy [vsn_mag={:.3} vsn_dir={:.3} sigma_mag={:.4} sigma_dir={:.4} drift_mag={:.3} drift_dir={:.3}] \
     eval_dist [eq={:.3} eh={:.3} ef={:.3}] \
     reward_contrib [popart={:.3} cf={:.3} trail_r={:.3} micro={:.3} la={:.3} seg={:.3}] \
     controller [anti_lr={} tau={} gamma={} clip={} cql={} cost={} fire_frac={:.2}] \
     explore [ent_mag={:.2} ent_dir={:.2} sigma_mean={:.4}]",
    epoch,
    // ... existing values ...
    // Track 1 — magnitude
    0.0_f32, 0.0_f32, 0.0_f32, 0.0_f32, 0.0_f32, 0.0_f32, 0.0_f32, 0.0_f32, 0.0_f32, 0.0_f32,
    // Track 1 — trail
    0.0_f32, 0.0_f32, 0.0_f32, 0.0_f32, 0.0_f32, 0.0_f32,
    // Track 1 — noisy/VSN/drift
    0.0_f32, 0.0_f32, 0.0_f32, 0.0_f32, 0.0_f32, 0.0_f32,
    // Track 1 — eval dist
    0.0_f32, 0.0_f32, 0.0_f32,
    // Track 2 — reward contrib
    0.0_f32, 0.0_f32, 0.0_f32, 0.0_f32, 0.0_f32, 0.0_f32,
    // Track 3 — controllers
    false, false, false, false, false, false, 0.0_f32,
    // Track 4 — explore
    0.0_f32, 0.0_f32, 0.0_f32,
);
```

- [ ] **Step 3: Compile**

Run: `SQLX_OFFLINE=true cargo check -p ml`
Expected: `Finished` with no errors. Warnings OK.

- [ ] **Step 4: Run existing td_propagation smoke to verify no regression**

Run: `SQLX_OFFLINE=true CUBLAS_WORKSPACE_CONFIG=:4096:8 FOXHUNT_TEST_DATA=test_data/futures-baseline cargo test -p ml --release --lib -- trainers::dqn::smoke_tests::td_propagation::test_td_propagation_sparse_rewards_multi_trial --ignored --nocapture 2>&1 | tail -5`
Expected: `test result: ok. 1 passed`. HEALTH_DIAG line now contains all the new zero fields.

- [ ] **Step 5: Commit**

Run:
```bash
git add crates/ml/src/trainers/dqn/trainer/training_loop.rs
git commit -m "diag(policy-quality): scaffold HEALTH_DIAG fields for 4-track V7 audit

All new fields wired with zero values. Later tasks replace zeros with
real GPU-sourced measurements. Smoke tests referenced by the design
spec at docs/superpowers/specs/2026-04-21-policy-quality-design.md"
git push
```

---

### Task 0.3: Track 1 (magnitude) — per-magnitude Q-value and var_scale readback

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` — add per-bin Q accessor
- Modify: `crates/ml/src/trainers/dqn/trainer/training_loop.rs` — replace zero stubs for q_mag_* / var_scale_mean / kelly_f_mean / avg_win_ratio

- [ ] **Step 1: Locate the existing Q-reduction path**

Run: `grep -n "reduce_current_q_stats\|q_value_mean\|avg_q_value" crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs | head`
Expected: find the reduction path used by the existing `avg_q_value` field.

- [ ] **Step 2: Add per-magnitude Q kernel (or extend the existing one)**

In `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs`, add a method:

```rust
impl GpuDqnTrainer {
    /// Returns (q_full, q_half, q_quarter) — mean Q-value per magnitude bin
    /// over the current batch. Launches the existing q_reduce kernel with
    /// magnitude-bin grouping.
    pub fn per_magnitude_q_means(&self) -> Result<[f32; 3], MLError> {
        // Use existing Q-readback infrastructure — iterate over the last
        // forward pass's q_all buffer, group by magnitude bin (mag = (action / 9) % 3),
        // sum + count per bin, divide.
        // Implementation: launch `per_magnitude_q_reduce` kernel (to be added
        // to monitoring_kernel.cu) that writes 3 f32 values to a pinned slot.
        // For Phase 0 scaffolding: return [0.0; 3] until kernel wired.
        Ok([0.0_f32; 3])
    }
}
```

Note: for Phase 0, this returns zeros. The actual kernel wiring is a later task if measurement confirms it's needed. Goal here is plumbing.

- [ ] **Step 3: Wire var_scale_mean and kelly_f_mean (already computed on GPU)**

The variance-conviction scaling happens in `experience_kernels.cu` per-sample. Add a host-side running mean in the trainer:

```rust
impl GpuDqnTrainer {
    pub fn var_scale_epoch_mean(&self) -> f32 {
        self.var_scale_running_mean // already tracked; just expose
    }
    pub fn kelly_f_epoch_mean(&self) -> f32 { self.kelly_f_running_mean }
    pub fn avg_win_ratio(&self) -> f32 {
        if self.kelly_loss_count < 1.0 { return 1.0; }
        (self.kelly_sum_wins / self.kelly_win_count.max(1.0))
            / (self.kelly_sum_losses / self.kelly_loss_count.max(1.0)).max(0.001)
    }
}
```

If `var_scale_running_mean` doesn't exist yet, add it to the trainer struct + update per-step in the experience-collection loop.

- [ ] **Step 4: Replace zero stubs in training_loop.rs**

In the HEALTH_DIAG line, change the Track-1 magnitude zero values to real calls:

```rust
// Replace:
//   0.0_f32, 0.0_f32, 0.0_f32, 0.0_f32, 0.0_f32, 0.0_f32, 0.0_f32, 0.0_f32, 0.0_f32, 0.0_f32,
// With:
{
    let [q_f, q_h, q_q] = self.fused_ctx.as_ref()
        .map(|f| f.trainer().per_magnitude_q_means().unwrap_or([0.0; 3]))
        .unwrap_or([0.0; 3]);
    let var_scale = self.fused_ctx.as_ref().map(|f| f.trainer().var_scale_epoch_mean()).unwrap_or(0.0);
    let kelly_f = self.fused_ctx.as_ref().map(|f| f.trainer().kelly_f_epoch_mean()).unwrap_or(0.0);
    let avg_wr = self.fused_ctx.as_ref().map(|f| f.trainer().avg_win_ratio()).unwrap_or(1.0);
    let grad_ratio = 0.0_f32; // wired in next task
    let dist_q = self.action_dist[0]; let dist_h = self.action_dist[1]; let dist_f = self.action_dist[2];
    (q_f, q_h, q_q, var_scale, kelly_f, avg_wr, grad_ratio, dist_q, dist_h, dist_f)
}
```

Where `self.action_dist` is computed in an earlier block (from the per-epoch actions_history). If it doesn't exist yet, add it.

- [ ] **Step 5: Compile + smoke**

Run: `SQLX_OFFLINE=true cargo check -p ml && SQLX_OFFLINE=true CUBLAS_WORKSPACE_CONFIG=:4096:8 FOXHUNT_TEST_DATA=test_data/futures-baseline cargo test -p ml --release --lib -- td_propagation --ignored --nocapture 2>&1 | tail -10`
Expected: builds clean; td_propagation passes; HEALTH_DIAG line now shows non-zero q_full/q_half/q_quarter/var_scale/kelly_f values.

- [ ] **Step 6: Commit**

```bash
git add crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs crates/ml/src/trainers/dqn/trainer/training_loop.rs
git commit -m "diag(policy-quality): wire Track 1 magnitude Q + var_scale + kelly HEALTH_DIAG"
git push
```

---

### Task 0.4: Track 1 — per-branch grad norm ratio (H4 detection)

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` — expose per-branch grad norm from Adam grad step

- [ ] **Step 1: Locate the per-weight grad norm reduction**

Run: `grep -n "grad_norm\|adam_grad\|grad_reduce" crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs | head -20`
Expected: find where grad_norm is computed globally. Find the weight tensors `w_b1fc`, `w_b0fc` etc.

- [ ] **Step 2: Add per-branch grad norm tracking**

```rust
// In GpuDqnTrainer:
pub fn grad_norm_magnitude_head(&self) -> f32 {
    // Sum grad²  for w_b1fc (magnitude head weights).
    // Use existing adam_grad buffer, sum² over the appropriate slice range.
    self.per_branch_grad_norm[1] // 0=dir, 1=mag, 2=order, 3=urgency
}
pub fn grad_norm_direction_head(&self) -> f32 { self.per_branch_grad_norm[0] }
pub fn grad_ratio_mag_dir(&self) -> f32 {
    let mag = self.grad_norm_magnitude_head();
    let dir = self.grad_norm_direction_head();
    if dir > 1e-9 { mag / dir } else { 0.0 }
}
```

Where `self.per_branch_grad_norm: [f32; 4]` is populated during the Adam grad step. Add population: reduce `adam_grad[slice_for_branch_k]` into per-branch scalars.

- [ ] **Step 3: Update HEALTH_DIAG**

Replace `grad_ratio = 0.0_f32; // wired in next task` with:

```rust
let grad_ratio = self.fused_ctx.as_ref()
    .map(|f| f.trainer().grad_ratio_mag_dir())
    .unwrap_or(0.0);
```

- [ ] **Step 4: Compile + smoke**

Run: `SQLX_OFFLINE=true cargo check -p ml && SQLX_OFFLINE=true CUBLAS_WORKSPACE_CONFIG=:4096:8 FOXHUNT_TEST_DATA=test_data/futures-baseline cargo test -p ml --release --lib -- td_propagation --ignored --nocapture 2>&1 | tail -5`
Expected: test passes; grad_ratio_mag_dir shows a non-zero value (likely ~0.05–0.5 for a healthy network).

- [ ] **Step 5: Commit**

```bash
git add crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs crates/ml/src/trainers/dqn/trainer/training_loop.rs
git commit -m "diag(policy-quality): wire Track 1 per-branch grad norm ratio (H4 signal)"
git push
```

---

### Task 0.5: Track 1 — trailing-stop per-magnitude fire rate (H6 detection)

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/experience_kernels.cu` — add per-magnitude trail counter
- Modify: `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` — readback trail fire rates

- [ ] **Step 1: Locate trail-stop firing site**

Run: `grep -n "trail_triggered\|trailing_stop\|check_trailing_stop" crates/ml/src/cuda_pipeline/experience_kernels.cu crates/ml/src/cuda_pipeline/trade_physics.cuh | head`
Expected: find where `trail_triggered = 1` in `unified_env_step_core` or similar.

- [ ] **Step 2: Add per-magnitude trail counter kernel**

In `experience_kernels.cu`, after the trail-trigger branch:

```c
// After trail_triggered is set to 1, increment per-magnitude counter
// trail_counts is a [3]-long i32 device buffer in GpuDqnTrainer.
if (trail_triggered) {
    // mag_idx is the current sample's magnitude bin (0=quarter, 1=half, 2=full)
    atomicAdd(&trail_counts_ptr[mag_idx], 1);
}
```

Pass `trail_counts_ptr` + `trade_counts_ptr` ([3] counts of total trades per bin) through the existing kernel arg list (add to prototype in both .cu and Rust loader).

- [ ] **Step 3: Readback in trainer + compute rate**

```rust
// In GpuDqnTrainer, after each epoch:
pub fn trail_fire_rates(&self) -> [f32; 3] {
    let mut h_trails = [0_i32; 3];
    let mut h_trades = [0_i32; 3];
    self.stream.memcpy_dtoh(&self.trail_counts, &mut h_trails).ok();
    self.stream.memcpy_dtoh(&self.trade_counts_per_mag, &mut h_trades).ok();
    let mut out = [0.0_f32; 3];
    for i in 0..3 {
        out[i] = if h_trades[i] > 0 { h_trails[i] as f32 / h_trades[i] as f32 } else { 0.0 };
    }
    // Zero counters for next epoch.
    self.stream.memset_zeros(&mut self.trail_counts).ok();
    self.stream.memset_zeros(&mut self.trade_counts_per_mag).ok();
    out
}

pub fn hold_time_at_exit_mean(&self) -> [f32; 3] {
    // Similar: sum hold_time at exit per bin, divide by exit count.
    // Counters tracked alongside trail_counts.
    self.hold_at_exit_mean
}
```

Note: this is a RARE case where atomicAdd is acceptable — the counts are debug-only per-epoch, not part of training gradient flow. Document in the kernel with a comment.

- [ ] **Step 4: Update HEALTH_DIAG**

Replace the `trail` group's 6 zeros with real calls:

```rust
let trail = self.fused_ctx.as_ref().map(|f| f.trainer().trail_fire_rates()).unwrap_or([0.0; 3]);
let hold = self.fused_ctx.as_ref().map(|f| f.trainer().hold_time_at_exit_mean()).unwrap_or([0.0; 3]);
// In the info! args: trail[0], trail[1], trail[2], hold[0], hold[1], hold[2]
```

- [ ] **Step 5: Compile + smoke**

Run: `touch crates/ml/src/cuda_pipeline/experience_kernels.cu && SQLX_OFFLINE=true cargo build -p ml --release 2>&1 | tail -5`
Expected: cubin rebuilds, release build succeeds.

Then: `SQLX_OFFLINE=true CUBLAS_WORKSPACE_CONFIG=:4096:8 FOXHUNT_TEST_DATA=test_data/futures-baseline cargo test -p ml --release --lib -- td_propagation --ignored --nocapture 2>&1 | tail -5`
Expected: passes; HEALTH_DIAG now shows per-mag trail fire rates.

- [ ] **Step 6: Commit**

```bash
git add crates/ml/src/cuda_pipeline/experience_kernels.cu crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs crates/ml/src/trainers/dqn/trainer/training_loop.rs
git commit -m "diag(policy-quality): Track 1 trailing-stop per-magnitude fire/hold (H6 signal)

Per-bin atomicAdd is debug-only, not on training gradient path — does
not violate the graph-capture determinism policy (counters read
outside capture region at end of epoch)."
git push
```

---

### Task 0.6: Track 1 — VSN mask + NoisyNets σ + target drift (H7, H8 detection)

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` — weight readbacks for VSN mask / σ / target drift

- [ ] **Step 1: Locate VSN + NoisyNets weight tensors**

Run: `grep -n "vsn\|noisy_sigma\|w_b1fc" crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs | head -15`
Expected: find VSN mask tensor + σ tensor names for magnitude and direction heads.

- [ ] **Step 2: Add readback methods**

```rust
pub fn vsn_mask_mean(&self) -> [f32; 2] {
    // [magnitude_branch_mean, direction_branch_mean]
    // Read vsn_mask_buf, average over the magnitude-head slice and direction-head slice.
    // Result is a 2-slot f32 pinned slot, computed end-of-epoch via small reduction.
    self.vsn_mask_mean_cached
}

pub fn noisy_sigma_per_branch(&self) -> [f32; 2] {
    // [sigma_mag_head_mean, sigma_dir_head_mean]
    // Read noisy σ tensor, average over b1_head and b0_head slices.
    self.noisy_sigma_per_branch_cached
}

pub fn target_drift_per_branch(&self) -> [f32; 2] {
    // [drift_mag, drift_dir]
    // At each target update, compute ‖target_w − online_w‖ per branch slice, accumulate.
    self.target_drift_per_branch_cached
}
```

- [ ] **Step 3: Update HEALTH_DIAG**

Replace the `noisy` group's zeros with real calls.

- [ ] **Step 4: Compile + smoke**

Same as prior tasks. Verify non-zero VSN masks + σ values appear in log.

- [ ] **Step 5: Commit**

```bash
git commit -am "diag(policy-quality): Track 1 VSN/NoisyNets/target-drift per-branch (H7,H8 signals)"
git push
```

---

### Task 0.7: Track 1 — eval-mode action distribution (H10 detection)

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` — track action dist during eval rollouts
- Modify: `crates/ml/src/trainers/dqn/trainer/training_loop.rs` — add eval-mode flag to action_dist accumulation

- [ ] **Step 1: Find where validation backtest actions flow**

Run: `grep -n "GpuBacktestEvaluator\|eval_.*action\|val_actions" crates/ml/src/trainers/dqn/trainer/training_loop.rs | head`

- [ ] **Step 2: Add eval_action_dist accumulation**

In the validation rollout path (where `evaluate()` or similar returns metrics), read back the actions_history buffer and compute per-magnitude distribution, store in a trainer field accessible by HEALTH_DIAG.

- [ ] **Step 3: Update HEALTH_DIAG**

Replace the `eval_dist` group's 3 zeros with calls to `self.last_eval_action_dist`.

- [ ] **Step 4: Compile + smoke**

Verify: training-mode `dist_*` and eval-mode `eq/eh/ef` both appear and can differ.

- [ ] **Step 5: Commit**

```bash
git commit -am "diag(policy-quality): Track 1 eval-mode action distribution (H10 signal)"
git push
```

---

### Task 0.8: Track 2 — reward term contribution instrumentation

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/experience_kernels.cu` — add per-term contribution sum buffer
- Modify: `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` — readback + normalization

- [ ] **Step 1: Identify additive reward insertion points in kernel**

Run: `grep -n "step_ret_core\|step_return\|micro_reward\|loss_aversion\|segment_patience" crates/ml/src/cuda_pipeline/experience_kernels.cu | head -20`

- [ ] **Step 2: Add 6-slot per-term contribution buffer**

Alongside the existing reward computation in `experience_env_step`, accumulate |each_additive_term| into `reward_contrib_buf[6]`, and |step_return_total| into `reward_total_sum`. Atomic additions are fine at this scale (not in capture graph).

For transform terms (PopArt) and sample-selector terms (CF flip, trailing stop), record the SIGNAL per the spec's 3-class measurement:
- PopArt: `|post_popart − pre_popart|` into `reward_contrib_buf[0]`
- CF flip: firing rate into `reward_contrib_buf[1]`
- Trailing stop: firing rate + grad-delta signal into `reward_contrib_buf[2]`

Additive terms (micro, loss_aversion, segment_patience) fill slots 3, 4, 5.

- [ ] **Step 3: Readback + normalize in trainer**

```rust
pub fn reward_contrib_fractions(&self) -> [f32; 6] {
    let mut h = [0.0_f32; 6];
    let mut total = 0.0_f32;
    // Read via memcpy_dtoh from pinned buffer
    // For additive terms: fraction = |term_sum| / |total_sum|
    // For transform: fraction = |delta| / |pre| ratio
    // For sample-selector: fraction = fire_rate (already in [0,1])
    h
}
```

- [ ] **Step 4: Update HEALTH_DIAG**

Replace the `reward_contrib` group's 6 zeros.

- [ ] **Step 5: Compile + smoke + commit**

```bash
git commit -am "diag(policy-quality): Track 2 reward-term contribution fractions"
git push
```

---

### Task 0.9: Track 3 — controller fire-rate tracking

**Files:**
- Modify: `crates/ml/src/trainers/dqn/trainer/training_loop.rs` — boolean flags set by each controller's adjustment code

- [ ] **Step 1: Find each controller's adjustment site**

The spec §5.3 lists C1–C7. For each, locate the code block that makes the adjustment and wrap with a `did_fire = true` assignment:

```rust
// Anti-LR (C1) — around line 2480:
let mut did_fire_anti_lr = false;
if /* anti-LR conditions */ {
    did_fire_anti_lr = true;
    // ... existing adjustment ...
}
self.controller_fire_epoch.anti_lr = did_fire_anti_lr;

// Similarly for:
// C2 adaptive-tau — search `adaptive_tau`
// C3 adaptive-gamma — search `adaptive_gamma`
// C4 adaptive grad-clip — search `adaptive_clip`
// C5 CQL alpha schedule — search `cql_alpha`
// C6 cost-anneal — search `cost_anneal`
```

- [ ] **Step 2: Add `ControllerFire` struct and epoch-level state**

```rust
#[derive(Default, Clone, Copy)]
pub struct ControllerFire {
    pub anti_lr: bool,
    pub tau: bool,
    pub gamma: bool,
    pub clip: bool,
    pub cql: bool,
    pub cost: bool,
}

impl ControllerFire {
    pub fn any(&self) -> bool {
        self.anti_lr || self.tau || self.gamma || self.clip || self.cql || self.cost
    }
}

// In DQNTrainer:
pub controller_fire_epoch: ControllerFire,
pub controllers_fire_count: u32,     // running: epochs where any fired
pub controllers_fire_total: u32,     // running: total epochs
```

- [ ] **Step 3: Update HEALTH_DIAG**

```rust
let cf = self.controller_fire_epoch;
let fire_frac = if self.controllers_fire_total > 0 {
    self.controllers_fire_count as f32 / self.controllers_fire_total as f32
} else { 0.0 };
// In info! args for controller group: cf.anti_lr, cf.tau, cf.gamma, cf.clip, cf.cql, cf.cost, fire_frac
```

Also increment counters at end of each epoch.

- [ ] **Step 4: Compile + smoke + commit**

```bash
git commit -am "diag(policy-quality): Track 3 controller fire-rate tracking"
git push
```

---

### Task 0.10: Track 4 — exploration entropy per-branch

**Files:**
- Modify: `crates/ml/src/trainers/dqn/trainer/training_loop.rs` — per-branch entropy from existing action dist

- [ ] **Step 1: Compute entropy from action distribution**

Already have `action_dist[0..3]` for magnitude. Need direction too. Add:

```rust
fn entropy_from_dist(dist: &[f32]) -> f32 {
    let sum: f32 = dist.iter().sum();
    if sum < 1e-9 { return 0.0; }
    let mut h = 0.0;
    for &p in dist {
        let p = p / sum;
        if p > 1e-9 { h -= p * p.ln(); }
    }
    h / (dist.len() as f32).ln()  // normalized to [0, 1]
}

// In the training loop epoch-end:
let ent_mag = entropy_from_dist(&self.action_dist_magnitude);
let ent_dir = entropy_from_dist(&self.action_dist_direction);
```

- [ ] **Step 2: NoisyNets σ mean (reuse Task 0.6 per-branch, compute overall)**

Add to `GpuDqnTrainer` in `gpu_dqn_trainer.rs`:

```rust
/// Returns the average NoisyNets σ across ALL network weights
/// (not per-branch). Used by HEALTH_DIAG exploration group.
pub fn noisy_sigma_overall_mean(&self) -> f32 {
    let pb = self.noisy_sigma_per_branch();  // [mag, dir] from Task 0.6
    // Per-branch averages weighted equally — OK because weight counts
    // are similar across heads. For more rigor, weight by tensor size.
    (pb[0] + pb[1]) * 0.5
}
```

Then in training_loop.rs HEALTH_DIAG block:

```rust
let sigma_mean = self.fused_ctx.as_ref()
    .map(|f| f.trainer().noisy_sigma_overall_mean())
    .unwrap_or(0.0);
```

- [ ] **Step 3: Update HEALTH_DIAG**

Replace `explore` group's 3 zeros.

- [ ] **Step 4: Compile + smoke + commit**

```bash
git commit -am "diag(policy-quality): Track 4 exploration entropy + sigma mean"
git push
```

---

### Task 0.11: Smoke test — `magnitude_distribution.rs`

**Files:**
- Create: `crates/ml/src/trainers/dqn/smoke_tests/magnitude_distribution.rs`
- Modify: `crates/ml/src/trainers/dqn/smoke_tests/mod.rs` — register module

- [ ] **Step 1: Write failing test**

Create `crates/ml/src/trainers/dqn/smoke_tests/magnitude_distribution.rs`:

```rust
//! Smoke test: magnitude distribution (Track 1 gate from policy-quality spec).

use super::helpers::{init_deterministic_cuda, smoke_trainer_with, smoke_params, test_data_dir};
use anyhow::Result;

#[test]
#[ignore] // Requires fxcache — run via:
          //   FOXHUNT_TEST_DATA=test_data/futures-baseline cargo test -p ml --lib \
          //   -- test_magnitude_distribution --ignored --nocapture
fn test_magnitude_distribution() -> Result<()> {
    init_deterministic_cuda();
    let data_dir = test_data_dir().expect("FOXHUNT_TEST_DATA or test_data/ must exist");
    let mut params = smoke_params();
    params.epochs = 20;
    params.early_stopping_enabled = false;
    params.min_epochs_before_stopping = 20;
    let mut trainer = smoke_trainer_with(params)?;
    let rt = tokio::runtime::Builder::new_current_thread().enable_all().build()?;
    let _metrics = rt.block_on(trainer.train(&data_dir, "ES.FUT", |_e, _b, _best| Ok("skip".to_owned())))?;

    // Smoke gate (not production gate — production is 15% in spec §2.1)
    let dist = trainer.magnitude_action_dist_final();  // [quarter, half, full]
    println!("[MAG_DIST] Quarter={:.3} Half={:.3} Full={:.3}", dist[0], dist[1], dist[2]);
    assert!(
        dist[1] >= 0.05,
        "F_Half {:.3} < 0.05 smoke threshold — magnitude branch under-explored",
        dist[1]
    );
    assert!(
        dist[2] >= 0.05,
        "F_Full {:.3} < 0.05 smoke threshold — magnitude branch under-explored",
        dist[2]
    );
    Ok(())
}
```

- [ ] **Step 2: Register in `mod.rs`**

Add `pub mod magnitude_distribution;` to `crates/ml/src/trainers/dqn/smoke_tests/mod.rs`.

- [ ] **Step 3: Add `magnitude_action_dist_final` accessor to DQNTrainer**

If not already present, expose the per-magnitude action distribution after training completes:

```rust
// crates/ml/src/trainers/dqn/trainer/mod.rs
impl DQNTrainer {
    pub fn magnitude_action_dist_final(&self) -> [f32; 3] {
        self.last_action_dist_magnitude  // populated during train()
    }
}
```

- [ ] **Step 4: Compile + run (expected to FAIL the assertion on current code)**

Run: `SQLX_OFFLINE=true CUBLAS_WORKSPACE_CONFIG=:4096:8 FOXHUNT_TEST_DATA=test_data/futures-baseline cargo test -p ml --release --lib -- magnitude_distribution --ignored --nocapture 2>&1 | tail -15`
Expected: **FAIL** with `F_Half 0.00X < 0.05` (confirms current magnitude collapse). This is the pre-fix baseline — the test will pass after Phase 2.

- [ ] **Step 5: Commit**

```bash
git add crates/ml/src/trainers/dqn/smoke_tests/magnitude_distribution.rs crates/ml/src/trainers/dqn/smoke_tests/mod.rs crates/ml/src/trainers/dqn/trainer/mod.rs
git commit -m "test(policy-quality): add magnitude_distribution smoke (Track 1 gate)

Expected to FAIL on current main — documents the bug. Will pass after
Phase 2 convergence commit lands the magnitude fixes."
git push
```

---

### Task 0.12: Smoke test — `reward_component_audit.rs`

**Files:**
- Create: `crates/ml/src/trainers/dqn/smoke_tests/reward_component_audit.rs`
- Modify: `crates/ml/src/trainers/dqn/smoke_tests/mod.rs`

- [ ] **Step 1: Write test**

```rust
//! Smoke test: reward component audit (Track 2 from policy-quality spec).

use super::helpers::{init_deterministic_cuda, smoke_trainer_with, smoke_params, test_data_dir};
use anyhow::Result;

#[test]
#[ignore]
fn test_reward_components_contribute() -> Result<()> {
    init_deterministic_cuda();
    let data_dir = test_data_dir().expect("FOXHUNT_TEST_DATA or test_data/ must exist");
    let mut params = smoke_params();
    params.epochs = 20;
    params.early_stopping_enabled = false;
    params.min_epochs_before_stopping = 20;
    let mut trainer = smoke_trainer_with(params)?;
    let rt = tokio::runtime::Builder::new_current_thread().enable_all().build()?;
    let _m = rt.block_on(trainer.train(&data_dir, "ES.FUT", |_e, _b, _best| Ok("skip".to_owned())))?;

    let contribs = trainer.reward_component_audit_summary();
    // Returned order per spec: [popart, cf_flip, trail, micro, loss_aversion, segment_patience]
    // Measurement class per term:
    //   additive → median epoch fraction of total reward
    //   transform (popart) → median epoch |delta|/|pre| ratio
    //   sample-selector (cf_flip, trail) → fire-rate in [0, 1]
    println!("[REWARD_AUDIT] {:?}", contribs);
    // Smoke-level assertion: none are NaN/Inf.
    for (i, &c) in contribs.iter().enumerate() {
        assert!(c.is_finite(), "reward_contrib[{}] = {} not finite", i, c);
    }
    // Full decision (DELETE if <1% of epochs over threshold) happens during Phase 1
    // triage, not here — this is a pre-flight check.
    Ok(())
}
```

- [ ] **Step 2: Register module in `mod.rs`**

- [ ] **Step 3: Add `reward_component_audit_summary` accessor**

```rust
// In DQNTrainer:
pub fn reward_component_audit_summary(&self) -> [f32; 6] {
    self.last_reward_contrib_fractions
}
```

- [ ] **Step 4: Compile + run**

Expected: passes (assertion is just is_finite). Output documents baseline contribution fractions.

- [ ] **Step 5: Commit**

```bash
git commit -am "test(policy-quality): add reward_component_audit smoke (Track 2 gate)"
git push
```

---

### Task 0.13: Smoke test — `controller_activity.rs`

**Files:**
- Create: `crates/ml/src/trainers/dqn/smoke_tests/controller_activity.rs`

- [ ] **Step 1: Write test**

```rust
//! Smoke test: controller activity (Track 3 from policy-quality spec).

use super::helpers::{init_deterministic_cuda, smoke_trainer_with, smoke_params, test_data_dir};
use anyhow::Result;

#[test]
#[ignore]
fn test_controllers_not_load_bearing() -> Result<()> {
    init_deterministic_cuda();
    let data_dir = test_data_dir().expect("FOXHUNT_TEST_DATA or test_data/ must exist");
    let mut params = smoke_params();
    params.epochs = 20;
    let mut trainer = smoke_trainer_with(params)?;
    let rt = tokio::runtime::Builder::new_current_thread().enable_all().build()?;
    let _m = rt.block_on(trainer.train(&data_dir, "ES.FUT", |_e, _b, _best| Ok("skip".to_owned())))?;

    let fires = trainer.controller_fire_rates_final();
    // Returned order: [anti_lr, tau, gamma, clip, cql, cost]
    println!("[CONTROLLER_FIRE] {:?}", fires);
    // High firing rate = load-bearing = production risk per spec §5.3.
    for (name, rate) in ["anti_lr", "tau", "gamma", "clip", "cql", "cost"].iter().zip(fires.iter()) {
        assert!(
            *rate <= 0.5,
            "Controller '{}' fires in {:.1}% of epochs (>50% = load-bearing)",
            name, rate * 100.0
        );
    }
    Ok(())
}
```

- [ ] **Step 2: Register module in `mod.rs`**

Add `pub mod controller_activity;` to `crates/ml/src/trainers/dqn/smoke_tests/mod.rs`.

- [ ] **Step 3: Add `controller_fire_rates_final` accessor**

In `crates/ml/src/trainers/dqn/trainer/mod.rs`:

```rust
impl DQNTrainer {
    /// Returns [anti_lr, tau, gamma, clip, cql, cost] firing rates across
    /// the training epochs that ran. Each entry is the fraction of epochs
    /// in which the named controller made an adjustment.
    pub fn controller_fire_rates_final(&self) -> [f32; 6] {
        let total = (self.controllers_fire_total as f32).max(1.0);
        [
            self.controller_fire_counts.anti_lr as f32 / total,
            self.controller_fire_counts.tau as f32 / total,
            self.controller_fire_counts.gamma as f32 / total,
            self.controller_fire_counts.clip as f32 / total,
            self.controller_fire_counts.cql as f32 / total,
            self.controller_fire_counts.cost as f32 / total,
        ]
    }
}
```

Add `controller_fire_counts: ControllerFireCounts` field to DQNTrainer, increment in Task 0.9.

- [ ] **Step 4: Run — expected to FAIL**

Run: `SQLX_OFFLINE=true CUBLAS_WORKSPACE_CONFIG=:4096:8 FOXHUNT_TEST_DATA=test_data/futures-baseline cargo test -p ml --release --lib -- controller_activity --ignored --nocapture 2>&1 | tail -10`
Expected: FAIL — anti-LR currently fires ~100% of epochs. Documents the issue per spec §5.3.

- [ ] **Step 5: Commit even if failing — records baseline**

```bash
git commit -am "test(policy-quality): add controller_activity smoke (Track 3 gate)"
git push
```

---

### Task 0.14: Smoke test — `exploration_coverage.rs`

**Files:**
- Create: `crates/ml/src/trainers/dqn/smoke_tests/exploration_coverage.rs`

- [ ] **Step 1: Write test**

```rust
//! Smoke test: exploration coverage (Track 4 from policy-quality spec).

use super::helpers::{init_deterministic_cuda, smoke_trainer_with, smoke_params, test_data_dir};
use anyhow::Result;

#[test]
#[ignore]
fn test_exploration_covers_magnitude_space() -> Result<()> {
    init_deterministic_cuda();
    let data_dir = test_data_dir().expect("FOXHUNT_TEST_DATA or test_data/ must exist");
    let mut params = smoke_params();
    params.epochs = 20;
    let mut trainer = smoke_trainer_with(params)?;
    let rt = tokio::runtime::Builder::new_current_thread().enable_all().build()?;
    let _m = rt.block_on(trainer.train(&data_dir, "ES.FUT", |_e, _b, _best| Ok("skip".to_owned())))?;

    let entropies = trainer.exploration_entropy_history();
    // entropies: Vec<(u32 epoch, f32 ent_magnitude)>
    // Spec: ≥ 0.5 at epoch 5, ≥ 0.3 at epoch 20.
    let ep5 = entropies.iter().find(|&&(e, _)| e == 5).map(|&(_, v)| v).unwrap_or(0.0);
    let ep20 = entropies.iter().find(|&&(e, _)| e == 19 /* 0-indexed */ || e == 20)
        .map(|&(_, v)| v).unwrap_or(0.0);
    println!("[EXPLORE] magnitude entropy @5={:.3} @20={:.3}", ep5, ep20);
    assert!(ep5 >= 0.5, "magnitude entropy at epoch 5 = {:.3} < 0.5 (exploration quenched too fast)", ep5);
    assert!(ep20 >= 0.3, "magnitude entropy at epoch 20 = {:.3} < 0.3 (full collapse)", ep20);
    Ok(())
}
```

- [ ] **Step 2: Register module in `mod.rs`**

- [ ] **Step 3: Add `exploration_entropy_history` accessor**

```rust
// In DQNTrainer:
/// Returns the per-epoch magnitude-branch entropy history as
/// (epoch_0_indexed, entropy_normalized_to_0_1) pairs.
pub fn exploration_entropy_history(&self) -> Vec<(u32, f32)> {
    self.explore_entropy_mag_history.clone()
}
```

The field `explore_entropy_mag_history: Vec<(u32, f32)>` is appended at end of each epoch in the training loop (see Task 0.10 where per-branch entropy is computed for HEALTH_DIAG — reuse the same value).

- [ ] **Step 4: Run — expected to FAIL**

Run: `SQLX_OFFLINE=true CUBLAS_WORKSPACE_CONFIG=:4096:8 FOXHUNT_TEST_DATA=test_data/futures-baseline cargo test -p ml --release --lib -- exploration_coverage --ignored --nocapture 2>&1 | tail -10`
Expected: may FAIL — documents current exploration dynamics.

- [ ] **Step 5: Commit**

```bash
git commit -am "test(policy-quality): add exploration_coverage smoke (Track 4 gate)"
git push
```

---

### Task 0.15: Smoke test — `multi_fold_convergence.rs`

**Files:**
- Create: `crates/ml/src/trainers/dqn/smoke_tests/multi_fold_convergence.rs`

- [ ] **Step 1: Write test**

```rust
//! Smoke test: multi-fold convergence (fast local variant of Phase 3 L40S gate).

use super::helpers::{init_deterministic_cuda, test_data_dir};
use anyhow::Result;

#[test]
#[ignore]
fn test_multi_fold_convergence() -> Result<()> {
    init_deterministic_cuda();
    let data_dir = test_data_dir().expect("FOXHUNT_TEST_DATA or test_data/ must exist");
    // Use the existing train_baseline_rl example as a subprocess — it already knows
    // how to run walk-forward on fxcache. We pass --epochs 20 --max-folds 3.
    // This matches the L40S Phase 3 run but is smaller.
    let status = std::process::Command::new("cargo")
        .args(&[
            "run", "--release", "-p", "ml", "--example", "train_baseline_rl", "--",
            "--data-dir", data_dir.to_str().unwrap(),
            "--symbol", "ES.FUT",
            "--epochs", "20",
            "--max-folds", "3",
            "--output-dir", "/tmp/policy_quality_multi_fold",
        ])
        .status()?;
    assert!(status.success(), "train_baseline_rl exited non-zero");

    // Read DONE marker, metrics.json
    let metrics_path = std::path::Path::new("/tmp/policy_quality_multi_fold/metrics.json");
    assert!(metrics_path.exists(), "metrics.json not produced");
    let metrics: serde_json::Value = serde_json::from_reader(std::fs::File::open(metrics_path)?)?;
    let folds = metrics["folds"].as_array().expect("folds array");
    assert_eq!(folds.len(), 3, "expected 3 folds, got {}", folds.len());
    let positive_sharpe_count = folds.iter()
        .filter(|f| f["best_val_sharpe"].as_f64().unwrap_or(0.0) > 0.0)
        .count();
    assert!(
        positive_sharpe_count >= 2,
        "only {}/3 folds had positive Best Sharpe (want ≥2)",
        positive_sharpe_count
    );
    Ok(())
}
```

Note: this smoke test requires `--max-folds` argument on `train_baseline_rl`. If that doesn't exist, add it as part of this task (inspect the Args struct in `crates/ml/examples/train_baseline_rl.rs`).

- [ ] **Step 2–5: Register, add `--max-folds` arg if missing, run, commit**

```bash
git commit -am "test(policy-quality): add multi_fold_convergence smoke (fast Phase 3 local)"
git push
```

---

### Task 0.16: Smoke test — `surrogate_noise_check.rs`

**Files:**
- Create: `crates/ml/src/trainers/dqn/smoke_tests/surrogate_noise_check.rs`

- [ ] **Step 1: Write test**

```rust
//! Smoke test: surrogate-noise false-positive detector (spec §4.1).
//!
//! Mandatory gate. Runs a trained model against 30 random-action surrogate
//! backtests; trained pooled Sharpe must exceed the 95th percentile of the
//! surrogate distribution. Protects against look-ahead / leakage bugs.

use super::helpers::test_data_dir;
use anyhow::Result;
use rand::{Rng, SeedableRng, rngs::StdRng};

#[test]
#[ignore]
fn test_surrogate_noise_check() -> Result<()> {
    let data_dir = test_data_dir().expect("FOXHUNT_TEST_DATA or test_data/ must exist");
    let ckpt = std::path::Path::new("/workspace/output/dqn_fold0_best.safetensors");
    assert!(
        ckpt.exists(),
        "surrogate-noise test requires a trained checkpoint at {:?}. \
         Run multi_fold_convergence or Phase 3 validation first.",
        ckpt
    );

    // Run trained-model backtest (existing evaluate_baseline path, aggregated across all 6 folds)
    let trained_sharpe = run_trained_backtest_pooled(&data_dir, ckpt)?;
    println!("[SURROGATE] trained pooled Sharpe = {:.3}", trained_sharpe);

    // Collect action marginal distribution from the trained-model backtest
    let trained_action_marginals = load_trained_action_marginals(&data_dir, ckpt)?;

    // Run N=30 surrogate backtests with matched marginal random actions
    let mut surrogate_sharpes: Vec<f32> = Vec::with_capacity(30);
    for seed in 0..30_u64 {
        let sh = run_surrogate_backtest_pooled(&data_dir, &trained_action_marginals, seed)?;
        surrogate_sharpes.push(sh);
    }
    surrogate_sharpes.sort_by(|a, b| a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal));
    let p95_idx = (surrogate_sharpes.len() as f32 * 0.95) as usize;
    let p95 = surrogate_sharpes[p95_idx.min(surrogate_sharpes.len() - 1)];

    println!("[SURROGATE] N=30 random-action pooled Sharpe 95th %ile = {:.3}", p95);
    println!("[SURROGATE] distribution: min={:.3} median={:.3} max={:.3}",
             surrogate_sharpes[0],
             surrogate_sharpes[surrogate_sharpes.len()/2],
             surrogate_sharpes[surrogate_sharpes.len()-1]);

    assert!(
        trained_sharpe > p95,
        "SURROGATE-NOISE FAILURE: trained Sharpe {:.3} is not beyond the 95th percentile \
         of random-action Sharpes ({:.3}). The apparent edge may be measurement noise, \
         data leakage, or look-ahead bias. DO NOT deploy.",
        trained_sharpe, p95
    );
    Ok(())
}

fn run_trained_backtest_pooled(_data: &std::path::Path, _ckpt: &std::path::Path) -> Result<f32> {
    // Shell out to evaluate_baseline with --surrogate-mode=off, return pooled Sharpe.
    // Concrete implementation: call `evaluate_baseline` binary with a flag that
    // emits pooled-fold Sharpe to stdout, parse the line. Flag will be added to
    // evaluate_baseline as part of this task if missing.
    unimplemented!("Implement via evaluate_baseline subprocess — see Task 0.16 step 2")
}

fn run_surrogate_backtest_pooled(_data: &std::path::Path, _marginals: &[f32], _seed: u64) -> Result<f32> {
    // Similar: evaluate_baseline with --surrogate-mode=random --surrogate-seed=N.
    unimplemented!("Implement via evaluate_baseline subprocess — see Task 0.16 step 2")
}

fn load_trained_action_marginals(_data: &std::path::Path, _ckpt: &std::path::Path) -> Result<Vec<f32>> {
    // Read action_marginals.json emitted by evaluate_baseline during trained run.
    unimplemented!("Implement via evaluate_baseline subprocess — see Task 0.16 step 2")
}
```

- [ ] **Step 2: Add `--surrogate-mode` + `--surrogate-seed` to evaluate_baseline**

Modify `crates/ml/examples/evaluate_baseline.rs` Args struct:

```rust
/// Surrogate-noise mode. Default "off" = normal eval. "random" = replace
/// model actions with random actions matching --surrogate-marginals.
#[arg(long, default_value = "off")]
surrogate_mode: String,

/// Seed for --surrogate-mode=random.
#[arg(long, default_value_t = 0)]
surrogate_seed: u64,

/// Emit action marginal distribution alongside normal output.
#[arg(long, default_value_t = false)]
emit_action_marginals: bool,

/// Pooled-Sharpe output flag — print single pooled Sharpe line for machine parsing.
#[arg(long, default_value_t = false)]
emit_pooled_sharpe: bool,
```

Wire these into the action-selection path + metrics output.

- [ ] **Step 3: Replace the `unimplemented!()` stubs**

Each stub becomes a subprocess call parsing stdout.

- [ ] **Step 4: Compile + run against a known checkpoint**

If no checkpoint exists yet (first run on clean branch), this test will fail with "trained checkpoint required." That's expected — it PASSES after Phase 3 produces a checkpoint.

- [ ] **Step 5: Commit**

```bash
git commit -am "test(policy-quality): add surrogate_noise_check smoke (mandatory gate)

Requires evaluate_baseline --surrogate-mode=random + --emit-pooled-sharpe
flags, also added in this commit. Test skipped until a checkpoint exists.
Will run as part of Phase 3 validation."
git push
```

---

### Task 0.17: Baseline metrics document

**Files:**
- Create: `docs/superpowers/specs/2026-04-21-policy-quality-baseline-metrics.md`

- [ ] **Step 1: Run all new smoke tests against the Phase 0 branch head**

```bash
for t in magnitude_distribution reward_component_audit controller_activity \
         exploration_coverage multi_fold_convergence; do
  echo "=== $t ==="
  SQLX_OFFLINE=true CUBLAS_WORKSPACE_CONFIG=:4096:8 \
    FOXHUNT_TEST_DATA=test_data/futures-baseline \
    cargo test -p ml --release --lib -- $t --ignored --nocapture 2>&1 | tail -10
done
```

- [ ] **Step 2: Record output**

Write `docs/superpowers/specs/2026-04-21-policy-quality-baseline-metrics.md`:

```markdown
# Policy Quality — Phase 0 Baseline Metrics

**Captured against commit:** <run `git rev-parse HEAD`>
**Hardware:** RTX 3050 (local)
**Data:** test_data/futures-baseline, ES.FUT

## Track 1 — Magnitude

- F_Quarter / F_Half / F_Full at epoch 20: <from magnitude_distribution output>
- q_mag_full / q_mag_half / q_mag_quarter mean per-epoch: <from HEALTH_DIAG>
- var_scale_mean trajectory: <from HEALTH_DIAG>
- kelly_f_mean / avg_win_ratio trajectory: <from HEALTH_DIAG>
- grad_ratio_mag_dir trajectory: <from HEALTH_DIAG>
- trail_fire_rate per magnitude: <from HEALTH_DIAG>
- hold_time_at_exit per magnitude: <from HEALTH_DIAG>
- vsn_mask magnitude vs direction: <from HEALTH_DIAG>
- noisy σ magnitude vs direction: <from HEALTH_DIAG>
- target_drift magnitude vs direction: <from HEALTH_DIAG>
- eval-mode action distribution: <from HEALTH_DIAG>

## Track 2 — Reward Contribution

<per-term fraction from reward_component_audit output>

## Track 3 — Controller Firing

<per-controller firing rates from controller_activity output>

## Track 4 — Exploration

<magnitude/direction entropy at epoch 5, 10, 15, 20 from HEALTH_DIAG>

## Smoke test results

| Test | Result | Notes |
|---|---|---|
| magnitude_distribution | <PASS/FAIL> | |
| reward_component_audit | <PASS/FAIL> | |
| controller_activity | <PASS/FAIL> | |
| exploration_coverage | <PASS/FAIL> | |
| multi_fold_convergence | <PASS/FAIL> | |
```

Fill in every `<...>` with actual captured values. No placeholders in the committed document.

- [ ] **Step 3: Commit**

```bash
git add docs/superpowers/specs/2026-04-21-policy-quality-baseline-metrics.md
git commit -m "docs(policy-quality): Phase 0 baseline metrics (pre-audit reference)"
git push
```

- [ ] **Step 4: Tag the final Phase-0 commit as `policy-quality-baseline`**

```bash
git tag -a policy-quality-baseline -m "Phase 0 complete — instrumentation in, pre-fix reference"
git push origin policy-quality-baseline
```

This tag is the rollback anchor for Phases 2+. If something breaks on main, `git reset --hard policy-quality-baseline` restores this known-good state.

---

## Phase 1 — Investigation Tracks

**Goal:** run each track's HEALTH_DIAG-instrumented training, collect evidence, classify each hypothesis/term/controller/mechanism as CONFIRMED or REJECTED.

**Exit criteria:** 4 triage documents committed to main. Phase-2 implementation plan written from consolidated findings.

**Discipline:** some hypotheses need temporary code edits (e.g., force epsilon=1.0 for H1). Edit the file, run the experiment, then `git checkout -- <file>` to revert. Don't commit experimental code. Only triage docs commit.

---

### Task 1.1: Track 1 — Magnitude investigation

- [ ] **Step 1: Baseline run — inherits Phase 0 instrumentation**

Already have HEALTH_DIAG from Phase 0. Run 20-epoch × 2-fold on L40S:

```bash
./scripts/argo-train.sh dqn --gpu-pool ci-training-l40s --epochs 20 --baseline
```

Wait for completion, dump logs:
```bash
argo logs -n foxhunt <workflow-name> > /tmp/track1-baseline.log
```

- [ ] **Step 2: Forced-exploration experiment (for H1 detection)**

H1 requires epsilon=1.0 for 1 epoch at mid-training.

1. Edit `crates/ml/src/trainers/dqn/trainer/training_loop.rs` — force `epsilon = 1.0` when `epoch == 10`.
2. Run the workflow:
   ```bash
   ./scripts/argo-train.sh dqn --gpu-pool ci-training-l40s --epochs 20 --baseline
   argo logs -n foxhunt <workflow> > /tmp/track1-forced-explore.log
   ```
3. Revert the edit: `git checkout -- crates/ml/src/trainers/dqn/trainer/training_loop.rs`

- [ ] **Step 3: Classify each hypothesis**

For each of H1–H10, check the detection signal against the threshold per spec §5.1. Record CONFIRMED or REJECTED with evidence in `docs/superpowers/specs/2026-04-21-policy-quality-track1-triage.md`:

```markdown
# Track 1 — Magnitude Collapse Triage

**Log refs:** /tmp/track1-baseline.log, /tmp/track1-forced-explore.log

## H1 — Reward gradient asymmetry
- Signal: mean(R_full)/std(R_full) = X vs mean(R_quarter)/std(R_quarter) = Y
- Threshold: Full/Quarter ratio ≤ 0.5
- Observed: <ratio>
- **Verdict:** CONFIRMED / REJECTED
- If CONFIRMED, proposed fix: <spec §5.1 fix sketch for H1>

(...repeat for H2–H10)
```

- [ ] **Step 4: Commit triage doc to main**

```bash
git add docs/superpowers/specs/2026-04-21-policy-quality-track1-triage.md
git commit -m "triage(track1): magnitude hypotheses H1-H10 verdicts with evidence"
git push
```

---

### Task 1.2: Track 2 — Reward V7 audit

- [ ] **Step 1: Baseline run**

```bash
./scripts/argo-train.sh dqn --gpu-pool ci-training-l40s --epochs 20 --baseline
argo logs -n foxhunt <workflow> > /tmp/track2-baseline.log
```

Phase 0 reward-contrib HEALTH_DIAG fields already capture per-term signals.

- [ ] **Step 2: For each of R1–R8, apply the V7 audit template**

Per spec §5.2:
1. Identify signal (git log, grep for the term, read its introducing commit)
2. Check better-form (does another term cover the same invariant?)
3. Measure (`reward_contrib_*` field from `/tmp/track2-baseline.log`)
4. Decide per the decision matrix

- [ ] **Step 3: Record triage + commit**

Write `docs/superpowers/specs/2026-04-21-policy-quality-track2-triage.md` (one section per R1–R8) and commit to main:

```bash
git add docs/superpowers/specs/2026-04-21-policy-quality-track2-triage.md
git commit -m "triage(track2): reward V7 audit R1-R8 verdicts"
git push
```

---

### Task 1.3: Track 3 — Controllers V7 audit

- [ ] **Step 1: Use the Phase 0 HEALTH_DIAG controller-fire-rate data (already captured)**

Controller fire rates appear in every HEALTH_DIAG line. Extract from `/tmp/track2-baseline.log` or run another 20-epoch if needed.

- [ ] **Step 2: Disable-and-measure experiments**

For candidates flagged "CANDIDATE FOR DELETE" per spec §5.3 (10–50% fire rate), disable each temporarily and compare outcomes:

1. Edit `training_loop.rs` — change the controller's if-block to `if false`.
2. `./scripts/argo-train.sh dqn --gpu-pool ci-training-l40s --epochs 20 --baseline`
3. Record Best Sharpe + grad stability vs baseline in the triage doc.
4. Revert: `git checkout -- crates/ml/src/trainers/dqn/trainer/training_loop.rs`
5. Repeat for next candidate.

- [ ] **Step 3: Triage document + commit**

Write `docs/superpowers/specs/2026-04-21-policy-quality-track3-triage.md`, one section per C1–C7. Commit to main.

---

### Task 1.4: Track 4 — Exploration V7 audit

- [ ] **Step 1: Use Phase 0 HEALTH_DIAG data**

`explore_entropy_*` and `noisy_sigma_mean` are already captured. No additional instrumentation needed for E1/E2/E4.

- [ ] **Step 2: For each E1–E5, apply V7 template**

Count-based bonus (E3) and entropy regularization (E4) may be redundant with NoisyNets (E2). For each mechanism:
1. Edit its enable-block to `if false` in training_loop.rs
2. Run 20-epoch
3. Record impact on magnitude entropy
4. `git checkout -- <file>`, move to next

- [ ] **Step 3: Triage document + commit**

Write `docs/superpowers/specs/2026-04-21-policy-quality-track4-triage.md`. Commit to main.

---

### Task 1.5: Write Phase 2 implementation plan

- [ ] **Step 1: Consolidate findings**

All 4 triage docs are on main. Skim each, compile a list of CONFIRMED findings.

- [ ] **Step 2: Invoke writing-plans against the consolidated triage**

```bash
# Use superpowers:writing-plans with input:
#   "Given the 4 triage docs at
#    docs/superpowers/specs/2026-04-21-policy-quality-track{1,2,3,4}-triage.md
#    produce a Phase 2 implementation plan with concrete TDD tasks per CONFIRMED
#    finding. Target: docs/superpowers/plans/2026-04-21-policy-quality-phase2.md"
```

- [ ] **Step 3: Commit the Phase 2 plan**

```bash
git add docs/superpowers/plans/2026-04-21-policy-quality-phase2.md
git commit -m "plan(policy-quality): Phase 2 implementation plan from triage findings"
git push
```

---

## Phase 2 — Convergence Fix

**Goal:** implement all fixes identified in Phase 1 triage on main. Each commit leaves main working + smokes green.

**Exit criteria:** all 6 Phase 0 smokes pass at their PRODUCTION thresholds (tighten from Phase 0's relaxed smoke thresholds). Existing smokes (`td_propagation`, `test_50_epoch_convergence`) still pass.

**Template for Phase 2 tasks (filled in by the Phase 2 plan written in Task 1.5):**

For each CONFIRMED finding in any of the 4 triages:
1. Write test asserting the fixed behaviour (e.g., magnitude bin usage at production threshold)
2. Implement the fix per the spec's fix sketch
3. Run all 6 Phase 0 smokes locally — expect all pass
4. Commit with message referencing the triage finding (e.g., `fix(dqn): H2 — clamp var_scale ≥ 0.5 per track1 triage`)
5. Push

Commits can be one-per-finding (preferred — easier to review) or grouped by track. Author's choice.

---

## Phase 3 — L40S Validation

**Goal:** confirm Phase 2 fix passes the full 6-fold × 50-epoch production gate.

### Task 3.1: Run validation workflow

- [ ] **Step 1: Ensure main HEAD has Phase 2 fixes committed and pushed**

Run: `git status && git log -1 --oneline`
Expected: clean, last commit is the Phase 2 convergence.

- [ ] **Step 2: Launch validation**

Run:
```bash
./scripts/argo-train.sh dqn --gpu-pool ci-training-l40s --epochs 50 --baseline
```
Note the workflow name (e.g. `train-xxxxx`).

- [ ] **Step 3: Monitor until completion**

Run: `argo watch -n foxhunt <workflow>`
Expected duration: ~1 hour.

### Task 3.2: Gate-by-gate evaluation

- [ ] **Step 1: Extract Best Sharpe per fold**

```bash
argo logs -n foxhunt <workflow> | grep "Best Sharpe:" | tee /tmp/phase3-sharpe.txt
```

- [ ] **Step 2: Extract per-fold WinRate**

```bash
argo logs -n foxhunt <workflow> | grep -E "Validation backtest:" | tee /tmp/phase3-winrate.txt
```

- [ ] **Step 3: Extract per-fold action distribution**

```bash
argo logs -n foxhunt <workflow> | grep "action_dist\|LOW EXPOSURE" | tee /tmp/phase3-actions.txt
```

- [ ] **Step 4: Check MANDATORY gates (spec §2.1)**

Manually check:
- B-1: Best val Sharpe > 10 on ≥5/6 folds — from `/tmp/phase3-sharpe.txt`
- B-2: Val WinRate > 55% on ≥5/6 folds — from `/tmp/phase3-winrate.txt`
- SN: surrogate-noise smoke test — requires running locally after workflow completes (see Task 3.3)

- [ ] **Step 5: Check SOFT gates (spec §2.2)**

- A-1: F_Half ≥ 15% AND F_Full ≥ 15% on ≥4/6 folds
- A-2: Val WinRate ≥ 55% across folds
- A-3: ≥100 trades per val window on ≥5/6 folds
- CA: Controller firing ≤ 50%

Record gate results in `docs/superpowers/specs/2026-04-21-policy-quality-phase3-results.md`.

### Task 3.3: Surrogate-noise check against produced checkpoint

- [ ] **Step 1: Fetch Phase 3 checkpoint**

```bash
gh api repos/root/foxhunt/packages/generic/foxhunt-training-results/...  # or gitlab equivalent
# Download dqn_fold0_best.safetensors etc. to /workspace/output/ local mirror
```

- [ ] **Step 2: Run surrogate-noise smoke**

```bash
SQLX_OFFLINE=true CUBLAS_WORKSPACE_CONFIG=:4096:8 \
  FOXHUNT_TEST_DATA=test_data/futures-baseline \
  cargo test -p ml --release --lib -- surrogate_noise_check --ignored --nocapture 2>&1 | tail -30
```

Expected: pass — trained pooled Sharpe exceeds 95th %ile of N=30 surrogates.

- [ ] **Step 3: Record result in phase3-results.md**

### Task 3.4: Branch decision

- [ ] **Step 1: Aggregate mandatory gate status**

| Gate | Pass/Fail |
|---|---|
| B-1 Sharpe | |
| B-2 WinRate | |
| SN Surrogate | |

- [ ] **Step 2a: If all mandatory pass → proceed to Phase 4**

Commit `phase3-results.md` and proceed to merge.

- [ ] **Step 2b: If any mandatory fails → iterate on main**

Up to 2 additional iterations total (max 3 Phase-3 validation runs). Each iteration:
- Diagnose from log which gate failed and why
- Amend Phase 2 commit with additional fix
- Re-run Phase 3
- If still failing after 3 iterations total, open triage spec (spec §2.3):
  - `docs/superpowers/specs/YYYY-MM-DD-policy-quality-triage.md`
  - Do NOT merge to main

---

## Phase 4 — Close

**Goal:** tag `policy-quality-v1` on the final validated main commit, update the spec status, close.

**Prerequisites:** Phase 3 all MANDATORY gates pass.

### Task 4.1: Tag the validated commit

- [ ] **Step 1: Confirm on main at the post-Phase-3 HEAD**

```bash
git checkout main
git log -1 --oneline
```

- [ ] **Step 2: Tag**

```bash
git tag -a policy-quality-v1 -m "Policy quality V7 audit complete — sub-project A closed.

All MANDATORY gates passed in Phase 3 validation:
  - B-Sharpe: Best val Sharpe > 10 on <N>/6 folds
  - B-WinRate: Val WinRate > 55% on <N>/6 folds
  - Surrogate-noise: trained pooled Sharpe beyond 95th %ile of 30 random surrogates

Phase 3 results: docs/superpowers/specs/2026-04-21-policy-quality-phase3-results.md"
git push origin policy-quality-v1
```

### Task 4.2: Update spec status + roadmap

- [ ] **Step 1: Add CLOSED header to the spec**

At top of `docs/superpowers/specs/2026-04-21-policy-quality-design.md`, insert:

```markdown
**Status (2026-MM-DD): CLOSED — tagged `policy-quality-v1`.**
**Phase 3 results:** `docs/superpowers/specs/2026-04-21-policy-quality-phase3-results.md`
```

- [ ] **Step 2: Commit + push**

```bash
git add docs/superpowers/specs/2026-04-21-policy-quality-design.md
git commit -m "docs(policy-quality): mark sub-project A closed, tagged policy-quality-v1"
git push
```

Sub-projects B, C, D (validation rigor, paper trading, production ops) are now unblocked.

---

## Rollback Procedure (emergency)

If a commit on main breaks something that wasn't caught by smokes:

```bash
git reset --hard policy-quality-baseline
git push -f origin main  # only with team coordination
```

The baseline tag = last Phase 0 commit = pre-fix reference state. Recovery is one command.

---

## References

- Spec: `docs/superpowers/specs/2026-04-21-policy-quality-design.md`
- Baseline metrics: `docs/superpowers/specs/2026-04-21-policy-quality-baseline-metrics.md` (produced in Task 0.17)
- Track triage docs: `docs/superpowers/specs/2026-04-21-policy-quality-track{1..4}-triage.md` (produced in Phase 1)
- Phase 3 results: `docs/superpowers/specs/2026-04-21-policy-quality-phase3-results.md` (produced in Phase 3)
- Phase 2 detailed plan: `docs/superpowers/plans/2026-04-21-policy-quality-phase2.md` (produced in Task 1.5)
