diff --git a/docs/superpowers/specs/2026-04-21-policy-quality-baseline-metrics.md b/docs/superpowers/specs/2026-04-21-policy-quality-baseline-metrics.md new file mode 100644 index 000000000..1828dcb4b --- /dev/null +++ b/docs/superpowers/specs/2026-04-21-policy-quality-baseline-metrics.md @@ -0,0 +1,123 @@ +# Policy Quality — Phase 0 Baseline Metrics + +**Captured against commit:** `f4ea0f6d1` (HEAD of `main` after Phase 0 completion; update before tagging `policy-quality-baseline`) +**Hardware:** RTX 3050 Ti Laptop GPU (local) — for production-baseline numbers re-run on L40S/H100 per Phase 3 +**Data:** `test_data/futures-baseline`, ES.FUT +**Smoke-test command template:** +```bash +SQLX_OFFLINE=true CUBLAS_WORKSPACE_CONFIG=:4096:8 \ + FOXHUNT_TEST_DATA=test_data/futures-baseline \ + cargo test -p ml --release --lib -- --ignored --nocapture +``` + +> **Status: SCAFFOLD ONLY — GPU runs pending.** +> Every `` marker below must be replaced with the actual captured value before the `policy-quality-baseline` tag is applied. The scaffold encodes the agreed metric set so reviewers can verify nothing was forgotten when the run completes. + +--- + +## Track 1 — Magnitude + +Drawn from `magnitude_distribution.rs` smoke test stdout + `[HEALTH_DIAG]` lines emitted by the trainer. + +| Metric | Source | Value | +|---|---|---| +| `F_Quarter` (action share) at epoch 20 | `magnitude_distribution` test output | `` | +| `F_Half` at epoch 20 | same | `` | +| `F_Full` at epoch 20 | same | `` | +| `q_mag_quarter` (mean per-epoch) | HEALTH_DIAG `mag [q_quarter=…]` | `` | +| `q_mag_half` | HEALTH_DIAG `mag [q_half=…]` | `` | +| `q_mag_full` | HEALTH_DIAG `mag [q_full=…]` | `` | +| `var_scale_mean` (trajectory, e5/e10/e15/e20) | HEALTH_DIAG `mag [var_scale=…]` | `` | +| `kelly_f_mean` (trajectory) | HEALTH_DIAG `mag [kelly_f=…]` | `` | +| `avg_win_ratio` (trajectory) | HEALTH_DIAG `mag [avg_win_ratio=…]` | `` | +| `grad_ratio_mag_dir` (trajectory) | HEALTH_DIAG `mag [grad_ratio_mag_dir=…]` | `` | +| `dist_q` / `dist_h` / `dist_f` (Q-distance per mag) | HEALTH_DIAG `mag [dist_q=… dist_h=… dist_f=…]` | `` | +| `trail_fire_rate[Q,H,F]` per mag | HEALTH_DIAG `trail [fire_q=… fire_h=… fire_f=…]` | `` | +| `hold_at_exit_mean[Q,H,F]` per mag | HEALTH_DIAG `trail [hold_q=… hold_h=… hold_f=…]` | `` | +| `vsn_mag` / `vsn_dir` | HEALTH_DIAG `noisy [vsn_mag=… vsn_dir=…]` | `` | +| `sigma_mag` / `sigma_dir` (NoisyNets σ) | HEALTH_DIAG `noisy [sigma_mag=… sigma_dir=…]` | `` | +| `drift_mag` / `drift_dir` (target weight drift) | HEALTH_DIAG `noisy [drift_mag=… drift_dir=…]` | `` | +| Eval-mode action distribution `eq/eh/ef` | HEALTH_DIAG `eval_dist [eq=… eh=… ef=…]` | `` | + +## Track 2 — Reward Contribution + +Drawn from `reward_component_audit.rs` smoke output + HEALTH_DIAG `reward_contrib`. + +| Slot | Term | Class | Baseline value | +|---|---|---|---| +| 0 | popart | transform (variance Δ ratio) | `` | +| 1 | cf | sample-selector (CF flip rate) | `` | +| 2 | trail_r | sample-selector (trail rate) | `` | +| 3 | micro | additive (\|Σ micro\| / \|Σ total\|) | `` | +| 4 | la | additive (\|Σ asym-clamp Δ\| / \|Σ total\|) | `` | +| 5 | seg | additive (\|Σ segment-patience\| / \|Σ total\|) | `` | + +## Track 3 — Controller Firing + +Drawn from `controller_activity.rs` smoke output + HEALTH_DIAG `controller`. + +| Controller | Firing rate (epoch 20) | +|---|---| +| anti_lr | `` | +| tau | `` | +| gamma | `` | +| clip | `` | +| cql | `` | +| cost | `` | +| `fire_frac` aggregate | `` | + +## Track 4 — Exploration + +Drawn from `exploration_coverage.rs` smoke output + HEALTH_DIAG `explore`. + +| Metric | epoch 5 | epoch 10 | epoch 15 | epoch 20 | +|---|---|---|---|---| +| `ent_mag` (magnitude entropy) | `

` | `

` | `

` | `

` | +| `ent_dir` (direction entropy) | `

` | `

` | `

` | `

` | +| `sigma_mean` (overall NoisyNets σ) | `

` | `

` | `

` | `

` | + +(`

` = ``) + +--- + +## Smoke test results + +| Test | Result | Wall-clock (RTX 3050 Ti) | Notes | +|---|---|---|---| +| `magnitude_distribution` | `` | `` | | +| `reward_component_audit` | `` | `` | Validates infra; real triage from log-grep on multi-fold L40S run (Phase 1) | +| `controller_activity` | `` | `` | | +| `exploration_coverage` | `` | `` | | +| `multi_fold_convergence` | `` | `` | Multi-fold; longest test | + +--- + +## Phase-0 task completion summary + +| Task | Status | Commit | +|---|---|---| +| 0.1 Verify starting state | ✅ Complete | (pre-session) | +| 0.2 HEALTH_DIAG scaffolding | ✅ Complete | (pre-session) | +| 0.3 Track 1 magnitude Q + var_scale + kelly | ⚠️ Partial — q_mag/var_scale/kelly_f/avg_win_ratio still stubbed (require GPU readback infra) | (pre-session) | +| 0.4 Per-branch grad norm ratio | ✅ Complete | `bb42c9963` | +| 0.5 Per-magnitude trail fire rate | ✅ Complete (no atomicAdd, per-sample arrays) | `48d962ee2` | +| 0.6 VSN + NoisyNets σ + target drift | ⚠️ Partial — σ wired (`999bb2fa0`); VSN mask + target drift TODO | `999bb2fa0` | +| 0.7 Eval-mode action distribution | ✅ Complete | (pre-session) | +| 0.8 Reward term contribution | ✅ Complete (seg slot honest stub: term not wired) | `bf8415c5d` | +| 0.9 Controller fire-rate tracking | ✅ Complete | (pre-session) | +| 0.10 Exploration entropy + sigma | ✅ Complete | `999bb2fa0` | +| 0.11 `magnitude_distribution` smoke | ✅ Complete | (pre-session) | +| 0.12 `reward_component_audit` smoke | ✅ Complete | `93d8c5ae4` | +| 0.13 `controller_activity` smoke | ✅ Complete | (pre-session) | +| 0.14 `exploration_coverage` smoke | ✅ Complete | (pre-session) | +| 0.15 `multi_fold_convergence` smoke | ✅ Complete | (pre-session) | +| 0.16 `surrogate_noise_check` smoke + CLI extensions | ✅ Complete (no checkpoint required to compile) | `f4ea0f6d1` | +| 0.17 Baseline metrics document | 🟡 Scaffold committed; values pending GPU run | this commit | + +**Remaining work for the `policy-quality-baseline` tag**: +1. Run all 5 smoke tests on this commit (`f4ea0f6d1` or HEAD). +2. Replace every `` marker above with captured values. +3. (Optional, for completeness) Wire the remaining Task 0.3 / 0.6 accessors so the corresponding rows aren't structural stubs. +4. Re-amend this document and `git tag -a policy-quality-baseline -m "Phase 0 complete — instrumentation in, pre-fix reference"`. + +The scaffold is intentionally exhaustive so reviewers can verify nothing was dropped when the GPU run is captured.