jgrusewski f4ea0f6d13 test(policy-quality): Task 0.16 surrogate_noise_check smoke (mandatory gate) ...

Adds evaluate_baseline CLI args:
  --surrogate-mode=off|random
  --surrogate-seed=<u64>
  --surrogate-marginals=<path.json>
  --emit-action-marginals
  --emit-pooled-sharpe

Random-action surrogate samples actions from marginal distribution (or
uniform fallback) using a seeded RNG, bypassing the model entirely.
Surrogate mode forces the CPU DQN path so action selection can be
overridden (GPU path would need invasive kernel changes and defeats
the bypass-the-model sanity check).

Flat action-index counts are tracked in the CPU DQN path only; the
ACTION_MARGINALS: line emits those as a JSON distribution, or emits
an honest stub marker when only the GPU path was exercised. Pooled
Sharpe is computed from concatenated per-fold returns (CPU path) or
falls back to mean-of-fold-Sharpes with a warning.

Smoke test runs 30 surrogate seeds + 1 trained run via evaluate_baseline
subprocess, asserts trained pooled Sharpe exceeds the 95th percentile of
surrogate Sharpes. Test is #[ignore]'d and requires a trained checkpoint
at /workspace/output/dqn_fold0_best.safetensors (Phase 3 deliverable);
FOXHUNT_SURROGATE_CKPT env var overrides for local testing. Uses the
compiled target/release/examples/evaluate_baseline if present, otherwise
falls back to cargo run.

Will pass once Phase 3 produces a checkpoint.

2026-04-21 23:22:27 +02:00

113 KiB

Raw Blame History

View Raw