test(policy-quality): Task 0.12 — reward_component_audit smoke

Adds smoke test asserting the reward-contribution diagnostic produces
finite, non-negative values for all 6 reward terms (popart, cf_flip,
trail, micro, loss_aversion, segment_patience).

The smoke test validates the DIAGNOSTIC INFRASTRUCTURE — that the
per-term contribution accessor doesn't return NaN/Inf/negative values
that would break log parsing in Phase 1 audit. The actual triage
(KEEP/DELETE per term per spec §5.2) happens against a multi-fold
L40S training log in Phase 1, not in this smoke.

DQNTrainer.reward_component_audit_summary() returns [f32; 6] with
explicit measurement-class semantics per spec §4.1:
  - additive: fraction of |total reward|
  - transform (popart): |delta| / |pre|
  - sample-selector (cf_flip, trail): firing rate

Currently returns zeros — Task 0.8 wires kernel-side instrumentation
to populate real values. Smoke remains green throughout (zeros pass
finite + non-negative gates), and ensures any future kernel-side
breakage that produces NaN/Inf is caught immediately.
This commit is contained in:
jgrusewski
2026-04-21 22:01:38 +02:00
parent 999bb2fa0c
commit 93d8c5ae4a
3 changed files with 77 additions and 0 deletions

View File

@@ -32,3 +32,5 @@ mod exploration_coverage;
mod controller_activity;
#[cfg(test)]
mod multi_fold_convergence;
#[cfg(test)]
mod reward_component_audit;

View File

@@ -0,0 +1,62 @@
//! Smoke test: reward component audit (Track 2 from policy-quality spec).
//!
//! Asserts the per-term reward contribution diagnostic in HEALTH_DIAG produces
//! finite, machine-readable values across an epoch's worth of training. The
//! actual triage (which terms get DELETE / KEEP per spec §5.2 V7 audit) is
//! Phase 1 work performed against logs from a multi-fold L40S run — this
//! smoke gates that the diagnostic infrastructure runs cleanly.
//!
//! The audit covers 8 reward terms split into 3 measurement classes per
//! spec §4.1:
//! - Additive (R1 step_return, R5 micro-reward, R6 loss-aversion, R7 segment-patience):
//! fraction of |total reward|
//! - Transform (R2 PopArt): |delta| / |pre| ratio
//! - Sample-selector (R3 CF-flip, R4 trailing-stop): firing rate
//!
//! Run: `FOXHUNT_TEST_DATA=test_data/futures-baseline \
//! cargo test -p ml --release --lib -- reward_component_audit --ignored --nocapture`
use super::helpers::*;
use anyhow::Result;
#[test]
#[ignore] // Requires fxcache
fn test_reward_components_contribute() -> Result<()> {
let data = load_smoke_fxcache().expect("fxcache required — run precompute_features first");
let mut params = smoke_params();
params.epochs = 20;
params.early_stopping_enabled = false;
params.min_epochs_before_stopping = 20;
let mut trainer = smoke_trainer_with(params)?;
init_trainer_from_fxcache(&mut trainer, &data, 200_000)?;
let data_dir = test_data_dir().expect("FOXHUNT_TEST_DATA or test_data/ must exist");
let rt = tokio::runtime::Builder::new_current_thread().enable_all().build()?;
let _metrics = rt.block_on(trainer.train(
&data_dir,
"ES.FUT",
|_epoch, _bytes, _best| Ok("skip".to_owned()),
))?;
// Reward-contribution diagnostic comes from HEALTH_DIAG `reward_contrib` group.
// Until kernel-side instrumentation (Task 0.8) populates real values, the
// current readback is all-zero stubs — which IS finite, satisfies the smoke
// gate, and confirms the infra is in place.
//
// Real triage uses log-grep against a 6-fold × 50-epoch L40S run (Phase 1).
// The smoke's job is to ensure the reporting scaffolding never regresses
// with NaN/Inf values, which would break log parsing downstream.
let summary = trainer.reward_component_audit_summary();
println!(
"[REWARD_AUDIT] popart={:.4} cf_flip={:.4} trail={:.4} micro={:.4} \
loss_aversion={:.4} segment_patience={:.4}",
summary[0], summary[1], summary[2], summary[3], summary[4], summary[5]
);
for (i, &v) in summary.iter().enumerate() {
let names = ["popart", "cf_flip", "trail", "micro", "loss_aversion", "segment_patience"];
assert!(v.is_finite(), "reward_contrib[{}] (={}) is non-finite — diagnostic infra broken", names[i], v);
assert!(v >= 0.0, "reward_contrib[{}] (={}) is negative — magnitudes should be unsigned", names[i], v);
}
Ok(())
}

View File

@@ -1479,6 +1479,19 @@ impl DQNTrainer {
&self.explore_entropy_mag_history
}
/// Per-term reward contribution summary (Track 2 audit).
/// Layout: [popart, cf_flip, trail, micro, loss_aversion, segment_patience].
/// Each entry's interpretation depends on the term's measurement class
/// per spec §4.1 (additive vs transform vs sample-selector). Used by
/// reward_component_audit smoke test.
pub fn reward_component_audit_summary(&self) -> [f32; 6] {
// Stub returns zeros until Task 0.8 wires kernel-side per-term
// contribution accumulation into HEALTH_DIAG fields. The smoke test
// uses this to guard against the diagnostic ever reporting non-finite
// values (which would break log parsing).
[0.0_f32; 6]
}
/// Per-controller firing rates across all completed epochs.
/// Layout: [anti_lr, tau, gamma, grad_clip, cql_alpha, cost_anneal].
/// Each rate = epochs-where-value-changed / total-epochs. Used by the