feat(dqn): add train_active_frac HEALTH_DIAG metric

Counterpart to existing val_active_frac. Reports the fraction of
TRAINING rollout actions that were Long or Short. Required for L3
verification gate of Plan D (Phase 3) which checks Thompson is
generating directional exploration during training.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-04-29 17:45:49 +02:00
parent f691d754eb
commit ae7aed56a6
5 changed files with 97 additions and 0 deletions

View File

@@ -581,6 +581,10 @@ impl DQNTrainer {
training_sharpe_ema_initialized: false,
last_action_entropy: None,
last_magnitude_dist: [0.0_f32; 3],
// Plan C Task 5 — populated each epoch from monitor.action_counts in
// training_loop.rs; consumed by HEALTH_DIAG `train_active_frac` line
// emitted alongside val_active_frac in `consume_validation_loss`.
last_train_active_frac: 0.0_f32,
explore_entropy_mag_history: Vec::new(),
prev_controller_values: super::ControllerPrevValues::default(),
controller_fire_counts: super::ControllerFireCounts::default(),

View File

@@ -857,6 +857,23 @@ impl DQNTrainer {
window_bars,
);
// Plan C Task 5 — training-rollout counterpart to val_active_frac.
// Source: `monitor.action_counts` aggregated over the previous
// epoch's training rollout (12-bin layout: dir*3 + mag); cached on
// the trainer at the end of `training_loop.rs`'s per-epoch metrics
// block (`self.last_train_active_frac`). `val_active_frac` above
// measures direction commitment during the EVAL backtest; this
// measures it during TRAINING rollout — divergence localises
// whether Thompson is generating directional exploration during
// experience collection (Plan D L3 verification gate, target
// > 0.40 = at least 40% of training-rollout actions are Long or
// Short, not Hold/Flat). Emitted next to val_active_frac so the
// two lines are adjacent in HEALTH_DIAG output and easy to compare.
tracing::info!(
"HEALTH_DIAG[{}]: train_active_frac={:.4} (Long+Short during training rollout)",
epoch, self.last_train_active_frac,
);
m.sharpe as f64
} else {
0.0

View File

@@ -331,6 +331,20 @@ pub struct DQNTrainer {
/// summed across all direction bins, normalized to [0, 1].
pub(crate) last_magnitude_dist: [f32; 3],
/// Plan C Task 5 — fraction of training-rollout actions where direction ∈
/// {Short, Long} (i.e. the policy committed to a directional bet rather
/// than Hold/Flat). Counterpart to the kernel-derived val_active_frac
/// emitted alongside the val [...] block in `consume_validation_loss`.
/// Required for Plan D's L3 verification gate (target > 0.40 = at least
/// 40% of training-rollout actions are Long or Short, evidence Thompson
/// is generating directional exploration during training).
///
/// Computed each epoch in the training_loop from `monitor.action_counts`
/// (12-bin layout: dir*3 + mag, dir ∈ {Short=0, Hold=1, Long=2, Flat=3});
/// active = bins[0..3] (Short) + bins[6..9] (Long), denominator = all 12.
/// `0.0` until the first epoch finishes its training rollout.
pub(crate) last_train_active_frac: f32,
/// Task 0.10 — per-epoch magnitude-branch action entropy history. Each
/// entry is (epoch_0_indexed, normalized_entropy). Used by the
/// exploration_coverage smoke test to assert the magnitude branch

View File

@@ -2870,6 +2870,20 @@ impl DQNTrainer {
// Task 0.14: append to exploration entropy history for smoke-test readback.
self.explore_entropy_mag_history.push((epoch as u32, ent_mag));
// Plan C Task 5 — training-rollout active fraction (counterpart to
// the kernel-derived val_active_frac emitted from
// `consume_validation_loss`). "Active" = direction ∈ {Short, Long}.
// 12-bin layout (dir*3 + mag): Short=[0..3], Hold=[3..6], Long=[6..9],
// Flat=[9..12]; denominator = all 12 bins (every training step
// contributes one tally). Cached on the trainer so the next epoch's
// `consume_validation_loss` emits it next to val_active_frac.
// Required for Plan D's L3 verification gate (target > 0.40).
self.last_train_active_frac = if action_total_epoch > 0 {
(dir0 + dir2).clamp(0.0, 1.0)
} else {
0.0_f32
};
// Task 0.9: adaptive-controller fire detection.
//
// A controller "fires" iff it made an ADAPTIVE INTERVENTION this

View File

@@ -1981,3 +1981,51 @@ clean (warnings unchanged from baseline). `cargo test -p ml --lib
--no-run` clean. No fingerprint change — buffer layouts and ISV
slots unchanged from the kernel's perspective.
## Plan C Task 5 (2026-04-29) — `train_active_frac` HEALTH_DIAG metric
Added the training-rollout counterpart to the existing
`val_active_frac` metric so Plan D's L3 verification gate has a
direct signal that Thompson is generating directional exploration
during training (target: `train_active_frac > 0.40`, i.e. at least
40% of training-rollout actions land on Short or Long rather than
Hold/Flat). `val_active_frac` is sourced from the eval-backtest
kernel's per-direction `buy_count`/`sell_count`/`hold_count`
reduction; `train_active_frac` is sourced from the in-memory
`monitor.action_counts` aggregator (12-bin layout, `dir*3 + mag`,
direction ∈ {Short=0, Hold=1, Long=2, Flat=3}). Active fraction =
`(Σ bins[0..3] + Σ bins[6..9]) / Σ bins[0..12]`.
Wire-up:
- `crates/ml/src/trainers/dqn/trainer/mod.rs` — added
`last_train_active_frac: f32` field on `DQNTrainer` (cached each
epoch from the in-loop monitor; consumed by the next epoch's
`consume_validation_loss` so the two `*_active_frac` lines are
emitted adjacent in HEALTH_DIAG output).
- `crates/ml/src/trainers/dqn/trainer/constructor.rs` —
`last_train_active_frac: 0.0_f32` initialisation (zero until the
first epoch finishes its training rollout).
- `crates/ml/src/trainers/dqn/trainer/training_loop.rs` —
populates `self.last_train_active_frac` in the per-epoch metrics
block immediately after the existing `dist_q/dist_h/dist_f` +
`ent_dir`/`ent_mag` calculations, reusing the already-computed
`dir0` (Short) and `dir2` (Long) per-direction shares (full-12-bin
denominator, identical to `monitor.action_counts.iter().sum()`),
with the same `action_total_epoch > 0` guard. Clamped to
`[0.0, 1.0]` defensively.
- `crates/ml/src/trainers/dqn/trainer/metrics.rs` — emits
`HEALTH_DIAG[{}]: train_active_frac=X.XXXX (Long+Short during
training rollout)` from `consume_validation_loss` immediately
after the existing `val [...]` block (which contains
`val_active_frac`). The two metrics now appear next to each
other in every epoch's HEALTH_DIAG output: `val_active_frac`
measures direction commitment during the eval backtest while
`train_active_frac` measures it during training rollout —
divergence localises whether Thompson is stochastic enough at
experience-collection time even when eval (deterministic argmax)
collapses to Hold/Flat.
No fingerprint change — buffer layouts, ISV slots, and kernel
signatures are untouched. Verification:
`SQLX_OFFLINE=true cargo check -p ml --lib` clean.