fix(dqn): Bug #5 — reset controller_fire_counts at fold boundary (Task 2.5)

Track 3 triage §C2/C5 fold-boundary artefact: the controller fire
counters (anti_lr, tau, gamma, grad_clip, cql_alpha, cost_anneal),
the running total-epochs denominator, and the prev-controller snapshot
were NOT reset in reset_for_fold(), causing the 2/60 fire rates for
tau and cql_alpha to accumulate across folds under cosine-annealed tau
jumps when train_step resets + cql_alpha schedule drift.

Fix: reset `controller_fire_counts`, `controller_total_epochs`, and
`prev_controller_values` at fold-entry in reset_for_fold(). This
decouples fold-boundary bookkeeping from intra-fold controller
interventions so the controller_activity smoke gate measures per-fold
fire rate rather than multi-fold running count.

`last_anti_mult` is intentionally NOT reset here — it is within-epoch
state already reset in reset_epoch_state.
This commit is contained in:
jgrusewski
2026-04-22 16:06:49 +02:00
parent 96ecd0ff46
commit 0cac3c84ce

View File

@@ -1466,6 +1466,18 @@ impl DQNTrainer {
.map_err(|e| anyhow::anyhow!("guard reset: {e}"))?;
}
// Task 2.5 Bug #5 — reset controller fire counters + total-epochs +
// prev-controller snapshot at fold boundary. Without this, Track 3
// C2/C5 fold-boundary artefacts (cosine-annealed tau jumps when
// train_step resets, cql_alpha schedule drift across folds) would
// accumulate into a multi-fold running count, misrepresenting the
// per-fold fire rate for the controller_activity smoke gate.
// `last_anti_mult` is within-epoch state (reset in reset_epoch_state)
// and not reset here.
self.controller_fire_counts = ControllerFireCounts::default();
self.controller_total_epochs = 0;
self.prev_controller_values = ControllerPrevValues::default();
// Keep gpu_data — the full dataset is already on GPU via init_from_fxcache.
// Clearing it would force a redundant re-upload on the next fold.