fix(dqn): Bug #5 — reset controller_fire_counts at fold boundary (Task 2.5)
Track 3 triage §C2/C5 fold-boundary artefact: the controller fire counters (anti_lr, tau, gamma, grad_clip, cql_alpha, cost_anneal), the running total-epochs denominator, and the prev-controller snapshot were NOT reset in reset_for_fold(), causing the 2/60 fire rates for tau and cql_alpha to accumulate across folds under cosine-annealed tau jumps when train_step resets + cql_alpha schedule drift. Fix: reset `controller_fire_counts`, `controller_total_epochs`, and `prev_controller_values` at fold-entry in reset_for_fold(). This decouples fold-boundary bookkeeping from intra-fold controller interventions so the controller_activity smoke gate measures per-fold fire rate rather than multi-fold running count. `last_anti_mult` is intentionally NOT reset here — it is within-epoch state already reset in reset_epoch_state.
This commit is contained in:
@@ -1466,6 +1466,18 @@ impl DQNTrainer {
|
||||
.map_err(|e| anyhow::anyhow!("guard reset: {e}"))?;
|
||||
}
|
||||
|
||||
// Task 2.5 Bug #5 — reset controller fire counters + total-epochs +
|
||||
// prev-controller snapshot at fold boundary. Without this, Track 3
|
||||
// C2/C5 fold-boundary artefacts (cosine-annealed tau jumps when
|
||||
// train_step resets, cql_alpha schedule drift across folds) would
|
||||
// accumulate into a multi-fold running count, misrepresenting the
|
||||
// per-fold fire rate for the controller_activity smoke gate.
|
||||
// `last_anti_mult` is within-epoch state (reset in reset_epoch_state)
|
||||
// and not reset here.
|
||||
self.controller_fire_counts = ControllerFireCounts::default();
|
||||
self.controller_total_epochs = 0;
|
||||
self.prev_controller_values = ControllerPrevValues::default();
|
||||
|
||||
// Keep gpu_data — the full dataset is already on GPU via init_from_fxcache.
|
||||
// Clearing it would force a redundant re-upload on the next fold.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user