fix(dqn): reset gradient_collapse_counter + training_steps at fold boundary (A.3)

Smoke smoke-test-vh9bj revealed: after F+H Q-drift kill fix, fold 0
trains successfully (5 epochs, Best Sharpe 36.03), but fold 1 fails
at epoch 1 with "Gradient collapse detected for 5 consecutive epochs"
despite epoch 1 grad_norm=296,417 (healthy). Root cause: the per-step
gradient_collapse_counter on DQN was already near patience (5) from
fold 0's late-epoch near-zero-grad steps, and DQNTrainer::reset_for_fold
didn't clear it. Plus, training_steps accumulating across folds made
the `past_warmup` gate always-true in fold 1+, removing the warmup
grace period for data distribution shifts.

Add DQN::reset_for_fold zeroing both. Wire from DQNTrainer::reset_for_fold
alongside the A.1 prev_epoch_q_mean reset. Pure additive — same
fold-boundary state-reset gap pattern as A.1, A.2, F.

Predicted impact: fold 1+ now starts with counter=0 and warmup window
restored; gradient collapse check has its full per-fold grace period.

crates/ml-dqn/src/dqn.rs

@@ -1943,6 +1943,33 @@ impl DQN {
         self.count_bonus.reset();
     }
 
+    /// Reset state that should not persist across walk-forward fold
+    /// boundaries. Pairs with `DQNTrainer::reset_for_fold`.
+    ///
+    /// What's reset (and why):
+    /// - `gradient_collapse_counter` — counts consecutive low-grad steps;
+    ///   carrying across folds caused fold 1 to trip patience after a
+    ///   single collapse-tier step (counter starts already near patience).
+    /// - `training_steps` — the warmup gate `past_warmup = training_steps
+    ///   > warmup_steps` becomes always-true after fold 0 completes,
+    ///   removing the per-fold warmup grace period for data distribution
+    ///   shifts at fold boundaries. Resetting gives fold 1+ a clean
+    ///   warmup window before the gradient-collapse check fires.
+    ///
+    /// What's NOT reset (model + agent state):
+    /// - Model parameters (the whole point of walk-forward; fold N+1
+    ///   bootstraps from fold N's learned weights)
+    /// - count_bonus state (Plan C T2 amendment removed direct dependence
+    ///   on UCB count bonus; the count_bonus persistence is handled by
+    ///   `reset_count_bonus` called explicitly elsewhere if needed)
+    /// - target_network (handled by `reset_target_network` if explicitly
+    ///   wanted; per project_fold_boundary_q_drift_resolved.md the IQN
+    ///   sync_target_from_online is the principled mechanism)
+    pub fn reset_for_fold(&mut self) {
+        self.gradient_collapse_counter = 0;
+        self.training_steps = 0;
+    }
+
     /// Get count bonuses for all actions (UCB exploration).
     /// Returns a Vec<f32> of length `num_actions` with exploration bonuses.
     pub fn get_count_bonuses(&self) -> Vec<f32> {

crates/ml/src/trainers/dqn/config.rs

@@ -112,6 +112,13 @@ impl DQNAgentType {
         self.agent.reset_count_bonus();
     }
 
+    /// Reset DQN state that should not persist across walk-forward folds.
+    /// Delegates to `DQN::reset_for_fold` (zeros `gradient_collapse_counter`
+    /// and `training_steps`). Pairs with `DQNTrainer::reset_for_fold`.
+    pub fn reset_for_fold(&mut self) {
+        self.agent.reset_for_fold();
+    }
+
     /// Get count bonuses for all actions (UCB exploration).
     /// Returns a Vec<f32> of length `num_actions` with exploration bonuses.
     pub fn get_count_bonuses(&self) -> Vec<f32> {

crates/ml/src/trainers/dqn/trainer/mod.rs

@@ -1749,6 +1749,20 @@ impl DQNTrainer {
         self.best_epoch = 0;
         self.best_val_loss = f64::INFINITY;
 
+        // Plan C T11 follow-up A.3 (2026-04-29): reset DQN agent state that's
+        // keyed to fold lifecycle (gradient_collapse_counter, training_steps).
+        // Pairs with the prev_epoch_q_mean reset above — see DQN::reset_for_fold
+        // for the contract. Without this, smoke smoke-test-vh9bj fold 1 ep1
+        // tripped the gradient-collapse patience instantly because fold 0's
+        // late-epoch near-zero-grad steps had pushed the counter near patience,
+        // and `past_warmup = training_steps > warmup_steps` was always-true in
+        // fold 1+ (no per-fold warmup grace period for data distribution shifts).
+        // Same fold-boundary state-reset gap pattern as A.1.
+        {
+            let mut agent = self.agent.write().await;
+            agent.reset_for_fold();
+        }
+
         // Plan 1 Task 3 + Plan 2 Task 4: registry-driven fold-boundary reset.
         // FoldReset entries are cleared/zeroed; SoftReset entries are written
         // to their bootstrap values (CPU-born input, not adaptive computation)

docs/dqn-wire-up-audit.md

@@ -2198,3 +2198,5 @@ Plan C Phase 2 follow-up H (2026-04-29): replace brittle `ratio = |q_mean| / |pr
 Plan C Phase 2 follow-up F (2026-04-29): wire `q_mag_bin_means_reduce` and `q_dir_bin_means_reduce` per-step alongside `update_isv_signals` in `fused_training.rs` (both the captured `adam_update` child graph at ~line 2228 AND the ungraphed step-0 fallback at ~line 1465). **Root cause for the smoke-test-n9xzr `ISV[16]=0.0500, ISV[21]=0.0500` floor stuck at the cold-start `max(0.5, 3 × 0.05) = 0.5` value**: the producers ran ONLY inside `reduce_current_q_stats` (epoch boundary cadence) at `gpu_dqn_trainer.rs:15223`, while the per-step captured `update_isv_signals` consumed scratch buffers that were therefore zero-initialized for all of epoch 0 and stale-by-one-epoch thereafter. With α=0.05 EMA and one effective producer fire per epoch, ISV[16,21] climbed glacially (0.05 → 0.0975 → 0.143 over three epochs vs the `q_mean=0.7647` magnitude actually being observed), so the adaptive floor never graduated above the 0.5 cold-start hard-floor. The kill criterion's `kill_floor = max(0.5, 3 × max(ISV[16], ISV[21]))` collapsed to the literal `0.5` constant for the entire smoke. The fix runs both `launch_q_mag_bin_means_reduce` and `launch_q_dir_bin_means_reduce` immediately before `update_isv_signals` in BOTH paths so per-step graph replays read fresh `q_out_buf` (already populated by `forward_child` earlier in the parent-graph topological order) and feed live scalars into the per-step ISV EMA. Epoch 1 onward then sees ISV[16,21] saturate to the policy's actual q_abs_ref scale within ~60 steps (95% saturation at α=0.05). Per `pearl_cold_path_no_exception_to_gpu_drives.md`: even cold-path EMAs stay on GPU and fire alongside their consumers — once-per-epoch was the wrong cadence. Per `feedback_no_partial_refactor.md`: graphed and ungraphed paths both migrate in this commit (same producer-consumer contract). Layout fingerprint unchanged — no kernel signature change, no ISV slot add, no param-tensor change.
 
 Plan C Phase 2 follow-up (2026-04-29): bonus reward optimism-coupling break in `experience_kernels.cu`. The C.4 timing bonus (~line 2521) and D.4b regime penalty (~line 2581) both used the trade-cumulative `|final_pnl|` / `|reward|` as their unbounded multiplicand. By the time bonus shaping runs in `experience_env_step`, `reward` is the Layer 2 vol-normalised return PLUS Layer 4-9 credits PLUS the C.4 timing bonus PLUS the D.4a persistence bonus — i.e. it is itself a function of all prior shaping inflation. Across trades this compounds into a multi-trade optimism feedback loop: bonus inflates → Q-target inflates (Bellman bootstraps off shaped reward) → next trade's reward inflates → bonus inflates further. Researcher reproduction (a25f669e9df953174, 2026-04-29) found the a52d99613 baseline ALSO hits bonus EMA=235 by F2 ep2 — the pathology is structural and pre-dates Plan C Phase 2. Per `pearl_one_unbounded_signal_per_reward.md` the rule is exactly ONE unbounded multiplicand per reward term, and per `feedback_isv_for_adaptive_bounds.md` adaptive bounds live in the ISV signal bus. **Surgical fix**: replace `|final_pnl|` (C.4) and `|reward|` (D.4b) with `min(|x|/ISV[Q_DIR_ABS_REF_INDEX=21], 1) × ISV[21]` — the multiplicand is now the direction-branch Q-scale EMA (already produced gradient-decoupled by `q_stats_kernel.cu` since Plan 1) rather than the trade-cumulative shaping output. Cap is adaptive (matches Q magnitude as it evolves) and breaks cross-trade compounding; |reward| can still drive the cap to its max (= ISV[21]) so directional information is preserved, just bounded. **Kernel signature**: `experience_env_step` gains a final `int q_dir_abs_ref_idx` parameter (after the existing `trade_target_rate_idx`); single Rust call site `gpu_experience_collector.rs:3800` passes `Q_DIR_ABS_REF_INDEX as i32`. `experience_action_select` not affected (it never read `|reward|`/`|final_pnl|` as a multiplicand). **Other shaping sites NOT touched** because they were already bounded — D.4a persist (multiplicand `drawdown_depth ≤ ~0.1` gates the term, `tanh(reward/dd)` is bounded), B.2 novelty (vol_proxy ≤ 0.01 gates, kl_amp ≤ 1.0), D.4c stable (vol_proxy gates + bounded stability/conviction). Per `feedback_no_partial_refactor.md` consistency: both fixed sites use the SAME ISV[21] bound and the SAME `(unit, capped)` two-step formula — symmetric migration. Predicted impact: bonus EMA O(100-256) → O(0.05-2.0); rc[5] → Bellman-target optimism loop broken; Plan C smoke F0 ep2 Q-drift kill should no longer fire on this pathway; a52d99613 baseline gets the same fix automatically (validates pre-existing pathology, not Plan-C-specific). Layout fingerprint unchanged — kernel signature change but no ISV slot add and no param-tensor change.
+
+Plan C T11 follow-up A.3 (2026-04-29): added `DQN::reset_for_fold` zeroing `gradient_collapse_counter` + `training_steps`. Wired from `DQNTrainer::reset_for_fold` next to the A.1 `prev_epoch_q_mean` reset. Discovered when `smoke-test-vh9bj`'s fold-0 succeeded (F+H fix worked) but fold-1 failed immediately at epoch 1 with "Gradient collapse detected for 5 consecutive" — the counter had carried over from fold 0's near-zero-grad late-epoch steps, plus the `past_warmup` gate was always-true in fold 1+ because `training_steps` accumulated. Resetting both restores per-fold warmup semantics. Same fold-boundary state-reset gap pattern as A.1.