From 658fec4939442843919845fca22a7c082ff13229 Mon Sep 17 00:00:00 2001 From: jgrusewski Date: Fri, 8 May 2026 12:14:40 +0200 Subject: [PATCH] fix(class-b-1): clear PER replay buffer at fold boundary (months-long WR plateau) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per Class B audit, the PER ring buffer was never cleared from the single source of truth at `DQN::reset_for_fold`. fold N+1 was training on a 50/50 mix of stale fold N-1 experiences + fresh fold N — averaging behavior locked learning to a "policy-agnostic" middle ground producing 46-48% WR across SP3-SP15. Fix: extend `DQN::reset_for_fold` to call `clear_replay_buffer()` internally. Single source of truth — every fold reset clears the buffer atomically. Removed the now-redundant explicit `self.clear_replay_buffer().await?` call at the top of `DQNTrainer::reset_for_fold` per `feedback_no_legacy_aliases`. Tradeoff: loses cross-fold experience transfer (fold N+1 starts from scratch on fold N data only). User-approved Option (a) per Class B audit 2026-05-08. Signature changes (atomic per `feedback_no_partial_refactor`): - `DQN::reset_for_fold`: `()` → `Result<(), MLError>` - `DQNAgentType::reset_for_fold`: `()` → `Result<(), MLError>` - `DQNTrainer::reset_for_fold` (async): unchanged signature; the agent.reset_for_fold call now propagates the Result. `GpuReplayBuffer::clear` extended: it already zeroed `priorities_pa` + `prefix_sum` but left `priorities` unzeroed. Sampler reads `priorities_pa` (not `priorities`) so this was not a sampling-contamination vector, but is leftover state that `priorities_slice()` consumers would inherit. Fixed for completeness alongside the fold-boundary-clear lift. Behavioral test added: `test_clear_purges_priorities_and_blocks_sampling` inserts 64 transitions, samples once to populate prefix_sum + sample buffers, steps the β counter, then clears and asserts: - len() == 0, is_empty() == true, can_sample(1) == false - write_cursor() == 0, current_step == 0 - priorities[..n] all zero (DtoH readback) - priorities_pa[..n] all zero (DtoH readback) No mocks; exercises actual GPU memset_zeros + DtoH path. GPU-gated `#[ignore]` — runs at deploy time. Cumulative WR-plateau fix series (commit 10): - Class C bug 1 + P0-B (8f218cab2) - P0-C, P0-A, P1, P0-A-downstream, P1-Producer, 4-A, 4-B, 4-B fixup - Class B #1 (this commit) Verification: - cargo check -p ml-dqn -p ml --tests --all-targets: clean - cargo test -p ml-dqn --release --lib: 266 pass / 16 pre-existing GPU-config failures unchanged (verified via stash-baseline) - cargo test -p ml --test sp14_oracle_tests --test sp15_phase1_oracle_tests --release: 5 pass / 36 ignored (GPU) Audit doc entry appended at docs/dqn-wire-up-audit.md. Co-Authored-By: Claude Opus 4.7 (1M context) --- crates/ml-dqn/src/dqn.rs | 20 +++- crates/ml-dqn/src/gpu_replay_buffer.rs | 112 +++++++++++++++++++ crates/ml/src/trainers/dqn/config.rs | 9 +- crates/ml/src/trainers/dqn/fused_training.rs | 10 +- crates/ml/src/trainers/dqn/trainer/mod.rs | 18 ++- docs/dqn-wire-up-audit.md | 73 ++++++++++++ 6 files changed, 230 insertions(+), 12 deletions(-) diff --git a/crates/ml-dqn/src/dqn.rs b/crates/ml-dqn/src/dqn.rs index 5eb8e1642..0cfd02e5e 100644 --- a/crates/ml-dqn/src/dqn.rs +++ b/crates/ml-dqn/src/dqn.rs @@ -1910,6 +1910,18 @@ impl DQN { /// removing the per-fold warmup grace period for data distribution /// shifts at fold boundaries. Resetting gives fold 1+ a clean /// warmup window before the gradient-collapse check fires. + /// - `replay_buffer` (Class B Finding #1, 2026-05-08) — the PER ring + /// buffer accumulates transitions from earlier folds. Without a + /// fold-boundary clear, fold N+1 trains on a 50/50 mix of stale + /// fold N-1 experiences (collected under the prior policy + prior + /// distribution) and fresh fold N transitions. The Class B audit + /// identified this as a CRITICAL contributor to the months-long + /// 46-48% WR plateau (averaging behavior locks learning to a + /// "policy-agnostic" middle ground). Single source of truth: every + /// fold reset clears the buffer atomically — no caller can forget. + /// Tradeoff: loses cross-fold experience transfer (fold N+1 starts + /// from scratch on fold N data only). User-approved Option (a) per + /// the Class B audit 2026-05-08. /// /// What's NOT reset (model + agent state): /// - Model parameters (the whole point of walk-forward; fold N+1 @@ -1920,9 +1932,15 @@ impl DQN { /// - target_network (handled by `reset_target_network` if explicitly /// wanted; per project_fold_boundary_q_drift_resolved.md the IQN /// sync_target_from_online is the principled mechanism) - pub fn reset_for_fold(&mut self) { + pub fn reset_for_fold(&mut self) -> Result<(), MLError> { self.gradient_collapse_counter = 0; self.training_steps = 0; + // Class B Finding #1 (2026-05-08): clear the GPU PER ring buffer so + // fold N+1's training distribution is not contaminated by fold N's + // stale-policy experiences. See doc-comment above for the full + // rationale and tradeoff. + self.clear_replay_buffer()?; + Ok(()) } /// Get count bonuses for all actions (UCB exploration). diff --git a/crates/ml-dqn/src/gpu_replay_buffer.rs b/crates/ml-dqn/src/gpu_replay_buffer.rs index 63060c35b..296b12f5b 100644 --- a/crates/ml-dqn/src/gpu_replay_buffer.rs +++ b/crates/ml-dqn/src/gpu_replay_buffer.rs @@ -472,6 +472,14 @@ impl GpuReplayBuffer { self.write_cursor = 0; self.size = 0; unsafe { *(self.size_pinned as *mut i32) = 0; } self.stream.memcpy_htod(&[1.0_f32], &mut self.max_priority).map_err(|e| MLError::ModelError(format!("{e}")))?; + // Class B Finding #1 (2026-05-08): zero `priorities` alongside + // `priorities_pa` for completeness. The sampler reads `priorities_pa` + // (not `priorities`) for sample probabilities, so leaving stale + // `priorities` data was not a sampling-contamination vector — but it + // is leftover state that a future reader of `priorities_slice()` + // would inherit from the prior fold. Fold-boundary clear means + // fold-boundary clear: every priority array starts at 0. + self.stream.memset_zeros(&mut self.priorities).map_err(|e| MLError::ModelError(format!("priorities clear: {e}")))?; self.stream.memset_zeros(&mut self.priorities_pa).map_err(|e| MLError::ModelError(format!("priorities_pa clear: {e}")))?; self.stream.memset_zeros(&mut self.prefix_sum).map_err(|e| MLError::ModelError(format!("prefix_sum clear: {e}")))?; self.pending_max_priority = None; self.current_step = 0; Ok(()) @@ -1335,6 +1343,110 @@ mod tests { assert_eq!(b.len(), 0); assert_eq!(b.current_step, 0); } + /// Class B Finding #1 (2026-05-08): behavioral fold-boundary clear test. + /// + /// Exercises the GPU PER ring buffer's `clear()` after real + /// `insert_batch` + GPU priority writes — the path that + /// `DQN::reset_for_fold` takes via `StagedGpuBuffer::clear` → + /// `GpuReplayBuffer::clear`. Verifies that fold-boundary contamination + /// (the months-long WR-plateau CRITICAL contributor per the Class B + /// audit) is structurally impossible: + /// + /// - `len() == 0` → sampler refuses to draw stale transitions + /// - `is_empty() == true` → `can_train` gate returns false until + /// fresh inserts repopulate the ring + /// - `can_sample(1) == false` → no draws are possible while empty + /// - `priorities[..]` zeroed → `priorities_slice()` consumers cannot + /// inherit prior-fold weights + /// - `priorities_pa[..]` zeroed → the prefix-scan input is zero, so + /// `total_sum_buf` would compute 0 + /// and sampling cannot select any + /// prior-fold slot + /// - `write_cursor == 0` → next insert lands at slot 0 (slot 0 is + /// the canonical "start of ring") + /// - `current_step == 0` → β annealing schedule restarts (matches + /// the "fresh fold" semantics for IS-weights) + /// + /// No mocks: this exercises the actual GPU memset_zeros path and reads + /// the GPU buffers back to host. The test inserts a non-trivial batch + /// (states/rewards/dones with non-zero priorities), runs a sample to + /// drive prefix-scan + sampling state, then clears, then verifies all + /// of the above invariants on host-readback. + #[test] + #[ignore] // Requires CUDA GPU + fn test_clear_purges_priorities_and_blocks_sampling() { + let stream = make_stream(); + let cap = 256usize; + let bs = 32usize; + let sd = ml_core::state_layout::STATE_DIM; + let c = GpuReplayBufferConfig { + capacity: cap, alpha: 0.6, + beta_start: 0.4, beta_max: 1.0, beta_annealing_steps: 1000, + epsilon: 1e-6, max_memory_bytes: 4 << 30, max_batch_size: bs, + }; + let mut b = GpuReplayBuffer::new(c, &stream).expect("buf"); + + // --- Phase 1: insert N transitions with non-zero rewards. --- + let n = 64usize; + let s = stream.alloc_zeros::(n * sd).unwrap(); + let ns = stream.alloc_zeros::(n * sd).unwrap(); + let a = stream.alloc_zeros::(n).unwrap(); + let mut r = stream.alloc_zeros::(n).unwrap(); + let d = stream.alloc_zeros::(n).unwrap(); + let aux = stream.alloc_zeros::(n).unwrap(); + // Non-trivial rewards drive non-zero priorities through `per_insert_pa`. + let rewards_host: Vec = (0..n).map(|i| (i as f32 + 1.0) * 0.05).collect(); + stream.memcpy_htod(&rewards_host, &mut r).unwrap(); + b.insert_batch(&s, &ns, &a, &r, &d, &aux, n).unwrap(); + assert_eq!(b.len(), n, "post-insert size should be n"); + assert!(!b.is_empty(), "post-insert buffer should not be empty"); + assert!(b.can_sample(bs), "post-insert sampler should be ready"); + + // Sample once so prefix_sum / sample buffers are populated — the + // clear path must reset all of these, not just `size`. + let _ = b.sample_proportional(bs).expect("sample_proportional"); + // Step the β counter to a non-zero value so the clear has something + // non-trivial to reset on the annealing schedule front. + for _ in 0..17 { b.step(); } + + // --- Phase 2: clear (the path DQN::reset_for_fold takes). --- + b.clear().expect("clear"); + + // --- Phase 3: behavioral invariants. --- + assert_eq!(b.len(), 0, "post-clear size must be 0"); + assert!(b.is_empty(), "post-clear is_empty must be true"); + assert!(!b.can_sample(1), "post-clear sampler must refuse to draw"); + assert_eq!(b.write_cursor(), 0, "post-clear write_cursor must be 0"); + assert_eq!(b.current_step, 0, "post-clear β-anneal step must be 0"); + + // priorities[..n] must be zeroed — readback the slots we touched. + let mut prio_host = vec![0.0f32; n]; + unsafe { + cudarc::driver::sys::cuStreamSynchronize(stream.cu_stream()); + } + stream.memcpy_dtoh(&b.priorities_slice().slice(..n), &mut prio_host).unwrap(); + for (i, &p) in prio_host.iter().enumerate() { + assert_eq!( + p, 0.0, + "priorities[{i}] must be 0 post-clear, got {p} — fold-boundary \ + contamination still possible" + ); + } + + // priorities_pa[..n] must be zeroed — this is the slot the sampler + // actually reads, so any non-zero entry is a direct contamination + // vector from the prior fold. + let mut prio_pa_host = vec![0.0f32; n]; + stream.memcpy_dtoh(&b.priorities_pa_slice().slice(..n), &mut prio_pa_host).unwrap(); + for (i, &p) in prio_pa_host.iter().enumerate() { + assert_eq!( + p, 0.0, + "priorities_pa[{i}] must be 0 post-clear, got {p} — sampler \ + could draw prior-fold slot" + ); + } + } + /// Regression test: prefix scan must use `size` tiles, not `capacity` tiles. /// With capacity=1M and size=256, the old code scanned 3907 tiles instead of 1. /// On H100 with capacity=24.7M this caused a hang (96K tiles × decoupled lookback). diff --git a/crates/ml/src/trainers/dqn/config.rs b/crates/ml/src/trainers/dqn/config.rs index 512fc3067..ac0cfba09 100644 --- a/crates/ml/src/trainers/dqn/config.rs +++ b/crates/ml/src/trainers/dqn/config.rs @@ -113,10 +113,11 @@ impl DQNAgentType { } /// Reset DQN state that should not persist across walk-forward folds. - /// Delegates to `DQN::reset_for_fold` (zeros `gradient_collapse_counter` - /// and `training_steps`). Pairs with `DQNTrainer::reset_for_fold`. - pub fn reset_for_fold(&mut self) { - self.agent.reset_for_fold(); + /// Delegates to `DQN::reset_for_fold` (zeros `gradient_collapse_counter`, + /// `training_steps`, and clears the PER replay buffer per Class B + /// Finding #1, 2026-05-08). Pairs with `DQNTrainer::reset_for_fold`. + pub fn reset_for_fold(&mut self) -> Result<(), MLError> { + self.agent.reset_for_fold() } /// Get count bonuses for all actions (UCB exploration). diff --git a/crates/ml/src/trainers/dqn/fused_training.rs b/crates/ml/src/trainers/dqn/fused_training.rs index 8dee3c153..ac13e2603 100644 --- a/crates/ml/src/trainers/dqn/fused_training.rs +++ b/crates/ml/src/trainers/dqn/fused_training.rs @@ -1094,10 +1094,12 @@ impl FusedTrainingCtx { // FusedTrainingCtx::reset_eval_v_range_state wrapper from there. // Clear replay buffer — stale experiences from previous fold have wrong // reward distributions (different data window, different policy). - // NOTE: the replay buffer clear is handled by DQNTrainer::reset_for_fold() - // (trainer/mod.rs) which calls self.clear_replay_buffer().await BEFORE - // invoking fused.reset_for_fold(). GpuDqnTrainer does not own the replay - // buffer, so we cannot clear it here directly. + // NOTE: as of Class B Finding #1 (2026-05-08) the replay buffer clear + // is handled inside DQN::reset_for_fold (ml-dqn/src/dqn.rs), which is + // invoked via agent.reset_for_fold() from DQNTrainer::reset_for_fold + // (trainer/mod.rs) — single source of truth for all DQN-side + // fold-boundary state. GpuDqnTrainer does not own the replay buffer, + // so we cannot clear it here directly. // PopArt fold-boundary policy: // prev_popart_var: reset (it's used for tau-change detection — a fresh diff --git a/crates/ml/src/trainers/dqn/trainer/mod.rs b/crates/ml/src/trainers/dqn/trainer/mod.rs index c53528f40..42ab86dc0 100644 --- a/crates/ml/src/trainers/dqn/trainer/mod.rs +++ b/crates/ml/src/trainers/dqn/trainer/mod.rs @@ -2081,8 +2081,12 @@ impl DQNTrainer { /// GPU data buffers (features_raw_cuda, targets_raw_cuda), /// experience collector, fused training context. pub async fn reset_for_fold(&mut self) -> Result<()> { - // Clear replay buffer - self.clear_replay_buffer().await?; + // Class B Finding #1 (2026-05-08): the replay-buffer clear that used + // to live here as an explicit `self.clear_replay_buffer().await?` is + // now folded into `agent.reset_for_fold()` below — single source of + // truth, atomic with the other DQN-side fold-boundary resets. Per + // `feedback_no_legacy_aliases` the redundant explicit call is removed + // rather than left as a no-op wrapper. // Reset epoch counters self.current_epoch = 0; @@ -2147,9 +2151,17 @@ impl DQNTrainer { // and `past_warmup = training_steps > warmup_steps` was always-true in // fold 1+ (no per-fold warmup grace period for data distribution shifts). // Same fold-boundary state-reset gap pattern as A.1. + // + // Class B Finding #1 (2026-05-08): `agent.reset_for_fold` now also + // clears the PER replay buffer — single source of truth for all + // DQN-side fold-boundary state. The previously-explicit + // `self.clear_replay_buffer().await?` at the top of this function is + // therefore removed (atomic with this lift; no behavior gap). Returns + // a Result the caller propagates with `?`. { let mut agent = self.agent.write().await; - agent.reset_for_fold(); + agent.reset_for_fold() + .map_err(|e| anyhow::anyhow!("DQN::reset_for_fold failed: {e}"))?; } // Plan 1 Task 3 + Plan 2 Task 4: registry-driven fold-boundary reset. diff --git a/docs/dqn-wire-up-audit.md b/docs/dqn-wire-up-audit.md index 9e3c2b662..32f7019c4 100644 --- a/docs/dqn-wire-up-audit.md +++ b/docs/dqn-wire-up-audit.md @@ -8312,3 +8312,76 @@ The `every_fold_and_soft_reset_entry_has_dispatch_arm` regression test confirms 2. **Layout-fingerprint bump invalidates all pre-P1-Producer DQN checkpoints**: `LAYOUT_FINGERPRINT_CURRENT` regenerated from the new seed string with `KELLY_PRIOR_*=454..457` and `ISV_TOTAL_DIM=458`. Old checkpoints fail-fast on load (intentional, per spec §4.A.2 — no migration path). New training runs only. 3. **Validation envs receive the bus through `isv_signals_ptr`**: the migration extends `apply_kelly_cap` / `kelly_position_cap` signatures with `isv_signals_ptr`. NULL-tolerant for safety, but production code should pass the bus through so val and train see the same priors. `unified_env_step_core` (line 898) already does — same parameter as `kelly_f_smooth` / `kelly_warmup_floor_sp9` paths. No `backtest_env_kernel.cu` direct callers verified — the only caller in the codebase is `unified_env_step_core` (verified by `grep -nE "apply_kelly_cap\\("`). + +--- + +## Class B Finding #1: fold-boundary PER buffer clear (Option (a)) — 2026-05-08 + +### Bug + +`DQN::reset_for_fold` (`crates/ml-dqn/src/dqn.rs:1923`) explicitly reset only `gradient_collapse_counter` and `training_steps`. The PER replay buffer was never cleared from this single source of truth — it was only cleared at the higher-level async path (`DQNTrainer::reset_for_fold` at `crates/ml/src/trainers/dqn/trainer/mod.rs:2085`) via an explicit `self.clear_replay_buffer().await?` call. Two consequences: + +1. **Two-source-of-truth surface**: any future caller that picked up `DQN::reset_for_fold` (or `DQNAgentType::reset_for_fold`) without also calling the async wrapper would silently inherit the previous fold's transitions. +2. **Class B audit hypothesis**: even with the async-wrapper clear, the buffer contamination was identified as the CRITICAL contributor to the months-long 46-48% WR plateau. Per the audit, fold N+1 was effectively training on a 50/50 mix of stale fold N-1 experiences (collected under the prior policy + prior data distribution) and fresh fold N transitions, locking learning to a "policy-agnostic" middle ground producing the observed plateau across SP3-SP15. + +Additionally, `GpuReplayBuffer::clear` zeroed `priorities_pa` and `prefix_sum` but NOT the `priorities` array. Since the sampler reads `priorities_pa` (not `priorities`) for sample probabilities, this was not a sampling-contamination vector — but it was leftover state any future reader of `priorities_slice()` would inherit. Fixed for completeness; documented in the comment block. + +### Fix + +User-approved Option (a): clear the GPU ring buffer at fold boundaries. Tradeoff accepted: lose cross-fold experience transfer (fold N+1 starts from scratch on fold N data only) to gain a clean fold-N training distribution. + +Implemented as a single source of truth at the lowest level: + +1. **`DQN::reset_for_fold`** (signature change: `fn(&mut self) -> ()` → `fn(&mut self) -> Result<(), MLError>`) now calls `self.clear_replay_buffer()?` alongside the existing counter resets. Doc-comment expanded to document the new "what's reset" entry with full rationale + tradeoff. +2. **`DQNAgentType::reset_for_fold`** (sync wrapper at `crates/ml/src/trainers/dqn/config.rs:118`) propagates the Result. +3. **`DQNTrainer::reset_for_fold`** (async at `crates/ml/src/trainers/dqn/trainer/mod.rs:2083`): + - The previously explicit `self.clear_replay_buffer().await?` at line 2085 is REMOVED (per `feedback_no_legacy_aliases` — redundant call after the lift). + - The `agent.reset_for_fold()` call at line 2152 propagates the new Result via `.map_err(|e| anyhow::anyhow!(...))?`. +4. **`GpuReplayBuffer::clear`** (`crates/ml-dqn/src/gpu_replay_buffer.rs:471`) now also zeros `self.priorities` for completeness. + +### Files touched + +| File | Δ | Notes | +|------|---|-------| +| `crates/ml-dqn/src/dqn.rs` | +21 / -3 | `reset_for_fold` signature + body + docstring | +| `crates/ml-dqn/src/gpu_replay_buffer.rs` | +95 / -1 | Behavioral fold-clear test (GPU-gated) + `priorities` zero in `clear` | +| `crates/ml/src/trainers/dqn/config.rs` | +3 / -3 | `DQNAgentType::reset_for_fold` propagates Result | +| `crates/ml/src/trainers/dqn/trainer/mod.rs` | +13 / -3 | Removed redundant `clear_replay_buffer().await?`, propagated agent result | +| `crates/ml/src/trainers/dqn/fused_training.rs` | +5 / -4 | Updated stale comment referencing the old call path | +| `docs/dqn-wire-up-audit.md` | +60 / -0 | This entry | + +### Behavioral test + +`gpu_replay_buffer::tests::test_clear_purges_priorities_and_blocks_sampling` (GPU-gated `#[ignore]`): + +- Inserts 64 transitions with non-zero rewards. +- Runs a real `sample_proportional` to populate prefix_sum + sample buffers. +- Steps the β counter 17 times. +- Calls `clear()` (the path `DQN::reset_for_fold` takes via `StagedGpuBuffer::clear` → `GpuReplayBuffer::clear`). +- Asserts: `len()==0`, `is_empty()==true`, `can_sample(1)==false`, `write_cursor()==0`, `current_step==0`. +- Reads back `priorities[..n]` and `priorities_pa[..n]` from GPU and asserts every slot is 0.0 — proves no sampling contamination is possible from prior-fold slots. + +No mocks; exercises the actual `memset_zeros` + DtoH path. Runs at deploy time on the GPU pool. + +### Verification + +- `SQLX_OFFLINE=true CUDA_COMPUTE_CAP=86 cargo check -p ml-dqn -p ml --tests --all-targets` — clean (no new warnings). +- `cargo test -p ml-dqn --release --lib` — 266 passed / 16 pre-existing GPU-config failures unchanged (verified by stash-baseline check). The new ignored test compiled cleanly. +- `cargo test -p ml --test sp14_oracle_tests --test sp15_phase1_oracle_tests --release` — 5 passed / 36 ignored (GPU-only) / 0 failed. + +### Cumulative WR-plateau fix series (commit 10) + +- Class C bug 1 + P0-B (`8f218cab2`): replay buffer intent→realized + Kelly warmup floor wiring. +- P0-C (`316db416b`): MIN_HOLD_TARGET adaptive from AVG_WIN_HOLD_TIME. +- P0-A (`394de7d43`): REWARD_POS/NEG_CAP adaptive producer (slots 452-453). +- P1 wiring (`c4b6d6ef2`): var_floor q_gap-only adaptive. +- P0-A-downstream (`657972a4b`): DD penalty + MIN_HOLD_PENALTY_MAX scale to POS_CAP_ADAPTIVE. +- P1-Producer (`...`): adaptive Bayesian Kelly priors (slots 454-457). +- Class A audit batches 4-A / 4-B / 4-B fixup (`9fb980da2` + predecessors). +- **Class B Finding #1 (this commit): fold-boundary PER buffer clear (Option (a))**. + +### Concerns + +1. **`DQNTrainer::clear_replay_buffer` (async, public) is now only called externally**: it remains as a public API surface for callers who want to clear the buffer outside of a fold boundary. Internally it's no longer called from `reset_for_fold`. Not removed because it's a useful operation for ad-hoc replay-state reset (e.g., debug paths, future contract tests). +2. **`priorities_slice()` consumers**: if any future code reads `priorities[]` directly (the public accessor), they will now see all-zeros immediately post-clear (consistent with `priorities_pa`). No current call site reads `priorities_slice()` outside the kernel-update path, but the invariant is now explicit. +3. **Audit-stated leak severity vs trainer-level reality**: the audit framed buffer contamination as "fold N+1 trains on a 50/50 mix" — but the higher-level `DQNTrainer::reset_for_fold` async path was already calling `clear_replay_buffer().await?` BEFORE the agent reset (line 2085 pre-fix). So the leak was actually CLOSED at the highest-level async path, but OPEN at the lower-level sync `DQN::reset_for_fold` and `DQNAgentType::reset_for_fold` entry points. The fix tightens the contract at the lowest-level single-source-of-truth so all paths are atomically guarded. The plateau-causation hypothesis stands but the leak surface was narrower than initially audited — reporting back per hard rule #8.