fix(dqn): SP3 Mech 8 — slow_ema fold-boundary reset (real root cause)
User-identified root cause: grad_norm_slow_ema (α=0.001, half-life
~693 steps) persists across fold boundaries while every other
distribution-tracking signal (Adam m/v, target nets, atom positions)
gets reset. Mech 6's upper_bound formula (100 × slow_ema × isv) was
anchored to the WRONG scale during F1 ramp-up, driving the
non-monotonic multiplier-tuning dance across smokes:
- smoke-test-fxvkk (mult=100×): F0=44, F1 NaN @ 3720
- smoke-test-ftdjz (mult=100×, +Mech7): F0=38, F1 collapse @ 2040
- smoke-test-d25vq (mult=5×): F0=21, F1 collapse @ 2820
The multiplier was searching the wrong dimension — the anchor itself
was stale.
Three coordinated changes (per feedback_no_partial_refactor):
1. RESTORE Mech 6 multiplier 5× → 100×. The original 100× was correct
for steady-state; the F1 saturation was driven by anchor staleness,
not multiplier looseness. Tightening it harmed F0 (over-clip during
ramp-up when slow_ema lagged grad_norm_ema).
2. ADD reset_grad_norm_slow_ema method on GpuDqnTrainer. Zeroes the
mapped-pinned scalar. First step of new fold builds up slow_ema
fresh, with MIN_CLIP=1.0 floor active during the brief transient.
3. WIRE Mech 8 call in fused_training.rs::reset_for_fold, alongside
Mech 4's existing Adam resets. Mech 6's anchor now aligns with
the new fold's grad scale from step 1 — same philosophy as Mech 3
(target net hard-sync) and Mech 4 (Adam EMA reset).
Net SP3 design: Mech 6 stays at 100× multiplier (broad, principled
headroom), Mech 8 keeps the anchor honest. The pair is more robust
than either change alone:
- Without Mech 8: anchor is stale, multiplier tuning has no winning
setting (5× hurts F0 ramp, 100× lets F1 saturate at slot 36).
- With Mech 8: anchor is fresh per fold; 100× multiplier provides
legitimate per-step headroom over CURRENT fold's grad norm.
Mech 7 stays reverted (per-element clip was misdiagnosis — over-clipped
legitimate gradient outliers without addressing the saturation root
cause).
F0 risk: low — F0 starts with slow_ema=0 anyway (cold start), so Mech 8
is a no-op on F0. Only changes F1+F2 fold-boundary behavior.
F1+F2 expectation: Mech 6's upper_bound now scales with the CURRENT
fold's grad norm, providing legitimate ~10× headroom per step without
allowing Adam EMA saturation. Slot 36-42 should stay quiet.
This commit is contained in:
@@ -2,6 +2,8 @@
|
||||
|
||||
**Status:** Populated during Plan 1 Task 6 (A.5 orphan audit). Updated on every commit per Invariant 7.
|
||||
|
||||
SP3 Mech 8 — `grad_norm_slow_ema` fold-boundary reset (2026-04-29): closed the partial-refactor gap where the α=0.001 grad-norm slow EMA (driving Mech 6's anchored upper bound on `adaptive_clip`) persisted across fold boundaries while every other distribution-tracking signal — Adam m/v (`reset_adam_state`), main DQN target params (`sync_target_from_online`), IQN target + Adam (`iqn.sync_target_from_online` + `iqn.reset_adam_state`), TLOB / IQL / IQL-low / 4-head attention Adam (`reset_adam_state` on each), C51 atom-position EMAs, replay buffer — already reset per existing `reset_for_fold` infrastructure. The α=0.001 half-life (~693 steps) meant `grad_norm_slow_ema` lagged the new fold's grad-norm regime by hundreds of steps; Mech 6's `100 × slow_ema × isv` upper bound was anchored to the WRONG fold's scale during F1 ramp-up, producing the non-monotonic multiplier-tuning dance observed across smokes (smoke-test-fxvkk 100× F1-NaN @ 3720 / smoke-test-ftdjz 100×+Mech7 F1-collapse @ 2040 / smoke-test-d25vq 5× F1-collapse @ 2820 — the multiplier was searching the wrong dimension). Three coordinated changes per `feedback_no_partial_refactor.md`: (1) Mech 6 multiplier restored 5× → 100× in `gpu_dqn_trainer.rs::update_adaptive_clip` (the original principled setting; v2's 5× was over-clipping legitimate F0-ramp gradients while still allowing F1-anchor saturation), (2) new `GpuDqnTrainer::reset_grad_norm_slow_ema` method zeroes the existing mapped-pinned scalar (no new buffer, no new ISV slot — reuses Plan C Phase 2 follow-up K's grad_norm_slow_ema_pinned at `gpu_dqn_trainer.rs:3337`), (3) wired into `fused_training.rs::reset_for_fold` immediately after the SP3 Mech 4 attention Adam reset block (line ~1066) — same fold-boundary contract consumer chain as Mech 3+4. F0 unaffected (cold-starts at slow_ema=0 already, so Mech 8 is a no-op on F0); F1+F2 first step transient sees `upper_bound = 100 × 0 × isv → MIN_CLIP=1.0` floor for ~10-50 steps as the EMA warms, then converges to legitimate 100× headroom over CURRENT fold's grad norm. Mech 7 stays reverted (per-element clip was misdiagnosis). Touched: `cuda_pipeline/gpu_dqn_trainer.rs` (Mech 6 comment + multiplier 5.0→100.0 line ~20370; new `reset_grad_norm_slow_ema` method line ~3858), `trainers/dqn/fused_training.rs` (call wired line ~1067). cargo check clean. No fingerprint change — touches an already-allocated mapped-pinned scalar, not param-tensor layout.
|
||||
|
||||
HEALTH_DIAG GPU port — Phase 1 (2026-04-28, follow-up to commit `333ea7184`'s Phase 0 inventory): added `HealthDiagSnapshot` `#[repr(C)]` POD struct and `MappedHealthDiagSnapshot` mapped-pinned wrapper in new `crates/ml/src/cuda_pipeline/health_diag.rs`. The struct holds 147 fields (every numeric value emitted across the 7 HEALTH_DIAG log sites) as `f32` or `u32` so CPU and GPU agree byte-for-byte; controller fire booleans (`fire_lr`/`fire_tau`/etc.) promoted from `u8` → `u32` per the design review to avoid `#[repr(C)]` padding mismatch when followed by `f32` fields. `MappedHealthDiagSnapshot::new()` wraps `cuMemHostAlloc(DEVICEMAP|PORTABLE)` of `sizeof(HealthDiagSnapshot)` + `cuMemHostGetDevicePointer_v2` — the same pattern as `MappedF32Buffer` but typed for the snapshot. Re-exported from `cuda_pipeline/mod.rs` for downstream consumers. **No producer kernel and no consumer wiring in this commit** — Phase 2 lands the kernel family (`health_diag_per_sample_reduce`, `health_diag_q_mag_reduce`, `health_diag_eval_histogram`, `health_diag_isv_mirror`, `health_diag_finalise`) and the `gpu_health_diag.rs` orchestrator; Phase 3 wires the launch into the captured graph alongside `launch_h_s2_rms_ema`; Phase 4 rewrites the CPU emit path and deletes the 12 CPU-bound feeder methods identified in the Phase 0 inventory. Three unit tests (`snapshot_size_is_stable`, `default_is_zeroed`, `alignment_is_4_bytes`) pin the layout so any future field add/reorder requires an explicit test update — guards the kernel-side offset table from silent drift. Touched: `cuda_pipeline/health_diag.rs` (+339 LOC, new), `cuda_pipeline/mod.rs` (+8 LOC re-exports). cargo check clean at 12 warnings (workspace baseline). No fingerprint change — separate mapped-pinned alloc, not part of param-tensor layout the fingerprint guards.
|
||||
|
||||
multi_fold_convergence smoke MBP-10 wiring (2026-04-28): the test now forwards `--mbp10-data-dir` and `--trades-data-dir` to its spawned `train_baseline_rl` subprocess, reading paths from `FOXHUNT_MBP10_DATA` / `FOXHUNT_TRADES_DATA` env vars (set by the L40S sanitizer-test + nsys-test Argo templates to `/data/test-data/{mbp10,trades}`) with fallback to `<data_dir>/../futures-baseline-{mbp10,trades}` for local repo runs. The test hard-fails fast if either path is missing, citing `feedback_mbp10_mandatory.md`. Without this, OFI features (state slots [18..26)) silently fell back to tick-rule proxy and the smoke validated a degraded model variant — a `feedback_no_partial_refactor.md` violation now closed.
|
||||
|
||||
Reference in New Issue
Block a user