diff --git a/docs/dqn-backward-nan-audit.md b/docs/dqn-backward-nan-audit.md index 3a030f016..0b35f723e 100644 --- a/docs/dqn-backward-nan-audit.md +++ b/docs/dqn-backward-nan-audit.md @@ -610,3 +610,60 @@ F0's Best Sharpe dropped from 55.87 (prior smoke `smoke-test-m5gxx`) to 34.55 in 3. **Buffer-allocation impact** — the nan_flags_buf 24→48 expansion allocates more pinned memory, could compete with other allocations. If F0 Sharpe stays below 53.08 (95% floor) after Task 6 fix, this becomes a separate investigation thread within SP1 (no deferral per operating principles). Most likely (1) — stochastic variance — but verify after Task 6. + +## Phase D — Multi-fold validation smoke (commit 19b008e1c) — FAIL + +**Workflow:** smoke-test-dr2bn (L40S, plain, 16m run) +**Commit:** 19b008e1c (Task 6 surgical fix + quality follow-up) +**Outcome:** ALL 7 SP1 pass criteria FAIL. + +### Per-criterion results + +| Criterion | Result | Value | +|---|---|---| +| 1. zero NaN-CLAMPED-TO-ZERO | FAIL | 5 lines (F2 cascade) | +| 2. zero NaN/Inf at step | FAIL | 2 lines (F1 step 240, loss=32.12, grad=inf) | +| 3. F0 ≥ 53.08 | FAIL | F0 = 35.24 (similar to pre-fix 34.55; Phase B regression unchanged by Phase C) | +| 4. F1 monotone | FAIL | F1 hard-failed at step 240 (halt_nan path) | +| 5. F2 monotone | FAIL | F2 grad-collapse cascade, weights inherited from F1 corruption | +| 6. zero flagged slots | FAIL | 6 flagged lines (F1 [6,12,26,32]; F2 cascade adds 2,3,7,8,24,25,29,33,34) | +| 7. 3 folds complete 5 epochs | FAIL | only F0 | + +### F1 first-fire signature (step 240) + +`flagged=[6=grad_buf, 12=save_h_s2, 26=iqn_trunk_m, 32=bn_d_concat_buf]` — IDENTICAL to pre-fix smoke `smoke-test-xvzgk` (commit f139a63ee), but at much earlier step (240 vs 890). + +### Diagnosis: the surgical fix DESTABILIZES F1 STARTUP, not prevents NaN + +The earlier-step NaN proves the surgical fix is causing problems, not just failing to address them. Hypothesis chain: + +1. Pre-clamp formula `(1e6 × ISV).max(1e3)` — at fold boundary, ISV[96] (H_S2_RMS_EMA) and ISV[21] (Q_DIR_ABS_REF) likely RESET to 0 (per `feedback_isv_dynamic` and the StateResetRegistry behavior at fold boundary). +2. With ISV = 0, formula yields `max(1e6 × 0, 1e3) = 1e3`. +3. F1 startup gradients can have natural magnitudes > 1e3 (post-fold-shift in Bellman target distribution). +4. Clamp aggressively clips legitimate gradients down to ±1e3 — significant signal loss. +5. Clipped gradients destabilize Adam EMAs → unstable weight updates → cuBLAS GEMM intermediate products overflow → NaN at slot 26 + 32. +6. Post-clamps catch the NaN (regression sentinel works) and sanitize, but the damage to weights is already done. + +The 1e3 ε floor was intended as a "numerical-stability ε for uninitialized state" (Invariant 1 carve-out), but it's actively destructive when fold-boundary ISV reset puts the bound into a regime where it clips legitimate signal. + +### Evidence — F0 regression unchanged by Phase C + +F0 Best Sharpe in three smokes: +- `smoke-test-m5gxx` (commit e9096c7be, baseline): 55.87 +- `smoke-test-xvzgk` (commit f139a63ee, Phase B alone): 34.55 +- `smoke-test-dr2bn` (commit 19b008e1c, Phase B + C): 35.24 + +Phase C surgical fix did NOT change F0 (within stochastic variance), confirming the Phase B instrumentation is the F0 regression source. F0 doesn't trigger the clamp (ISV[96] = ~1.0 by the time F0 starts; max_abs = 1e6, way above F0 inputs). So F0 regression is from a DIFFERENT mechanism in Phase B (e.g., 11 new check_nan_f32 kernel launches per step interfering with graph-capture timing or memory ordering). + +### Next iteration (Task 6 fix-up #2) + +**Fix proposal:** change ε floor formula from `(1e6 * isv).max(1e3)` to `(1e6 * isv.max(1.0_f32)).max(1e6_f32)` — equivalent to `1e6 * isv.max(1.0)`. This guarantees max_abs ≥ 1e6 regardless of ISV state: +- ISV = 0 → max_abs = 1e6 × max(0, 1.0) = 1e6 +- ISV = 0.5 → max_abs = 1e6 × 1.0 = 1e6 +- ISV = 2.0 → max_abs = 1e6 × 2.0 = 2e6 +- ISV = 100 → max_abs = 1e8 + +This preserves the F0 paper-review intent (1e6 = wide guard band) while removing the cold-start clamp pathology. F1 startup gradients ≤ 1e6 (which is the entire reasonable range) won't be clipped. + +**F0 regression — separate investigation thread within SP1.** Per no-deferrals principle, must be addressed before SP1 can close. Likely root cause: Phase B instrumentation kernel-launch overhead OR graph-capture memory ordering. Investigation deferred to a Task 7.5 sub-task pending Task 6 fix-up validation. +