diff --git a/docs/dqn-backward-nan-audit.md b/docs/dqn-backward-nan-audit.md index d29ddcee0..e2707363f 100644 --- a/docs/dqn-backward-nan-audit.md +++ b/docs/dqn-backward-nan-audit.md @@ -804,3 +804,65 @@ If Gate 1 fails, halt before SP3 starts and re-investigate F0 cause (alignment e 5. Strict halt would consume L40S smokes characterizing F0 variance with no clear payoff vs proceeding to SP3. **Phase B (SP3) starts at commit e270b5cc4.** + +--- + +## Phase F — SP3 Q-learning structural stability (5 mechanisms) + +**Status:** SP3 implementation complete (Tasks B1-B7 on branch `plan-c-phase-2-thompson`). Gate 2 smoke pending (Task B9). + +### Mechanism summary + +| # | Mechanism | Commit | File(s) | +|---|---|---|---| +| Mech 1 | Target-Q clipping at single-point source (`compute_denoise_target_q` end) | `27ed7daa6` | gpu_dqn_trainer.rs | +| Mech 2 | C51 atom-position growth bounds (3 sites: atoms_update, iql_value, experience) | `96b77043e` | atoms_update_kernel.cu, iql_value_kernel.cu, experience_kernels.cu | +| Mech 3 | Hard target sync at fold boundary (DQN main + IQN target) | pre-existing | fused_training.rs::reset_for_fold | +| Mech 4 | Comprehensive Adam EMA reset at fold transitions (incl. NEW: GpuAttention) | `ef429c25d` | gpu_attention.rs, fused_training.rs | +| Mech 5 | Adam-centric diagnostic in slots 36-47 via fused-kernel extension | `45e077188` + `5f7db7d8d` | dqn_utility_kernels.cu, gpu_dqn_trainer.rs, training_loop.rs | + +**Supporting:** Task B1 (`aee11f321`) — 24 diagnostic accessors (Adam m/v + weight + target_q + atom_positions). + +### Mech 5 slot allocation (slots 36-47) + +| Slot | Name | Threshold | Source | +|---|---|---|---| +| 36 | trunk_adam_m_max | ≥ 100 × q_abs_ref_eff | trunk Adam m slice | +| 37 | value_adam_m_max | same | value Adam m slice | +| 38 | branch_adam_m_max | same | branch Adam m slice | +| 39 | iqn_adam_m_max | same | IQN Adam m (separate buffer) | +| 40 | trunk_adam_v_max | ≥ 1e6 × q_abs_ref_eff² | trunk Adam v slice | +| 41 | value_adam_v_max | same | value Adam v slice | +| 42 | branch_adam_v_max | same | branch Adam v slice | +| 43 | iqn_adam_v_max | same | IQN Adam v (separate buffer) | +| 44 | trunk_weight_max | ≥ 1e3 × q_abs_ref_eff | trunk params slice | +| 45 | heads_weight_max | same | heads params slice | +| 46 | target_q_post_clip | ≥ 95 × q_abs_ref_eff | denoise_target_q_buf | +| 47 | atom_span_max | ≥ 190 × q_abs_ref_eff | per_sample_support | + +`q_abs_ref_eff = max(isv[Q_ABS_REF_INDEX=16], 1.0)` — ε on multiplier per SP1 pearl. Cold-start ISV (~0) → all thresholds floor at their multiplier × 1. + +### ISV slot bindings + +All SP3 mechanisms use existing `Q_ABS_REF_INDEX = 16`. NO new ISV slots. + +### Expected validation outcome (Gate 2 smoke) + +All 7 SP1 pass criteria satisfied: +1. Zero NaN-CLAMPED-TO-ZERO log lines +2. Zero NaN/Inf at step errors +3. F0 Best Sharpe ≥ 53.08 (currently ~46.58 from SP2 Gate 1; SP3 Mech 4 Adam EMA reset + Mech 3 target sync may further restore F0) +4. F1 last-epoch ≥ first-epoch +5. F2 last-epoch ≥ first-epoch +6. Zero flagged slots throughout (slots 24-47 all remain at zero) +7. All 3 folds complete 5 epochs + +### Diagnostic interpretation if Gate 2 fails + +Per slot 36-47 firing pattern: +- Slots 38-43 (Adam m/v) fire: Adam EMAs saturated despite reset; need more frequent resets OR post-Adam weight clamp +- Slots 44-45 (weights) fire alone, Adam clean: direct weight drift; add post-Adam weight clamp +- Slot 46 (target_q post-clip) NEVER fires: Mech 1 unnecessary defense; document, no action +- Slot 47 (atom span) fires often: Mech 2 doing real work; no action +- Slots 24-35 fire post-fix: SP3 missed pathology class; new investigation thread +