docs(dqn): SP3 Phase F — audit summary of B1-B7 + Gate 2 prep

This commit is contained in:
jgrusewski
2026-04-30 10:30:37 +02:00
parent 5f7db7d8d8
commit b9edccfc19

View File

@@ -804,3 +804,65 @@ If Gate 1 fails, halt before SP3 starts and re-investigate F0 cause (alignment e
5. Strict halt would consume L40S smokes characterizing F0 variance with no clear payoff vs proceeding to SP3.
**Phase B (SP3) starts at commit e270b5cc4.**
---
## Phase F — SP3 Q-learning structural stability (5 mechanisms)
**Status:** SP3 implementation complete (Tasks B1-B7 on branch `plan-c-phase-2-thompson`). Gate 2 smoke pending (Task B9).
### Mechanism summary
| # | Mechanism | Commit | File(s) |
|---|---|---|---|
| Mech 1 | Target-Q clipping at single-point source (`compute_denoise_target_q` end) | `27ed7daa6` | gpu_dqn_trainer.rs |
| Mech 2 | C51 atom-position growth bounds (3 sites: atoms_update, iql_value, experience) | `96b77043e` | atoms_update_kernel.cu, iql_value_kernel.cu, experience_kernels.cu |
| Mech 3 | Hard target sync at fold boundary (DQN main + IQN target) | pre-existing | fused_training.rs::reset_for_fold |
| Mech 4 | Comprehensive Adam EMA reset at fold transitions (incl. NEW: GpuAttention) | `ef429c25d` | gpu_attention.rs, fused_training.rs |
| Mech 5 | Adam-centric diagnostic in slots 36-47 via fused-kernel extension | `45e077188` + `5f7db7d8d` | dqn_utility_kernels.cu, gpu_dqn_trainer.rs, training_loop.rs |
**Supporting:** Task B1 (`aee11f321`) — 24 diagnostic accessors (Adam m/v + weight + target_q + atom_positions).
### Mech 5 slot allocation (slots 36-47)
| Slot | Name | Threshold | Source |
|---|---|---|---|
| 36 | trunk_adam_m_max | ≥ 100 × q_abs_ref_eff | trunk Adam m slice |
| 37 | value_adam_m_max | same | value Adam m slice |
| 38 | branch_adam_m_max | same | branch Adam m slice |
| 39 | iqn_adam_m_max | same | IQN Adam m (separate buffer) |
| 40 | trunk_adam_v_max | ≥ 1e6 × q_abs_ref_eff² | trunk Adam v slice |
| 41 | value_adam_v_max | same | value Adam v slice |
| 42 | branch_adam_v_max | same | branch Adam v slice |
| 43 | iqn_adam_v_max | same | IQN Adam v (separate buffer) |
| 44 | trunk_weight_max | ≥ 1e3 × q_abs_ref_eff | trunk params slice |
| 45 | heads_weight_max | same | heads params slice |
| 46 | target_q_post_clip | ≥ 95 × q_abs_ref_eff | denoise_target_q_buf |
| 47 | atom_span_max | ≥ 190 × q_abs_ref_eff | per_sample_support |
`q_abs_ref_eff = max(isv[Q_ABS_REF_INDEX=16], 1.0)` — ε on multiplier per SP1 pearl. Cold-start ISV (~0) → all thresholds floor at their multiplier × 1.
### ISV slot bindings
All SP3 mechanisms use existing `Q_ABS_REF_INDEX = 16`. NO new ISV slots.
### Expected validation outcome (Gate 2 smoke)
All 7 SP1 pass criteria satisfied:
1. Zero NaN-CLAMPED-TO-ZERO log lines
2. Zero NaN/Inf at step errors
3. F0 Best Sharpe ≥ 53.08 (currently ~46.58 from SP2 Gate 1; SP3 Mech 4 Adam EMA reset + Mech 3 target sync may further restore F0)
4. F1 last-epoch ≥ first-epoch
5. F2 last-epoch ≥ first-epoch
6. Zero flagged slots throughout (slots 24-47 all remain at zero)
7. All 3 folds complete 5 epochs
### Diagnostic interpretation if Gate 2 fails
Per slot 36-47 firing pattern:
- Slots 38-43 (Adam m/v) fire: Adam EMAs saturated despite reset; need more frequent resets OR post-Adam weight clamp
- Slots 44-45 (weights) fire alone, Adam clean: direct weight drift; add post-Adam weight clamp
- Slot 46 (target_q post-clip) NEVER fires: Mech 1 unnecessary defense; document, no action
- Slot 47 (atom span) fires often: Mech 2 doing real work; no action
- Slots 24-35 fire post-fix: SP3 missed pathology class; new investigation thread