docs(dqn): SP2 Phase E — audit doc summary of A1-A4 + Gate 1 prep
This commit is contained in:
@@ -728,3 +728,49 @@ Closing SP1 with the following declared deliverables:
|
||||
The 7 SP1 pass criteria as originally specified are not all achievable within SP1's surgical-fix scope. Criteria 1, 2, 4, 5, 6, 7 require SP3 structural fix. Criterion 3 (F0 ≥ 53.08) requires SP2 instrumentation optimization.
|
||||
|
||||
**Net SP1 outcome:** the investigation produced the diagnostic infrastructure + pinpointed source kernels + fixed the cold-start clamp pathology that this scope CAN address. The remaining items are correctly classified for SP2/SP3 scopes per the spec's 3-sub-project decomposition.
|
||||
|
||||
---
|
||||
|
||||
## Phase E — SP2 fused NaN check kernel (instrumentation overhead fix)
|
||||
|
||||
**Status:** SP2 implementation complete (Tasks A1-A4 on branch `plan-c-phase-2-thompson`). Gate 1 smoke pending (Task A6).
|
||||
|
||||
### Change summary
|
||||
|
||||
Replaces 8 per-step `dqn_nan_check_f32` launches in `run_nan_checks_post_backward` with a single launch of `dqn_nan_check_fused_f32_kernel` (1 kernel × 12 blocks). Slots 24-30 + 32 covered by the fused kernel; slot 31 deferred via null entry; slots 33-35 inline checks at backward orchestration phases stay separate (multi-point localization preserved).
|
||||
|
||||
### Commits
|
||||
|
||||
| Task | SHA | Scope |
|
||||
|------|-----|-------|
|
||||
| A1 | `0250722a0` | Append `dqn_nan_check_fused_f32_kernel` to `dqn_utility_kernels.cu` |
|
||||
| A2 | `82b6bd369` | Add `nan_check_buf_ptrs`/`nan_check_buf_lens` device buffer fields |
|
||||
| A3 | `fbf48df9d` | Refactor fields to `MappedU64Buffer`/`MappedI32Buffer` (eliminates HtoD); add `populate_nan_check_meta` + `launch_nan_check_fused_f32`; register kernel; call populate from constructor |
|
||||
| A4 | `b5a064f6c` | Replace 8 individual launches in `run_nan_checks_post_backward` with single fused launch; simplify signature to no-args |
|
||||
|
||||
Net delta: -109 lines (code shrinks via call-site collapse).
|
||||
|
||||
### Mapped-pinned design
|
||||
|
||||
The metadata buffers (`nan_check_buf_ptrs: MappedU64Buffer`, `nan_check_buf_lens: MappedI32Buffer`) use `cuMemHostAlloc(DEVICEMAP|PORTABLE) + cuMemHostGetDevicePointer_v2` — host-side write at construction, kernel reads via device-mapped pointer. Zero HtoD copies introduced. Per `feedback_no_htod_htoh_only_mapped_pinned`.
|
||||
|
||||
### Sticky-flag semantics preserved
|
||||
|
||||
Kernel writes `1` on NaN, never clears. Slot 24-35 firing topology should be IDENTICAL to pre-SP2 (proven by F1 step-3540 regression test that the SP1 audit characterized).
|
||||
|
||||
### Slots 33-35 inline checks (intentionally separate)
|
||||
|
||||
The fused kernel handles slots 24-30 + 32 (8 slots with concrete buffers) + slot 31 (null/deferred) + slots 33-35 (null entries — the inline checks fire at distinct backward orchestration phases for localization). Inline call sites:
|
||||
- Slot 33 (`bw_d_h_s2_post_main`): `gpu_dqn_trainer.rs:18580` (post `backward_full` + branch concat)
|
||||
- Slot 34 (`bw_d_h_s2_post_aux`): `gpu_dqn_trainer.rs:18601` (post `aux_heads_backward` SAXPY)
|
||||
- Slot 35 (`bw_d_h_s2_post_iqn`): `gpu_dqn_trainer.rs:6939` (post IQN DtoD overwrite)
|
||||
|
||||
Consolidating slot 33-35 into the fused kernel would lose the multi-point snapshot — these stay as-is.
|
||||
|
||||
### Expected validation outcome (Gate 1 smoke)
|
||||
|
||||
- F0 Best Sharpe ≥ 53.08 (return to ~55 baseline expected, vs ~35 in 3 prior smokes)
|
||||
- Slots 24-35 firing topology unchanged at F1 step ~3540 (the existing regression test)
|
||||
- No new errors / warnings
|
||||
|
||||
If Gate 1 fails, halt before SP3 starts and re-investigate F0 cause (alignment effects? stochastic variance?).
|
||||
|
||||
Reference in New Issue
Block a user