diff --git a/docs/dqn-wire-up-audit.md b/docs/dqn-wire-up-audit.md index 004a9bc99..d075ee053 100644 --- a/docs/dqn-wire-up-audit.md +++ b/docs/dqn-wire-up-audit.md @@ -17291,3 +17291,31 @@ Workflow: `train-qsltr` @ commit `5106e3b117e89ae5a7e6f0de9f09ed0fe600ad87`, 3 e **Diagnostic to add for next smoke** (deliberately deferred this run): - W[0..4] direct readback per epoch via mapped pinned [4] f32 + DtoD copy outside captured graph + HEALTH_DIAG log line. Would directly show Adam updates on W. + +#### Smoke scope clarification — α ACTIVE, β DEAD (2026-05-13) + +While preparing the verdict template, audited the β reward shaping path's producer chain. Finding: + +**β producer wired but UNFED**: `experience_kernels.cu:3614-3628` correctly computes `r_aux_align = scale_β × alignment × profit_pos` and writes to `reward_components_per_sample[out_off * 7 + 6]`. But `scale_β = isv_signals[SP22_AUX_ALIGN_SCALE_INDEX=537]` and slot 537 has NO PRODUCER — the SP11 `reward_subsystem_controller_kernel.cu` still treats `N_COMPONENTS = 6.0f` (popart/cf/trail/micro/opp_cost/bonus). Phase B6 (extend SP11 controller to 7 components with w_aux_align anchor) hasn't landed. + +**Net effect**: slot 537 stays at alloc_zeros sentinel 0 throughout training → `r_aux_align = 0` always → β reward shaping is a runtime no-op. + +**Implication for smoke verdict**: This smoke validates the **α atom-shift mechanism ONLY**. The β reward bonus (aux-aligned profitable trades earn extra reward) is dormant. If WR shifts off baseline, α is the cause; if it doesn't, the verdict is α alone is insufficient — β extension (Phase B6) becomes the next priority before investing in Phase D. + +**Phase B6 scope** (deferred): Extend `reward_subsystem_controller_kernel.cu`: +1. `N_COMPONENTS: 6.0 → 7.0`. +2. Read 7th component magnitude from `ISV[REWARD_AUX_ALIGN_EMA_INDEX=536]` (EMA already produced by reward_component_ema_kernel.cu's extended 0..7 loop — Phase B3 landed). +3. Compute w_aux_align via the same blend formula as other components (winner × improving + diversifier × stagnant); anchor target = ~10% of total reward magnitude; bounds `[0.05, 2.0]`. +4. Write to `ISV[SP22_AUX_ALIGN_SCALE_INDEX=537]`. + +Estimated ~1-2 hr engineering (single-kernel edit + launcher update if N_COMPONENTS is kernel-arg vs constant). + +**Updated outcome table** (replacing "Possible outcomes" in the verdict template): + +| Pattern | Interpretation | Next action | +|---------|----------------|-------------| +| WR ≥ 53% epoch 3, dist shifts | α atom-shift alone moves WR | Land B6 for β + Phase D for eval validation | +| WR 50-53%, dist shifts | α partial effect | Tune W magnitudes OR land B6 to add β reward signal | +| WR ~46%, dist shifts | Mechanism active but no WR translation | β probably necessary — land B6 next | +| WR ~46%, no dist shift | α inactive (state_121 not flowing) | Investigate aux head + state_121 wiring | +| Training collapse | Loss landscape unstable | Reduce W init magnitude or revert |