diff --git a/docs/dqn-wire-up-audit.md b/docs/dqn-wire-up-audit.md index 8e4950e0a..edd7ce00c 100644 --- a/docs/dqn-wire-up-audit.md +++ b/docs/dqn-wire-up-audit.md @@ -3822,19 +3822,19 @@ Audit grid covering all host-side EMA arithmetic sites identified during the SP4 | # | File:line | EMA / compute | Input source | Consumer | Verdict | Commit / Status | |---|--------------------------------------------------------|----------------------------------------------|--------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------|------------------------| | 1 | gpu_dqn_trainer.rs:6772-6776 (pre-C2) | iqn_loss_ema + readiness gauge | `iqn.read_total_loss()` (mapped-pinned readback) | `iqn_readiness_pinned` (c51_loss CVaR α + IQN gradient gating via dev_ptr) | MIGRATE | C2 (`605a8f526`) | -| 2 | gpu_dqn_trainer.rs:6814-6816 | q_div_ema in `compute_adaptive_tau` | `q_divergence_pinned` (mapped-pinned) | Helper return value — **ZERO production callers** | DEFER (orphan; flagged for `feedback_wire_everything_up`) | not in sweep | +| 2 | gpu_dqn_trainer.rs:6814-6816 (deleted) | q_div_ema in `compute_adaptive_tau` | `q_divergence_pinned` (mapped-pinned) | Helper return value — **ZERO production callers** | CLOSED — orphan deleted per `feedback_wire_everything_up` (close-out 2026-05-01) | `a385c1d2b` (SP4 close-out follow-up) | | 3 | gpu_dqn_trainer.rs:18056-18063 (pre-C3) | utilization_ema (atom util) | `host[6]` from `q_readback_pinned` (mapped-pinned readback) | `homeostatic_obs_pinned[1]` mirror + accessor for collector's `set_utilization_ema` + `adaptive_entropy` derivation | MIGRATE | C3 (`ca6315860`) | | 4 | gpu_dqn_trainer.rs:22669-22674 (pre-C4) | grad_norm_ema (winsorized) full chain | `gr.raw_grad_norm` (mapped-pinned) + ISV[GRAD_CLIP_BOUND_INDEX] | `adaptive_clip_pinned` (Adam clip kernel via dev_ptr) | MIGRATE (full 6-step chain) | C4 (`1112abc2a`) | | 5 | gpu_experience_collector.rs:1886 | `set_utilization_ema(util)` | n/a — receives already-EMA'd value from caller | n/a — setter, not arithmetic | NOT A VIOLATION (misclassification) | n/a | | 6 | metrics.rs:23-25 (HealthEmaTrackers::update) | q_gap_ema, q_var_ema, grad_norm_ema (3×) | (a) `per_branch_q_gap_ema` synchronous DtoH at epoch boundary, (b) `epoch_q_max - epoch_q_min` host-aggregated, (c) `avg_grad.abs()` host-aggregated | `LearningHealth.update` (composes 7 normalised components → another EMA → clamp → broadcasts to ISV[LEARNING_HEALTH_INDEX] + host gates) | DOCUMENTED EXCEPTION (mixed: q_gap is GPU-input but the q_var/grad_norm components are host-aggregated; the LearningHealth composition is multi-step host normalization. Migrating would require porting the whole 7-component normalize+compose pipeline; falls under the brief's "architectural redesign" stop-condition.) | not in sweep | | 7 | training_loop.rs:4862-4870 | max_dd_ema + low_dd_ratio (adversarial-active hysteresis) | `epoch_max_dd` from `compute_epoch_financials` (host log-space cumulative + drawdown-sequence aggregation over downloaded `step_returns`) | Host control-flow only (`adversarial_active` flag drives saboteur updates, etc.); NO GPU consumer | DOCUMENTED EXCEPTION (host-aggregated PnL chain input + host-only consumer; no GPU computation the kernel could replace) | not in sweep | | 8 | training_loop.rs:4910-4916 | training_sharpe_ema | `epoch_sharpe` from `compute_epoch_financials` host aggregation over downloaded `step_returns` | (a) ISV[SHARPE_EMA_INDEX=22] (GPU consumer via `write_isv_signal_at`), (b) host-side logging | DOCUMENTED EXCEPTION (host-aggregated PnL input — moving the EMA arithmetic into a kernel that takes host-passed `epoch_sharpe` is feasible but the input source itself is host aggregation; brief explicit stop-condition example) | not in sweep | -| 9 | gpu_dqn_trainer.rs:4607-4615 (calibrate_homeostatic_targets) | per-step host EMA loop over 6 mapped-pinned slots `targets[k] = (1-α) × old + α × obs` | `homeostatic_obs_pinned[k]` (mapped-pinned, GPU-readable) + `homeostatic_targets_pinned[k]` (mapped-pinned) | `homeostatic_targets_pinned[k]` (mapped-pinned, consumed by homeostatic_regularizer kernel via dev_ptr) | NEW DISCOVERY — would migrate cleanly (mirror of update_iqn_readiness pattern, 6-element loop kernel); deferred per scope budget; flagged for separate Layer C task | not in sweep | +| 9 | gpu_dqn_trainer.rs:7958-7994 (GPU launcher) + `calibrate_homeostatic_kernel.cu` | per-step host EMA loop migrated to GPU kernel: `targets[k] = (1-α) × old + α × obs` for k∈{1..5}, `targets[0]=0` invariant | `homeostatic_obs_pinned[k]` (mapped-pinned, GPU-readable) + `homeostatic_targets_pinned[k]` (mapped-pinned) | `homeostatic_targets_pinned[k]` (mapped-pinned, consumed by homeostatic_regularizer kernel via dev_ptr) | CLOSED — GPU-ported per `feedback_no_cpu_compute_strict` via 6-thread single-block kernel mirroring C1/C2/C3 pattern (close-out 2026-05-01) | `6d0ac7beb` (SP4 close-out follow-up) | | 10 | training_loop.rs:1032 | gamma blend `blended = 0.9 × γ + 0.1 × eval_γ` | `result.gamma` from host-aggregated eval result | `self.hyperparams.gamma` clamped + later kernel arg | DOCUMENTED EXCEPTION (host-aggregated input from eval result; one-shot per epoch boundary; same class as G7/G8) | not in sweep | #### Sweep summary -Migrations completed: **3** (C2, C3, C4) plus the pre-existing C1 redesigned baseline (`24accea77`). New GPU kernels created: 3 (`update_iqn_readiness_kernel.cu`, `update_utilization_ema_kernel.cu`, `update_adaptive_clip_kernel.cu`). All four migrations follow the same pattern: +Migrations completed: **4** (C2, C3, C4, plus close-out site #9) plus the pre-existing C1 redesigned baseline (`24accea77`). New GPU kernels created: 4 (`update_iqn_readiness_kernel.cu`, `update_utilization_ema_kernel.cu`, `update_adaptive_clip_kernel.cu`, `calibrate_homeostatic_kernel.cu`). Orphan helpers deleted: 1 (site #2 `compute_adaptive_tau` per close-out follow-up). All migrations follow the same pattern: - Single-thread, single-block kernel mirroring `update_grad_norm_emas_kernel` shape. - Mapped-pinned scalar storage for EMA state (replaces host `f32` field). - `__threadfence_system()` on writeback for PCIe-visibility to host_ptr accessors and other-stream dev_ptr reads. @@ -3842,14 +3842,12 @@ Migrations completed: **3** (C2, C3, C4) plus the pre-existing C1 redesigned bas - State-reset registry entries updated where applicable. - No behaviour change — pure architectural fixes. -Documented exceptions: **5** (sites 6, 7, 8, 9, 10 plus the misclassification site 5). Two architectural-stop-condition exceptions (G7/G8 and the freshly-discovered site 10): host-aggregated PnL/eval-result inputs where migrating would require porting the entire host aggregation chain (compute_epoch_financials, eval-result enrichment) to GPU, which the brief explicitly cited as the redesign threshold. One mixed exception (F6): the LearningHealth composition pipeline is multi-step host normalization on partly-host inputs — defer until a coherent port of that whole module. +Documented exceptions: **4** (sites 6, 7, 8, 10 plus the misclassification site 5). Two architectural-stop-condition exceptions (G7/G8 and the freshly-discovered site 10): host-aggregated PnL/eval-result inputs where migrating would require porting the entire host aggregation chain (compute_epoch_financials, eval-result enrichment) to GPU, which the brief explicitly cited as the redesign threshold. One mixed exception (F6): the LearningHealth composition pipeline is multi-step host normalization on partly-host inputs — defer until a coherent port of that whole module. -Deferrals flagged for separate tasks: -- Site 2 (`compute_adaptive_tau` q_div_ema): orphan helper — wire it into production OR delete per `feedback_wire_everything_up.md`. Out-of-scope for the EMA-migration sweep. -- Site 9 (`calibrate_homeostatic_targets`): NEW VIOLATION discovered during this sweep — would migrate cleanly as a 6-element-loop kernel mirroring the C1/C2/C3 pattern. Deferred only because it's a separate consumer chain (`homeostatic_targets_pinned`) and the sweep's existing 4 commits already cover the originally-listed 8-site grid. +Deferrals: **0**. Sites 2 and 9 were resolved in the 2026-05-01 close-out follow-up (commits `a385c1d2b` and `6d0ac7beb` respectively). See "Sweep close-out" subsection below. Final post-sweep grep `git grep -nE "= EMA_BETA \*|= \(1\.0 - EMA_BETA\)|= \(1\.0 - \w*ALPHA\) \* self\.\w*_ema|= \w*_BETA \* self\.\w*_ema" crates/ml/src/cuda_pipeline/ crates/ml/src/trainers/dqn/`: -- 1 match remaining (gpu_dqn_trainer.rs:7052 in orphan `compute_adaptive_tau` — site 2 above). +- 0 matches remaining post-close-out (site #2 `compute_adaptive_tau` deleted in `a385c1d2b`). Broader-pattern grep also surfaces the Group F + Group G + new sites 9, 10 — all are explicitly documented exceptions / deferrals above.