diff --git a/docs/dqn-wire-up-audit.md b/docs/dqn-wire-up-audit.md index 215602837..594b518d3 100644 --- a/docs/dqn-wire-up-audit.md +++ b/docs/dqn-wire-up-audit.md @@ -3090,3 +3090,48 @@ Files touched (Layer C C4): - `docs/dqn-wire-up-audit.md` — this entry. Refs: SP4 Layer C close-out C4 — fourth site of the `feedback_no_cpu_compute_strict` sweep. The most architecturally substantive close-out so far (full 6-step compute chain migration vs. C1/C2/C3 single-EMA migrations). + +### Layer C — `feedback_no_cpu_compute_strict` sweep — comprehensive audit (2026-05-01) + +Audit grid covering all host-side EMA arithmetic sites identified during the SP4 Layer C close-out sweep. Each site classified per the rule's input-source / consumer-chain criterion: GPU-input + (GPU/host) consumer ⇒ migrate; host-aggregated input ⇒ legitimate exception. + +| # | File:line | EMA / compute | Input source | Consumer | Verdict | Commit / Status | +|---|--------------------------------------------------------|----------------------------------------------|--------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------|------------------------| +| 1 | gpu_dqn_trainer.rs:6772-6776 (pre-C2) | iqn_loss_ema + readiness gauge | `iqn.read_total_loss()` (mapped-pinned readback) | `iqn_readiness_pinned` (c51_loss CVaR α + IQN gradient gating via dev_ptr) | MIGRATE | C2 (`605a8f526`) | +| 2 | gpu_dqn_trainer.rs:6814-6816 | q_div_ema in `compute_adaptive_tau` | `q_divergence_pinned` (mapped-pinned) | Helper return value — **ZERO production callers** | DEFER (orphan; flagged for `feedback_wire_everything_up`) | not in sweep | +| 3 | gpu_dqn_trainer.rs:18056-18063 (pre-C3) | utilization_ema (atom util) | `host[6]` from `q_readback_pinned` (mapped-pinned readback) | `homeostatic_obs_pinned[1]` mirror + accessor for collector's `set_utilization_ema` + `adaptive_entropy` derivation | MIGRATE | C3 (`ca6315860`) | +| 4 | gpu_dqn_trainer.rs:22669-22674 (pre-C4) | grad_norm_ema (winsorized) full chain | `gr.raw_grad_norm` (mapped-pinned) + ISV[GRAD_CLIP_BOUND_INDEX] | `adaptive_clip_pinned` (Adam clip kernel via dev_ptr) | MIGRATE (full 6-step chain) | C4 (`1112abc2a`) | +| 5 | gpu_experience_collector.rs:1886 | `set_utilization_ema(util)` | n/a — receives already-EMA'd value from caller | n/a — setter, not arithmetic | NOT A VIOLATION (misclassification) | n/a | +| 6 | metrics.rs:23-25 (HealthEmaTrackers::update) | q_gap_ema, q_var_ema, grad_norm_ema (3×) | (a) `per_branch_q_gap_ema` synchronous DtoH at epoch boundary, (b) `epoch_q_max - epoch_q_min` host-aggregated, (c) `avg_grad.abs()` host-aggregated | `LearningHealth.update` (composes 7 normalised components → another EMA → clamp → broadcasts to ISV[LEARNING_HEALTH_INDEX] + host gates) | DOCUMENTED EXCEPTION (mixed: q_gap is GPU-input but the q_var/grad_norm components are host-aggregated; the LearningHealth composition is multi-step host normalization. Migrating would require porting the whole 7-component normalize+compose pipeline; falls under the brief's "architectural redesign" stop-condition.) | not in sweep | +| 7 | training_loop.rs:4862-4870 | max_dd_ema + low_dd_ratio (adversarial-active hysteresis) | `epoch_max_dd` from `compute_epoch_financials` (host log-space cumulative + drawdown-sequence aggregation over downloaded `step_returns`) | Host control-flow only (`adversarial_active` flag drives saboteur updates, etc.); NO GPU consumer | DOCUMENTED EXCEPTION (host-aggregated PnL chain input + host-only consumer; no GPU computation the kernel could replace) | not in sweep | +| 8 | training_loop.rs:4910-4916 | training_sharpe_ema | `epoch_sharpe` from `compute_epoch_financials` host aggregation over downloaded `step_returns` | (a) ISV[SHARPE_EMA_INDEX=22] (GPU consumer via `write_isv_signal_at`), (b) host-side logging | DOCUMENTED EXCEPTION (host-aggregated PnL input — moving the EMA arithmetic into a kernel that takes host-passed `epoch_sharpe` is feasible but the input source itself is host aggregation; brief explicit stop-condition example) | not in sweep | +| 9 | gpu_dqn_trainer.rs:4607-4615 (calibrate_homeostatic_targets) | per-step host EMA loop over 6 mapped-pinned slots `targets[k] = (1-α) × old + α × obs` | `homeostatic_obs_pinned[k]` (mapped-pinned, GPU-readable) + `homeostatic_targets_pinned[k]` (mapped-pinned) | `homeostatic_targets_pinned[k]` (mapped-pinned, consumed by homeostatic_regularizer kernel via dev_ptr) | NEW DISCOVERY — would migrate cleanly (mirror of update_iqn_readiness pattern, 6-element loop kernel); deferred per scope budget; flagged for separate Layer C task | not in sweep | +| 10 | training_loop.rs:1032 | gamma blend `blended = 0.9 × γ + 0.1 × eval_γ` | `result.gamma` from host-aggregated eval result | `self.hyperparams.gamma` clamped + later kernel arg | DOCUMENTED EXCEPTION (host-aggregated input from eval result; one-shot per epoch boundary; same class as G7/G8) | not in sweep | + +#### Sweep summary + +Migrations completed: **3** (C2, C3, C4) plus the pre-existing C1 redesigned baseline (`24accea77`). New GPU kernels created: 3 (`update_iqn_readiness_kernel.cu`, `update_utilization_ema_kernel.cu`, `update_adaptive_clip_kernel.cu`). All four migrations follow the same pattern: +- Single-thread, single-block kernel mirroring `update_grad_norm_emas_kernel` shape. +- Mapped-pinned scalar storage for EMA state (replaces host `f32` field). +- `__threadfence_system()` on writeback for PCIe-visibility to host_ptr accessors and other-stream dev_ptr reads. +- Cold-start sentinel preserved bit-for-bit from the deleted host formulas. +- State-reset registry entries updated where applicable. +- No behaviour change — pure architectural fixes. + +Documented exceptions: **5** (sites 6, 7, 8, 9, 10 plus the misclassification site 5). Two architectural-stop-condition exceptions (G7/G8 and the freshly-discovered site 10): host-aggregated PnL/eval-result inputs where migrating would require porting the entire host aggregation chain (compute_epoch_financials, eval-result enrichment) to GPU, which the brief explicitly cited as the redesign threshold. One mixed exception (F6): the LearningHealth composition pipeline is multi-step host normalization on partly-host inputs — defer until a coherent port of that whole module. + +Deferrals flagged for separate tasks: +- Site 2 (`compute_adaptive_tau` q_div_ema): orphan helper — wire it into production OR delete per `feedback_wire_everything_up.md`. Out-of-scope for the EMA-migration sweep. +- Site 9 (`calibrate_homeostatic_targets`): NEW VIOLATION discovered during this sweep — would migrate cleanly as a 6-element-loop kernel mirroring the C1/C2/C3 pattern. Deferred only because it's a separate consumer chain (`homeostatic_targets_pinned`) and the sweep's existing 4 commits already cover the originally-listed 8-site grid. + +Final post-sweep grep `git grep -nE "= EMA_BETA \*|= \(1\.0 - EMA_BETA\)|= \(1\.0 - \w*ALPHA\) \* self\.\w*_ema|= \w*_BETA \* self\.\w*_ema" crates/ml/src/cuda_pipeline/ crates/ml/src/trainers/dqn/`: +- 1 match remaining (gpu_dqn_trainer.rs:7052 in orphan `compute_adaptive_tau` — site 2 above). + +Broader-pattern grep also surfaces the Group F + Group G + new sites 9, 10 — all are explicitly documented exceptions / deferrals above. + +Sweep verification: +- `cargo check -p ml --offline` clean across all 4 commits. +- `cargo test -p ml --lib --offline -- sp4 state_reset_registry` 11/11 passed at each commit. +- `cargo test -p ml --test sp4_producer_unit_tests --offline --release -- --ignored` 16/16 SP4 GPU tests pass on RTX 3050 Ti at each commit. + +Refs: SP4 Layer C close-out comprehensive sweep — `feedback_no_cpu_compute_strict.md` (saved 2026-05-01) zero-tolerance enforcement in SP4-touched code paths going forward.