gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

audit(dqn-v2): h_s2 audit anchor corrections (post-2c.3+4 reconnaissance)

Browse Source 
The 2c.3+4 dispatch agent surfaced two stale anchors in the
2026-04-25 h_s2 consumer audit:

1. Row 11 (IQN target trunk): the kernel referenced as
   `iqn_compute_target_h_s2` in `iqn_dual_head_kernel.cu:1031` is
   actually `iqn_trunk_forward_kernel`, and it is ALREADY ORPHANED
   (loaded into IqnHead::trunk_forward_kernel but never invoked).
   The real IQN target trunk runs through cuBLAS iqn_lt_matmul calls
   in gpu_iqn_head.rs::execute_training_pipeline:820-846 (with a
   cached fast-path at 803-847). Mitigation reduced to: delete dead
   kernel + extract target_encoder_forward_only + replace cuBLAS
   fallback path with that extraction.

2. Row 22 (relu_mask sites): line numbers drifted. Audit said
   5581/5798; current code is 5655/5697/5872. Crucially, 5697 is
   the h_s1 mask, not h_s2/IQN-aux — the audit's "3 sites" claim
   needs verification per site before editing. Updated to the
   current line numbers with a "verify before editing" note.

These are documentation-only corrections. The mitigation strategy
(GRN backward chain replaces relu_mask; target_encoder_forward_only
unifies trunk implementation) is unchanged in spirit.
This commit is contained in:
jgrusewski
2026-04-25 12:29:47 +02:00
parent b7f941f7fa
commit 4402dbaf3d
1 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
docs/h_s2_consumers_audit.md
View File

@@ -35,7 +35,7 @@ can be negative.
| 8 | Multi-head feature attention (post-graph): `attn_in = h_s2`, output overwrites `save_h_s2` | `gpu_dqn_trainer.rs:6712-6743` (`apply_attention_forward`); CUDA kernels `attention_backward_kernel.cu:140-167` (`attn_sdp_fwd`) and `attention_kernel.cu` | Linear Q/K/V projections (sign-agnostic), feature-level attention with sigmoid gate `1/(1+exp(-Q·K/√d))`, output GEMM, residual `state + projected`, LayerNorm. The `attn_concat_input` kernel that builds the [D,B] attention input is also a plain copy of states and OFI — no arithmetic on `h_s2`. The init comment says "Zero W_O makes attention start as identity: output = LN(h_s2 + 0 @ attn) ≈ h_s2" — this property holds for signed `h_s2` because LN of a signed vector is well-defined. | safe-with-signed | None. Attention's existing residual+LayerNorm idempotently absorbs signed input. |
| 9 | IQN dual-head forward (per-quantile): `combined = h_s2 ⊙ embed`, `combined ⊙ relu_embed` | `iqn_dual_head_kernel.cu:191`, `:236-274`, `:301-310` (forward) and `:411` (target path); launched from `gpu_iqn_head.rs::execute_training_pipeline`. The element-wise product is `h_s2 ⊙ ReLU(W_embed·cos(τ))`. | Element-wise product of `h_s2` with the ReLU'd embedding. ReLU makes the embedding non-negative, but multiplying by signed `h_s2` is mathematically fine — `combined` is signed, fed to a per-branch linear `q = W_b · combined + b_b`. No log/sqrt/sigmoid is applied to `h_s2` itself. Backward (`iqn_dual_head_kernel.cu:684`) computes `d_pre = dq · w_sum · h_s2 · (embed_pre > 0)` — multiplies by `h_s2`, fine when signed. | safe-with-signed | None. Note that the kernel today ingests `h_s2` as `float*` even though the header comment says "(bf16 from DQN trunk)" — that's a stale comment; the actual storage is f32 already. |
| 10 | IQN tile + Hadamard-sigmoid path: `out = h_s2_tiled · sigmoid(embed)` | `iqn_dual_head_kernel.cu:1262` (`iqn_tile_hadamard_sigmoid`), `:1284` (backward `d_h_s2_tiled = d_combined · sigmoid(embed)`); tile/reduce kernels at `:1361-1396` | Element-wise product of `h_s2_tiled` with `sigmoid(embed_pre) ∈ (0,1)`. Sign of `h_s2` propagates linearly through the sigmoid-gated multiply. Backward is symmetric. | safe-with-signed | None. |
| 11 | IQN target trunk recompute kernel `iqn_compute_target_h_s2` | `iqn_dual_head_kernel.cu:1031-1092` (`state -> LeakyReLU·W_s1 -> LeakyReLU·W_s2 -> h_s2`) writing to `h_s2_out` for the target network | This kernel **reproduces** the legacy ReLU/LeakyReLU trunk to compute *target* `h_s2`. Once Task 2c lands, the *online* trunk uses GRN but this target-recompute kernel still applies LeakyReLU, so it diverges from the online network. The kernel is itself sign-agnostic on its inputs, but its output won't match the new GRN trunk. | assumes-non-negative *(by reproduction of legacy trunk)* | This kernel must be migrated to the new GRN topology in the same commit as Task 2c, or replaced by a `forward_target_raw` call that uses the unified Rust trunk path. Otherwise online and target diverge. |
| 11 | IQN target trunk — `iqn_trunk_forward_kernel` (audit-orig name `iqn_compute_target_h_s2`) | `iqn_dual_head_kernel.cu` kernel function. **CORRECTION (2026-04-25 post-2c.3+4 recon):** the kernel is **already orphaned** — loaded into `IqnHead::trunk_forward_kernel` but never invoked. The actual IQN target trunk runs through cuBLAS `iqn_lt_matmul` calls in `gpu_iqn_head.rs::execute_training_pipeline` lines 803-847 (cached fast-path) and 820-846 (cuBLAS fallback). | The cuBLAS fallback path reproduces the legacy `Linear → ReLU → Linear → ReLU` via `launch_trunk_bias_relu`. Once GRN lands, online and target diverge unless the cuBLAS path is replaced with a `target_encoder_forward_only` call (Task 4-style extraction on the target-net path). | **assumes-non-negative** *(via cuBLAS path's `launch_trunk_bias_relu`)* | (a) Delete the dead `iqn_trunk_forward_kernel` + its `load(...)` line + struct field. (b) Extract `target_encoder_forward_only` mirroring Task 4's online encoder split. (c) Replace the cuBLAS fallback path in `execute_training_pipeline:820-846` with `target_encoder_forward_only`. The cached fast-path stays valid as long as `cached_target_h_s2_ptr` is updated by whoever populates it. |
| 12 | Risk-budget head: SH2 cooperative load, hidden = ReLU(W_risk_fc · [h_s2; ISV; pred_err] + b) | `experience_kernels.cu:5867-5917` (`risk_budget_forward`), launched at `gpu_dqn_trainer.rs:4592-4626` | Concats `h_s2[B,SH2]` with `ISV[12]` and `predicted_error[1]`, runs linear + ReLU + sigmoid. Backward (`experience_kernels.cu:5954-6037`) reads `h_s2[i,k]` as `input_val` for `dW`. | safe-with-signed | None. |
| 13 | Recursive confidence head: `pred = sigmoid(W·h_s2 + b)` | `experience_kernels.cu:6447-6462` (forward) + `:6476-6543` (backward); Rust at `gpu_dqn_trainer.rs:4001-4027`/`:4443-4496` | Linear projection through sigmoid. Backward writes `d_h_s2 += d_sigmoid · w_conf` (signed contribution) and `dW += d_sigmoid · h_s2`. Both are sign-agnostic. | safe-with-signed | None. |
| 14 | Selectivity head: `sel = sigmoid(W_sel · h_s2 + b_sel)` | `experience_kernels.cu:3998-4014` (forward), `:4056-4068` (dW reduce) | Plain linear projection through sigmoid. Backward `dW += d_z · h_s2[b,k]` is sign-agnostic. | safe-with-signed | None. |
@@ -46,7 +46,7 @@ can be negative.
| 19 | Predictive coding loss: `L = λ · mean_j(h_s2[i] − h_s2[i+1])²` | `experience_kernels.cu:5713-5736` (forward), `:5766-5800` (backward); launched at `gpu_dqn_trainer.rs:4781-4827` | Squared difference between consecutive samples; output is non-negative regardless of input sign. Backward computes `dL/dh = 2λ/SH2 · (h_self − h_neighbour)`, also sign-symmetric. | safe-with-signed | None. |
| 20 | Mag-concat backward `strided_accumulate` extracts `d_h_s2 += d_mag_concat[:SH2]` | `experience_kernels.cu:3708-3726`; called at `gpu_dqn_trainer.rs:4831+` (`accumulate_d_h_s2_from_concat`) | Pure copy/accumulate; no nonlinearity. | safe-with-signed | None. |
| 21 | Branch-decoder VSN+GLU `launch_vsn_glu_branch` (per branch path) | `batched_forward.rs:1143-1306` calls `vsn_kernel`, then `kan_gate_combine_kernel` (KAN spline gate). The first stage is the VSN above (#5). | The KAN spline gate `kan_gate_combine` in this path operates on the *bottleneck* output of a per-branch FC, not on `h_s2` directly. Its input is `b1_proj` (after the per-branch linear+ReLU). KAN doesn't touch `h_s2`. | safe-with-signed (no direct read) | None — KAN sees only post-FC bottleneck features. |
| 22 | **Trunk-backward ReLU mask: `d_h_s2 *= (save_h_s2 > 0)`** | `batched_backward.rs:1778` (`relu_mask`), `gpu_dqn_trainer.rs:5581-5599` (IQN aux trunk grad), `gpu_dqn_trainer.rs:5798-5801`. Kernels: `relu_mask_kernel.cu` and `backward_kernels.cu:18`. | This is the **load-bearing ReLU assumption** in the entire pipeline. The mask zeros gradients where `save_h_s2 <= 0`, which is correct mathematics for `h = ReLU(z)`. With LayerNorm output (zero-mean, signed), this would zero ~50% of gradients corresponding to **valid** negative activations — a silently wrong gradient with no NaN/Inf. | **assumes-non-negative** | The `relu_mask` step **must be removed** (or replaced with the GRN's actual jacobian) on the post-Task-2c trunk-out. LayerNorm has its own backward (already implemented in `attn_layer_norm_bwd_*` family); the GRN block's ELU+GLU+LN backward must replace the bare `relu_mask` call here. This is the single hardest mitigation. |
| 22 | **Trunk-backward ReLU mask: `d_h_s2 *= (save_h_s2 > 0)`** | `batched_backward.rs:1778` (`relu_mask`), `gpu_dqn_trainer.rs:5655` (online trunk grad post-h_s2), `gpu_dqn_trainer.rs:5697` (h_s1 mask, NOT h_s2 — verify before editing), `gpu_dqn_trainer.rs:5872` (target-sync per-target-update). Kernels: `relu_mask_kernel.cu` and `backward_kernels.cu:18`. **Line numbers updated 2026-04-25 post-recon — original audit had stale 5581/5798 anchors.** | This is the **load-bearing ReLU assumption** in the entire pipeline. The mask zeros gradients where `save_h_s2 <= 0`, which is correct mathematics for `h = ReLU(z)`. With LayerNorm output (zero-mean, signed), this would zero ~50% of gradients corresponding to **valid** negative activations — a silently wrong gradient with no NaN/Inf. | **assumes-non-negative** | The `relu_mask` step **must be removed** (or replaced with the GRN's actual jacobian) on the post-Task-2c trunk-out. The 2c.1 GRN kernel module already provides the full LN+GLU+ELU backward; the 2c.2 wrapper sequences it. Each of the 3 sites has different save-buffer lifetime semantics (online per-step, IQN-aux per-step in different trainer, target-sync per-target-update) — verify save-buffer plumbing per site. |
| 23 | Target-network trunk forward: `tg_h_s2_buf` is produced/consumed by the same path as #1, #2, #4 | `batched_forward.rs:980-1090` (`forward_target_raw`); IQN reuses `tg_h_s2_buf` via `gpu_iqn_head.rs::set_cached_target_h_s2_ptr` → `cached_target_h_s2_ptr` | Same arithmetic as the online trunk: linear + ReLU(legacy) → branches via linear+ReLU. After Task 2c, target trunk MUST run the same GRN block as the online trunk. | **assumes-non-negative** *(via shared kernel reuse)* | Target trunk weight set must be migrated alongside online. The relu_mask in `apply_iqn_trunk_gradient` (`gpu_dqn_trainer.rs:5581`) currently masks against `save_h_s2`, the **online** trunk activation; once that buffer is LN-output the mask is wrong (see #22). |
| 24 | Experience-collector `exp_h_s2_f32` storage | `gpu_experience_collector.rs:559,690,1231-1299,3306` | Plain f32 storage of past `h_s2` for replay/episode-level data. Read back by replay/IQN as ordinary tensor data. No nonlinearity at the storage layer. | safe-with-signed | None — storage layout is sign-agnostic. |
| 25 | Q-stats / monitoring path (`tg_h_s2_ptr` exposed for monitoring) | `gpu_dqn_trainer.rs:4936-4938` accessor | The pointer is exposed but the call sites are restricted to monitoring readback. Monitoring kernels we audited do not perform sign-dependent ops on `h_s2`. | safe-with-signed | None. |