CRITICAL ARCHITECTURAL FIX discovered while planning fused kernel:
The previous Stage 2 `add_inv_broadcast` helper in
`multi_horizon_attention.rs` was a STUB that returned Ok(()) without
doing anything. Practical consequences:
- Forward: `inv_pooled_d` (output of inverted_attention_pool.forward)
was never added into `ctx_h_d`. Downstream MoE + heads never saw
the inverted-attention signal. Axis E contributed ZERO to the
forward output and the loss.
- Backward: `inv_pool.backward` was being fed `grad_ctx_mean` as
its "upstream gradient", but that's the gradient at the CHAIN
TERMINUS — not the gradient w.r.t. inv_pooled_d (which is zero
by construction since inv_pooled wasn't in the loss). The bwd
was injecting incorrect noise into `grad_ln_out`.
Net: paying inverted_attention compute for no gain, plus polluting
ln_b's gradient. Two perf rewrite rounds earlier today showed no
wall-time movement precisely because the slow path was wired into
training while the optimized one was dead.
FIX:
(a) New kernel `cuda/inv_pooled_merge.cu`:
fwd: ctx_h[b, h, d] += inv_pooled[b, d] (broadcast over h)
bwd: grad_inv_pooled[b, d] = Σ_h grad_ctx_h[b, h, d]
Tiny — single block-per-batch, no syncthreads, coalesced reads.
(b) Host binding `src/inv_pooled_merge.rs` (InvPooledMerge).
(c) `MultiHorizonAttention` adds:
- `merge: InvPooledMerge` field.
- `grad_inv_pooled_d [B, H]` buffer for the real upstream of inv_pool.bwd.
(d) `MultiHorizonAttention.forward` now calls `merge.forward(...)`
between inv_pool.forward and moe.forward. Axis E is now actually
in the model's forward output.
(e) `MultiHorizonAttention.backward` now calls `merge.backward` after
moe.bwd writes `grad_ctx_h_d`, producing `grad_inv_pooled_d`.
`inv_pool.backward` consumes the REAL upstream gradient
(`grad_inv_pooled_d`) instead of the prior `grad_ctx_mean` fake.
(f) Old stub `add_inv_broadcast` deleted.
CORRECTNESS:
- perception_overfit 9/9 PASS after the wiring. Loss still shrinks
on the constant-signal test (0.67 → -0.99 over 250 steps), now
with axis E actually contributing.
- All numgrad kernel tests still pass (kernels themselves unchanged
in this commit; only the wiring).
PERF IMPACT (expected):
- +2 tiny launches per step (merge fwd + bwd). Negligible.
- The inverted-attention compute that was previously dead now
actually feeds the loss → same wall-time, but it's earning the
cost. This unblocks meaningful axis-E perf measurement on next
smoke.
NEXT: re-run cluster smoke to confirm wall-time + verify axis E
gradient flow is healthy. After that, consider full MHA forward
fusion as a follow-up.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
4ae9a27f48