Files
foxhunt/crates/ml/examples
jgrusewski 2feeeda8bb fix(phase-e-4-a): GPU kernel for h_enriched slot copy — eliminate per-step CPU roundtrip
The Phase E.4.A T8 wiring stored Mamba2's per-step cache.h_enriched
into h_enriched_buf_dev via a dtoh+htod sequence:

  let h_host = stream.clone_dtoh(cache.h_enriched.cuda_data())?;
  let mut buf_host = stream.clone_dtoh(&h_enriched_buf_dev)?;  // <- whole buffer
  for j in 0..hidden_dim { buf_host[slot_offset + j] = h_host[j]; }
  stream.memcpy_htod(&buf_host, &mut h_enriched_buf_dev)?;     // <- whole buffer

This violates feedback_cpu_is_read_only AND
feedback_no_htod_htoh_only_mapped_pinned. Worse, the buffer-wide
dtoh+htod every step is ~20K floats × 600 steps × 500 eps × 30 cells
= ~9M roundtrips totaling significant PCIe latency in the backtest.

Fix: new tiny CUDA kernel alpha_h_enriched_store_kernel in
alpha_window_push.cu (one thread per hidden-dim feature, writes
src[j] → buf[slot_offset + j]). Replaces the dtoh/htod sequence
in both smoke and backtest binaries.

Estimated speed-up at backtest scale: 3-6× on the temporal eval
path. Pure-GPU per-step inference restored — no synchronisation
points on the hot path.

docs/isv-slots.md updated per kernel-audit-doc hook.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-15 22:20:28 +02:00
..