fix(rl): mapped-pinned for all R7d/R8 CPU↔GPU paths + diag JSONL + guard
Two concerns in one commit since they're entangled:
## 1. feedback_no_htod_htoh_only_mapped_pinned violations
R7d (PER push/sample) + R8 (CLI binary) + the new per-step diag dump
shipped with 8 raw `stream.memcpy_htod` / `stream.memcpy_dtoh` calls.
The rule is explicit: "mapped-pinned only for CPU↔GPU; tests not
exempt." A raw `stream.memcpy_*` on a regular `&[T]` / `&mut [T]` is
NOT mapped-pinned — the source/dest slice isn't page-locked, so the
CUDA driver does an internal blocking HtoD/DtoH that stalls the
stream.
Refactored all 8 violations to use the mapped-pinned + DtoD pattern
(cuMemHostAlloc DEVICEMAP — host writes via `host_ptr`, kernel reads
`dev_ptr`, DtoD between them via `cudarc::driver::result::memcpy_dtod_async`).
New shared helpers in `trainer/integrated.rs`:
* `read_slice_i32_d` — DtoH for `i32` device buffers via
`MappedI32Buffer` staging. Counterpart to the existing
`read_slice_d` (f32 version).
* `write_slice_f32_d` — CPU→GPU upload for `f32` via
`MappedF32Buffer.write_from_slice` + DtoD into destination.
* `write_slice_i32_d` — CPU→GPU upload for `i32` via
`MappedI32Buffer.host_slice_mut().copy_from_slice` + DtoD.
`pub fn` wrappers (`read_slice_*_d_pub`) expose the f32/i32 helpers
to the CLI binary so the per-step diag DtoH uses the same canonical
pattern.
Call-site refactors:
* `push_to_replay`: 4× `stream.memcpy_dtoh` → `read_slice_*_d`.
* `sample_and_gather`: 3× `stream.memcpy_htod` → `write_slice_*_d`.
* `step_with_lobsim` pre-PER ISV refresh: raw `memcpy_dtoh` →
`read_slice_d` (424 floats per step).
* `step_with_lobsim` post-Q PER priority TD readback: raw
`memcpy_dtoh` → `read_slice_d` (b_size floats per step).
* `step_synthetic` ISV mirror refresh: raw `memcpy_dtoh` →
`read_slice_d` (pre-existing pre-R9 violation; fixed in the
same commit since it's the same pattern in the same file).
* Init-time (one-shot) `prng_state` upload: raw `memcpy_htod` →
inline mapped-pinned DtoD (custom because cast through i32 for
the u32 buffer).
* Init-time (one-shot) `atom_supports` upload: raw `memcpy_htod`
→ `write_slice_f32_d`.
* `examples/alpha_rl_train.rs` per-step diag DtoH (3 calls) →
`read_slice_*_d_pub`.
## 2. Pre-commit guard gap — diff-aware HtoD/DtoH check
The existing GPU hot-path guard (`scripts/gpu-hotpath-guard.sh`)
EXPLICITLY skips memcpy_htod/dtoh on the assumption that such calls
only appear in `cuda_pipeline/` (where mapped-pinned is the
convention). That assumption was falsified by R7d/R8 — the guard
shipped 8 violations green.
Added `check_no_raw_htod_dtoh` to `scripts/pre-commit-hook.sh` (the
real file behind the `.git/hooks/pre-commit` symlink). The check is
DIFF-AWARE: it greps only the `+` lines of `git diff --cached -U0`,
so pre-existing violations elsewhere (143 sites across the codebase)
don't block commits touching unrelated files. NEW additions of
`\.memcpy_(htod|dtoh)\(` are flagged with a clear error pointing at
the mapped-pinned alternative. Suppress per-line with `// gpu-ok:
<reason>` (same convention as the existing guards).
Pre-existing violations in `ml-alpha/src/aux_heads.rs`,
`mamba2_block.rs`, `cfc/`, `data/`, etc. are a separate cleanup —
not blocked by this commit's check because the diff-aware filter
ignores anything that was already on `HEAD~1`.
## Verified gates (post-fix, local sm_86)
G1 isv_bootstrap ✅ unchanged
G3 controllers_emit ✅ unchanged
G4 target_soft_update ✅ unchanged
G6 r7d_per_wiring ✅ unchanged (PER round-trips all
mapped-pinned now)
R3 ema/advantage (3 tests) ✅ unchanged
R4 action kernels (3 tests) ✅ unchanged
end integrated_trainer_smoke ✅ unchanged
Mapped-pinned is semantically equivalent to raw memcpy_htod/dtoh —
just routed through page-locked staging so the driver doesn't have
to do its own internal pinning. Behaviour identical; the cost shifts
from "driver hidden HtoD per call" to "mapped-pinned alloc + DtoD
per call." For the smoke (b_size=1, 1000 steps), the cost difference
is in the microseconds.
## Per-step diag JSONL (separate concern, same commit)
Added `--diag-jsonl <PATH>` flag to `alpha_rl_train.rs` (default:
`<out>/diag.jsonl`). After each `step_with_lobsim`, writes one JSON
record capturing:
* step number, elapsed wall time
* all 5 head losses + λs
* all 7 RL controller outputs (γ τ ε coef n_roll per_α scale)
* all 5 per-head learning rates (lr_bce/q/pi/v/aux)
* all 7 EMA inputs the controllers consume
* replay buffer length
* per-step reward stats (sum, max, min, abs_max)
* per-step done count
* per-step action histogram (9 action classes)
Critical for cluster smoke debugging — the prior CLI only flushed
an `eprintln` progress line every N steps (default 100), making
in-flight controller drift / replay stagnation / reward explosion
invisible until they produced a NaN abort. The JSONL is line-
buffered + flushed every `log_every` steps so `tail -f` shows
progress live.
The stderr progress line is also beefed up to include γ / ε / per_α /
reward_scale / dones / rew_sum at each tick so a casual `argo logs`
inspection sees the controller behaviour without parsing JSONL.
## Why R9 cluster submission needs this
Without the diag dump, an R9 1000-step smoke is "blind" — only the
final summary tells us what happened. With the dump, post-hoc
analysis can answer:
* Did the controllers adapt or stay at bootstrap?
* Did the reward scale stabilise or saturate?
* Did the PER buffer fill?
* Was the action histogram dominated by any one action?
* Where did the per-head losses converge to?
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -227,10 +227,67 @@ check_no_isv_migrations() {
|
||||
fi
|
||||
}
|
||||
|
||||
# Diff-aware guard: any NEW `stream.memcpy_htod(` or `stream.memcpy_dtoh(`
|
||||
# line added in this commit is a violation of
|
||||
# `feedback_no_htod_htoh_only_mapped_pinned`. The only permitted CPU↔GPU
|
||||
# path is mapped-pinned staging (cuMemHostAlloc DEVICEMAP — host writes
|
||||
# via `host_ptr`, kernels read `dev_ptr`, DtoD copy between them via
|
||||
# `cudarc::driver::result::memcpy_dtod_async`).
|
||||
#
|
||||
# The existing gpu-hotpath-guard.sh explicitly skips memcpy_htod/dtoh
|
||||
# (see scripts/gpu-hotpath-guard.sh:103-105) on the assumption that
|
||||
# such calls only appear in cuda_pipeline/ where mapped-pinned is the
|
||||
# convention. That assumption was falsified in 2026-05-23 when the R7d
|
||||
# PER push/sample + R8 CLI diag landed 8 raw `stream.memcpy_htod/dtoh`
|
||||
# calls in ml-alpha/src/trainer/ and ml-alpha/examples/ — the guard
|
||||
# was deliberately blind to them and they shipped.
|
||||
#
|
||||
# This diff-aware check fires on NEW + lines only, so pre-existing
|
||||
# violations elsewhere don't block commits touching unrelated files.
|
||||
# Suppress per-line with `// gpu-ok: <reason>`.
|
||||
check_no_raw_htod_dtoh() {
|
||||
local staged
|
||||
staged=$(git diff --cached --name-only --diff-filter=ACM | grep '\.rs$' || true)
|
||||
if [ -z "$staged" ]; then return 0; fi
|
||||
|
||||
local bad=""
|
||||
while IFS= read -r f; do
|
||||
[ -z "$f" ] && continue
|
||||
local hits
|
||||
hits=$(git diff --cached -U0 -- "$f" 2>/dev/null \
|
||||
| grep -E '^\+[^+]' \
|
||||
| grep -E '\.memcpy_(htod|dtoh)\(' \
|
||||
| grep -v 'gpu-ok:' \
|
||||
|| true)
|
||||
if [ -n "$hits" ]; then
|
||||
bad+="${bad:+$'\n'}$f:"$'\n'"$hits"
|
||||
fi
|
||||
done <<< "$staged"
|
||||
|
||||
if [ -n "$bad" ]; then
|
||||
echo "❌ feedback_no_htod_htoh_only_mapped_pinned violation: raw"
|
||||
echo " stream.memcpy_htod/dtoh on a regular &[T]/&mut [T] is"
|
||||
echo " forbidden. The source/dest slice is not page-locked, so the"
|
||||
echo " CUDA driver does an internal blocking HtoD/DtoH copy."
|
||||
echo " Use the mapped-pinned pattern instead: MappedF32Buffer /"
|
||||
echo " MappedI32Buffer (host_ptr write + dev_ptr DtoD via"
|
||||
echo " cudarc::driver::result::memcpy_dtod_async). See the"
|
||||
echo " read_slice_d / write_slice_*_d helpers in"
|
||||
echo " crates/ml-alpha/src/trainer/integrated.rs for the canonical"
|
||||
echo " reference implementation."
|
||||
echo ""
|
||||
echo " Lines flagged (NEW additions only; pre-existing violations"
|
||||
echo " are a separate cleanup tracked outside this guard):"
|
||||
echo "$bad" | sed 's/^/ /'
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
check_audit_doc_updates || exit 1
|
||||
check_no_todo_fixme || exit 1
|
||||
check_no_isv_migrations || exit 1
|
||||
check_no_dtod_via_pinned || exit 1
|
||||
check_no_raw_htod_dtoh || exit 1
|
||||
check_sp18_consumer_audit || exit 1
|
||||
|
||||
echo "✅ All pre-commit checks passed!"
|
||||
|
||||
Reference in New Issue
Block a user