fix: eliminate ALL hot-path cuMemcpy — pinned device-mapped everywhere
Per-step (16K/epoch):
- total_loss, mse_loss, grad_norm, q_divergence: CudaSlice → pinned
device-mapped. GPU kernels write via dev_ptr, CPU reads via host_ptr.
Zero copies in replay_adam_and_readback (was 4x cuMemcpyDtoHAsync).
- readback_scalars_sync, execute_train_scalars_only: sync DtoH → direct
pinned read after cuStreamSynchronize.
Per-50-steps:
- eval_v_range: cuMemcpyHtoDAsync → pinned host write (CPU writes
v_min/v_max, GPU reads via dev_ptr, no copy).
- per_branch_q_gaps: cuMemcpyHtoD → pinned host write (CPU writes 4
Q-gaps, GPU reads via dev_ptr in qlstm_step + liquid_tau_rk4_step).
- q_stats + q_out readback: stack destination → pinned DtoHAsync
destination (DMA-capable, faster async transfer).
Structural changes:
- launch_loss_reduce signature: &CudaSlice<f32> → u64 dev_ptr
- loss_gpu_buf/grad_norm_gpu_buf → loss_gpu_ptr/grad_norm_gpu_ptr (u64)
- memset_zeros on CudaSlice → cuMemsetD8Async on dev_ptr
- 6 new pinned allocations in constructor, freed in Drop
Only cuMemcpy remaining: constructor init, checkpoint save/restore,
xavier_init upload, trajectory backtracking, causal intervention,
compute_q_values inference. All per-step training copies eliminated.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ff562b07b2