cudarc's CudaSlice::Drop checked has_async_alloc (always false) and
fell back to stream.synchronize() + free_sync() on every drop. This
blocked the host for ~12.6ms per sync × 4.7 drops/step = ~59ms/step
of pure idle waiting (35.8% of wall time per nsys).
With has_async_alloc=true, drops use cuMemFreeAsync (non-blocking,
same-stream ordering). CUDA 12.4 on sm_86+ fully supports this.
Combined with the background diag writer and fused label gather,
the training loop should now be GPU-bound at ~13ms/step → ~77 sps.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
49dd4146ee