jgrusewski
99707f7e4e
perf(rl): eliminate 5 of 7 stream.synchronize() from step hot path
Replace per-step cuStreamSynchronize calls with deferred-read and
event-based cross-stream sync patterns:
- ISV staging reads (2 syncs): use previous step's staging data
(one-step delay acceptable for slow-moving EMA controllers).
Queue DtoD async for next step's read, flushed by batched
loss sync.
- Loss scalar reads (3 syncs → 1): batch 3 individual
read_scalar_via_staging calls (each with DtoD+sync) + FRD
DtoD+sync into 4 concurrent DtoDs + 1 sync via pre-allocated
loss_staging_3 MappedF32Buffer.
- Cross-stream syncs (3 syncs → 0 host-blocking): replace
cuStreamSynchronize with CudaEvent record/wait pairs for
push_done, sample_done, replay_done inter-stream dependencies.
Events sync GPU timelines without blocking the host.
Hot path: 7 explicit syncs → 1 batched loss sync + 1 step-start
train_stream sync. Expected: ~5× reduction in host-wait time
per step (14ms → 2-4ms).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>