Eliminated 5 per-step GPU allocations:
- insert_prio_buf, insert_idx_buf, insert_ep_buf: 3x cuMemAlloc per
insert_batch/insert_batch_bf16 call (up to 4M elements each)
- scratch_f32: reusable 1-element temp for flush_max_priority and
apply_max_priority_scalar (was 2x a32f per call)
- update_batch_spare: pre-allocated spare for None→Some priority swap,
replenished by flush_max_priority (was 1x a32f per epoch start)
cuMemAlloc is a synchronizing driver call that stalls the GPU pipeline.
With 8192 batch_size and multiple inserts per epoch, this was measurable
overhead on H100.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
29d58efca2