perf: move Q-stats to epoch boundary only — zero sync in training loop

Per-step reduce_current_q_stats created GPU pipeline bubbles every 50 steps.
Each sync waited for the GPU to drain all pending graph replays before
the CPU could continue. Q-stats are now computed at epoch boundary only
(process_epoch_boundary already calls reduce_current_q_stats).

Training loop is now fully async: 178 graph launches queue in <1ms,
GPU executes back-to-back with zero pipeline stalls.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-04-18 14:36:36 +02:00
parent ff2b599965
commit c8d87626d3

View File

@@ -1565,9 +1565,9 @@ impl DQNTrainer {
} // end guard frequency check
guard_total_us += guard_start.elapsed().as_micros() as u64;
// Q-stats + v_range adaptation every step.
// Cost: ~1 kernel (q_stats_reduce) + sync for 76B DtoH readback.
// q_out_buf is already populated by graph replay — no re-computation.
// Q-stats computed at epoch boundary only (process_epoch_boundary).
// Per-step Q-stats sync was creating GPU pipeline bubbles.
if false {
if let Some(ref mut fused) = self.fused_ctx {
if let Ok(stats) = fused.reduce_current_q_stats() {
self.cached_avg_q = stats.q_mean as f64;
@@ -1586,6 +1586,7 @@ impl DQNTrainer {
fused.update_eval_v_range(stats.q_mean, q_std, q_gap, per_branch_q_gaps);
}
}
} // end if false (Q-stats disabled per-step)
train_step_count += 1;
self.gradient_logging_step += 1;
}