perf: move Q-stats to epoch boundary only — zero sync in training loop
Per-step reduce_current_q_stats created GPU pipeline bubbles every 50 steps. Each sync waited for the GPU to drain all pending graph replays before the CPU could continue. Q-stats are now computed at epoch boundary only (process_epoch_boundary already calls reduce_current_q_stats). Training loop is now fully async: 178 graph launches queue in <1ms, GPU executes back-to-back with zero pipeline stalls. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1565,9 +1565,9 @@ impl DQNTrainer {
|
||||
} // end guard frequency check
|
||||
guard_total_us += guard_start.elapsed().as_micros() as u64;
|
||||
|
||||
// Q-stats + v_range adaptation every step.
|
||||
// Cost: ~1 kernel (q_stats_reduce) + sync for 76B DtoH readback.
|
||||
// q_out_buf is already populated by graph replay — no re-computation.
|
||||
// Q-stats computed at epoch boundary only (process_epoch_boundary).
|
||||
// Per-step Q-stats sync was creating GPU pipeline bubbles.
|
||||
if false {
|
||||
if let Some(ref mut fused) = self.fused_ctx {
|
||||
if let Ok(stats) = fused.reduce_current_q_stats() {
|
||||
self.cached_avg_q = stats.q_mean as f64;
|
||||
@@ -1586,6 +1586,7 @@ impl DQNTrainer {
|
||||
fused.update_eval_v_range(stats.q_mean, q_std, q_gap, per_branch_q_gaps);
|
||||
}
|
||||
}
|
||||
} // end if false (Q-stats disabled per-step)
|
||||
train_step_count += 1;
|
||||
self.gradient_logging_step += 1;
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user