fix: acquire write lock once per epoch, not per training step

The async write lock (self.agent.write().await) was acquired inside the
per-step training loop, causing 696s/epoch overhead from async yield on
every iteration — 178 steps × ~4s per lock acquisition. Actual GPU compute
was only 3.6s (graph replay at 2ms/step).

Fix: acquire the lock once before the loop. The training loop is the only
writer during this phase — no contention, no need to release between steps.

Expected: epoch time drops from 700s to ~5s.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-04-18 13:15:25 +02:00
parent 8d88c63434
commit c3cdb39081

View File

@@ -1473,10 +1473,10 @@ impl DQNTrainer {
}
}
// Acquire write lock ONCE for the entire training loop — not per-step.
// Per-step lock acquisition was burning 696s/epoch due to async yield overhead.
let mut agent = self.agent.write().await;
for _step in 0..num_training_steps {
// Single write lock for both sample + fused step — eliminates
// read→write lock cycling overhead (~43ms per step from contention).
let mut agent = self.agent.write().await;
let sample_start = std::time::Instant::now();
{