PER sample, update_priority, and tree_rebuild run on a dedicated
train_stream. The env-step pipeline (encoder through push_flush)
runs on the main stream. Sync points:
1. Top of step_with_lobsim: train_stream.synchronize() ensures
previous step's priority/rebuild completed before push touches
the replay buffer.
2. Before K-loop: stream.synchronize() ensures push_flush is done
before train_stream's rl_per_sample reads the replay buffer.
3. After rl_per_sample: train_stream.synchronize() ensures sampled_*
buffers are written before step_synthetic/dqn_replay_step reads
them on the main stream.
4. After dqn_replay_step (k>0 path): stream.synchronize() ensures
td_per_sample_d is written before train_stream's priority update.
(step_synthetic already syncs internally.)
The last K-iter's priority update + tree rebuild are NOT synced at
step end — they overlap with the next step's encoder forward on the
main stream, hiding ~1ms of PER maintenance.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
b06355456a