docs: spec for deterministic eval via training graph replay
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,45 @@
|
||||
# Deterministic Forward via Training Graph Replay
|
||||
|
||||
## Problem
|
||||
|
||||
The backtest evaluator produces different val_Sharpe across process invocations despite identical training weights. Root cause: cublasLtMatmul in separate CUDA Graph capture sessions produces different internal kernel configurations.
|
||||
|
||||
## Solution
|
||||
|
||||
Replay the training `graph_forward` for validation. Same captured graph, same kernels, bit-identical results.
|
||||
|
||||
## Design
|
||||
|
||||
### Forward Pass for Validation
|
||||
|
||||
1. Write states into `trainer.states_buf` via `launch_pad_states`
|
||||
2. Replay `graph_forward` (cuBLAS forward + C51 loss + C51 grad + cuBLAS backward)
|
||||
3. Launch `compute_expected_q` kernel on the logits
|
||||
4. Read Q-values from `q_out_buf`
|
||||
|
||||
Loss/grad/backward writes go to scratch buffers overwritten on next training step. Harmless.
|
||||
|
||||
### Changes
|
||||
|
||||
**gpu_dqn_trainer.rs:**
|
||||
- Delete `compute_q_values_graphed` and `eval_fwd_graph` field
|
||||
- Add `replay_forward_for_q_values(batch_size)`: replays graph_forward + expected_q
|
||||
- Keep `graph_forward` alive even in mega-graph mode (currently set to None)
|
||||
|
||||
**fused_training.rs:**
|
||||
- QValueProvider calls `replay_forward_for_q_values` instead of `compute_q_values_graphed`
|
||||
- Delete pre-capture logic from mega graph capture
|
||||
|
||||
### Constraints
|
||||
|
||||
- `graph_forward` must remain captured (not cleared when mega graph supersedes)
|
||||
- Batch size: QValueProvider chunks through trainer batch_size
|
||||
- v_range: pinned device-mapped pointer (graph reads updated value on replay)
|
||||
|
||||
### Performance
|
||||
|
||||
~0.3ms extra per batch from wasted loss/grad/backward. Once per epoch. Negligible.
|
||||
|
||||
### Success Criteria
|
||||
|
||||
4 consecutive test runs produce identical val_Sharpe across all 30 epochs.
|
||||
Reference in New Issue
Block a user