46 lines
1.6 KiB
Markdown
46 lines
1.6 KiB
Markdown
# Deterministic Forward via Training Graph Replay
|
|
|
|
## Problem
|
|
|
|
The backtest evaluator produces different val_Sharpe across process invocations despite identical training weights. Root cause: cublasLtMatmul in separate CUDA Graph capture sessions produces different internal kernel configurations.
|
|
|
|
## Solution
|
|
|
|
Replay the training `graph_forward` for validation. Same captured graph, same kernels, bit-identical results.
|
|
|
|
## Design
|
|
|
|
### Forward Pass for Validation
|
|
|
|
1. Write states into `trainer.states_buf` via `launch_pad_states`
|
|
2. Replay `graph_forward` (cuBLAS forward + C51 loss + C51 grad + cuBLAS backward)
|
|
3. Launch `compute_expected_q` kernel on the logits
|
|
4. Read Q-values from `q_out_buf`
|
|
|
|
Loss/grad/backward writes go to scratch buffers overwritten on next training step. Harmless.
|
|
|
|
### Changes
|
|
|
|
**gpu_dqn_trainer.rs:**
|
|
- Delete `compute_q_values_graphed` and `eval_fwd_graph` field
|
|
- Add `replay_forward_for_q_values(batch_size)`: replays graph_forward + expected_q
|
|
- Keep `graph_forward` alive even in mega-graph mode (currently set to None)
|
|
|
|
**fused_training.rs:**
|
|
- QValueProvider calls `replay_forward_for_q_values` instead of `compute_q_values_graphed`
|
|
- Delete pre-capture logic from mega graph capture
|
|
|
|
### Constraints
|
|
|
|
- `graph_forward` must remain captured (not cleared when mega graph supersedes)
|
|
- Batch size: QValueProvider chunks through trainer batch_size
|
|
- v_range: pinned device-mapped pointer (graph reads updated value on replay)
|
|
|
|
### Performance
|
|
|
|
~0.3ms extra per batch from wasted loss/grad/backward. Once per epoch. Negligible.
|
|
|
|
### Success Criteria
|
|
|
|
4 consecutive test runs produce identical val_Sharpe across all 30 epochs.
|