docs: spec for deterministic eval via training graph replay

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-04-13 09:26:14 +02:00
parent 37eb4918dd
commit 0f5b2b07b0

View File

@@ -0,0 +1,45 @@
# Deterministic Forward via Training Graph Replay
## Problem
The backtest evaluator produces different val_Sharpe across process invocations despite identical training weights. Root cause: cublasLtMatmul in separate CUDA Graph capture sessions produces different internal kernel configurations.
## Solution
Replay the training `graph_forward` for validation. Same captured graph, same kernels, bit-identical results.
## Design
### Forward Pass for Validation
1. Write states into `trainer.states_buf` via `launch_pad_states`
2. Replay `graph_forward` (cuBLAS forward + C51 loss + C51 grad + cuBLAS backward)
3. Launch `compute_expected_q` kernel on the logits
4. Read Q-values from `q_out_buf`
Loss/grad/backward writes go to scratch buffers overwritten on next training step. Harmless.
### Changes
**gpu_dqn_trainer.rs:**
- Delete `compute_q_values_graphed` and `eval_fwd_graph` field
- Add `replay_forward_for_q_values(batch_size)`: replays graph_forward + expected_q
- Keep `graph_forward` alive even in mega-graph mode (currently set to None)
**fused_training.rs:**
- QValueProvider calls `replay_forward_for_q_values` instead of `compute_q_values_graphed`
- Delete pre-capture logic from mega graph capture
### Constraints
- `graph_forward` must remain captured (not cleared when mega graph supersedes)
- Batch size: QValueProvider chunks through trainer batch_size
- v_range: pinned device-mapped pointer (graph reads updated value on replay)
### Performance
~0.3ms extra per batch from wasted loss/grad/backward. Once per epoch. Negligible.
### Success Criteria
4 consecutive test runs produce identical val_Sharpe across all 30 epochs.