diff --git a/docs/superpowers/specs/2026-04-13-deterministic-forward-design.md b/docs/superpowers/specs/2026-04-13-deterministic-forward-design.md new file mode 100644 index 000000000..957fd27a4 --- /dev/null +++ b/docs/superpowers/specs/2026-04-13-deterministic-forward-design.md @@ -0,0 +1,45 @@ +# Deterministic Forward via Training Graph Replay + +## Problem + +The backtest evaluator produces different val_Sharpe across process invocations despite identical training weights. Root cause: cublasLtMatmul in separate CUDA Graph capture sessions produces different internal kernel configurations. + +## Solution + +Replay the training `graph_forward` for validation. Same captured graph, same kernels, bit-identical results. + +## Design + +### Forward Pass for Validation + +1. Write states into `trainer.states_buf` via `launch_pad_states` +2. Replay `graph_forward` (cuBLAS forward + C51 loss + C51 grad + cuBLAS backward) +3. Launch `compute_expected_q` kernel on the logits +4. Read Q-values from `q_out_buf` + +Loss/grad/backward writes go to scratch buffers overwritten on next training step. Harmless. + +### Changes + +**gpu_dqn_trainer.rs:** +- Delete `compute_q_values_graphed` and `eval_fwd_graph` field +- Add `replay_forward_for_q_values(batch_size)`: replays graph_forward + expected_q +- Keep `graph_forward` alive even in mega-graph mode (currently set to None) + +**fused_training.rs:** +- QValueProvider calls `replay_forward_for_q_values` instead of `compute_q_values_graphed` +- Delete pre-capture logic from mega graph capture + +### Constraints + +- `graph_forward` must remain captured (not cleared when mega graph supersedes) +- Batch size: QValueProvider chunks through trainer batch_size +- v_range: pinned device-mapped pointer (graph reads updated value on replay) + +### Performance + +~0.3ms extra per batch from wasted loss/grad/backward. Once per epoch. Negligible. + +### Success Criteria + +4 consecutive test runs produce identical val_Sharpe across all 30 epochs.