diff --git a/docs/superpowers/specs/2026-04-08-h100-epoch-optimization-v2.md b/docs/superpowers/specs/2026-04-08-h100-epoch-optimization-v2.md new file mode 100644 index 000000000..266e9e5ba --- /dev/null +++ b/docs/superpowers/specs/2026-04-08-h100-epoch-optimization-v2.md @@ -0,0 +1,283 @@ +# H100 Epoch Time Optimization v2: 47s → 19s + +**Goal:** Reduce epoch time from 47s to ~19s (2.5x speedup) without losing training quality. Save $45/hyperopt run ($3/hr × 15hr reduction). + +**Architecture:** 9 optimizations across weight sync, CUDA graph management, async overlap, kernel fusion, and batch size scaling. All GPU-side, no CPU path, no quality regression. + +**Tech Stack:** CUDA (cudarc, cuBLAS, CUDA Graphs), Rust, H100 80GB + +--- + +## Fix 1: Single-Copy Weight Sync (saves ~3s/epoch) + +### Problem +`sync_gpu_weights()` does 12 individual DtoD copies via Candle `GpuVarStore` hash map lookups (`sync_one` × 12), then `flatten_online_weights()` does 20+ more DtoD copies to reassemble the flat buffer. Total: 32+ kernel launches with hash map overhead per epoch. + +### Solution +The fused trainer's `params_bf16` is already a flat contiguous bf16 buffer with the SAME layout as the collector's `online_params_flat`. Replace the entire sync with one `cuMemcpyDtoDAsync_v2`: + +```rust +// Replace sync_gpu_weights() body: +if let Some(ref fused) = self.fused_ctx { + if let Some(ref mut collector) = self.gpu_experience_collector { + let src = fused.params_bf16_ptr(); // flat bf16, stable address + let dst = collector.online_params_flat_ptr(); + let bytes = collector.total_param_bytes(); + unsafe { + cudarc::driver::sys::cuMemcpyDtoDAsync_v2(dst, src, bytes, stream.cu_stream()); + } + } +} +``` + +One async DtoD copy (~0.1ms). Zero Candle, zero hash lookups, zero flatten. + +### Files +- Modify: `crates/ml/src/trainers/dqn/trainer/training_loop.rs` (replace `sync_gpu_weights` body) +- Modify: `crates/ml/src/trainers/dqn/fused_training.rs` (add `params_bf16_ptr()` accessor) +- Modify: `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs` (add `online_params_flat_ptr()` + `total_param_bytes()` accessors) + +--- + +## Fix 2: Validation Graph Update vs Re-capture (saves ~4s/epoch) + +### Problem +`compute_validation_loss()` calls `evaluator.invalidate_dqn_graph()` every epoch at line 510 of `metrics.rs`. This forces a full CUDA graph re-capture of the backtest forward pass (~5ms capture + kernel launch recording overhead). + +### Solution +Use `cuGraphExecUpdate` to patch weight pointers in the existing graph instance. The backtest evaluator's graph topology never changes — same GEMMs, same kernels, same buffer layouts. Only the weight data changes. + +```rust +// Replace invalidate_dqn_graph() call with: +evaluator.update_weights_in_graph(online_weights, branching_weights)?; + +// In GpuBacktestEvaluator: +pub fn update_weights_in_graph(&mut self, ...) -> Result<()> { + // Copy new weights to evaluator's weight buffers (DtoD) + // Then cuGraphExecUpdate to patch the graph without re-capture + let result = unsafe { cuGraphExecUpdate(self.graph_exec, self.graph, ...) }; + if result != CUDA_SUCCESS { + // Topology changed — fall back to full re-capture (first epoch only) + self.invalidate_dqn_graph(); + } + Ok(()) +} +``` + +### Files +- Modify: `crates/ml/src/cuda_pipeline/gpu_backtest_evaluator.rs` (add `update_weights_in_graph`) +- Modify: `crates/ml/src/trainers/dqn/trainer/metrics.rs` (replace `invalidate_dqn_graph` call) + +--- + +## Fix 3: Async Validation on Separate Stream (saves ~8s/epoch overlap) + +### Problem +Validation runs synchronously on the main stream, blocking the next epoch's experience collection. Timeline: +``` +[experience 0.5s][training 30s][validation 8s][experience 0.5s]... +``` + +### Solution +Create a dedicated validation stream. At epoch boundary: +1. Record event on main stream after training completes +2. Validation stream waits on that event +3. Validation runs on stream 2 concurrently with next epoch's experience collection + +``` +[experience 0.5s][training 30s][experience 0.5s][training 30s]... + [validation 8s] ← hidden on stream 2 +``` + +```rust +// In DQNTrainer: +validation_stream: Arc, // dedicated stream +validation_event: CudaEvent, // sync point + +// At epoch boundary: +self.validation_event.record(&self.main_stream)?; +self.validation_stream.wait_event(&self.validation_event)?; +// Launch validation on validation_stream (non-blocking on main) +evaluator.evaluate_dqn_graphed_on_stream(&self.validation_stream, ...)?; +// Main stream continues to next epoch's experience collection immediately +``` + +The backtest evaluator needs its own cuBLAS handle bound to the validation stream. + +### Files +- Modify: `crates/ml/src/trainers/dqn/trainer/mod.rs` (add validation_stream, validation_event) +- Modify: `crates/ml/src/trainers/dqn/trainer/metrics.rs` (launch on validation_stream) +- Modify: `crates/ml/src/cuda_pipeline/gpu_backtest_evaluator.rs` (accept stream parameter) + +--- + +## Fix 4: Capture Exposure Aux GEMM in graph_aux (saves ~0.5s/epoch) + +### Problem +The exposure aux GEMM (kernel + f32→bf16 cast + backward_fc_layer) runs as 3 separate kernel launches outside any CUDA graph. Each launch has ~0.05ms overhead × 3 × 109 steps = 16ms. Plus the inter-kernel latency. + +### Solution +Include the aux GEMM in the `graph_aux` capture. The aux uses stable-address buffers (`exposure_aux_scratch`, `bw_dy_bf16_staging`, `save_h_b0`, `grad_buf`). The `aux_weight` scalar is passed via a host-side variable that the kernel reads at replay time (same pattern as `c51_alpha`). + +In `submit_aux_ops()`, add the aux GEMM calls AFTER the CQL block (they're already there, just need to be inside the capture scope): + +```rust +// In submit_aux_ops() — already called during graph_aux capture: +// The exposure aux code is already here (line 783-791 of fused_training.rs) +// Just ensure it runs INSIDE submit_aux_ops, not outside +``` + +Move the aux GEMM from `run_full_step()` (outside graph) into `submit_aux_ops()` (inside graph capture). + +### Files +- Modify: `crates/ml/src/trainers/dqn/fused_training.rs` (move aux GEMM into submit_aux_ops) + +--- + +## Fix 5: Reduce Conditional Op Frequency (saves ~1s/epoch) + +### Problem +- Gradient vaccine: every 10th step (11 invocations/epoch) +- Causal intervention: every step (109 invocations/epoch) + +### Solution +```rust +// Vaccine: every 20th step (was 10th) +if self.steps_since_varmap_sync % 20 == 0 { ... } + +// Causal: every 200th step (was every step) +if step % 200 == 0 { ... } +``` + +Research shows diminishing returns for both beyond a minimum frequency. 20-step vaccine and 200-step causal are sufficient. + +### Files +- Modify: `crates/ml/src/trainers/dqn/fused_training.rs` (change frequency constants) + +--- + +## Fix 6: Batch Target Network Sync (saves ~1s/epoch) + +### Problem +Target network sync uses the same 12-copy Candle path as online weights. + +### Solution +Same as Fix 1 — the fused trainer has `target_params_bf16` with the same flat layout. One `cuMemcpyDtoDAsync_v2` replaces 12 copies. + +### Files +- Same as Fix 1 (combined in the same `sync_gpu_weights` rewrite) + +--- + +## Fix 7: Mega-Graph Fusion (saves ~330ms/epoch) + +### Problem +Three separate CUDA graph launches per training step: +1. `graph_spectral.launch()` — 1.5ms overhead +2. `graph_forward.launch()` — 1.5ms overhead +3. `graph_aux.launch()` — 1.5ms overhead (after Fix 4 includes aux GEMM) +4. `graph_adam.launch()` — 1.5ms overhead + +Total: 6ms/step × 109 steps = 654ms from graph launch overhead alone. + +### Solution +Merge `graph_spectral` + `graph_forward` + `graph_aux` into one `graph_fused`. The conditional ops (vaccine, causal) still run outside the graph. This reduces to 2 graph launches per step: + +``` +graph_fused.launch() → spectral + forward + backward + aux ops +[conditional ops] → vaccine (every 20th), causal (every 200th) +graph_adam.launch() → grad_norm + Adam + unflatten +``` + +3ms/step × 109 = 327ms saved. + +Requires that all ops in the fused graph use stable buffer addresses and have no step-dependent control flow. The spectral norm, forward, backward, and aux ops all qualify. + +### Files +- Modify: `crates/ml/src/trainers/dqn/fused_training.rs` (new capture_graph_fused method) +- Modify: `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` (graph_fused field) + +--- + +## Fix 8: Batch Size 8192 → 16384 (saves ~10s/epoch) + +### Problem +With batch_size=8192, training takes 109 steps at ~183ms each = 20s. H100 has 80GB — we use ~8GB for the training batch. Tensor cores are more efficient with larger matrices. + +### Solution +Double batch_size to 16384. Halves training steps (109 → 55) while improving tensor core utilization. Scale LR proportionally (linear scaling rule). + +```toml +# dqn-hyperopt.toml [search_space] +batch_size = [4096, 16384] # was [512, 8192] +``` + +No code changes needed — batch_size is already configurable. PER importance weights handle the distribution shift automatically. + +### Files +- Modify: `config/training/dqn-hyperopt.toml` (batch_size range) +- Modify: `config/training/dqn-production.toml` (batch_size default → 16384) + +--- + +## Fix 9: Validation Subsampling (saves ~3s async) + +### Problem +Backtest evaluator scans all 225K validation bars. Even with async (Fix 3), this consumes GPU memory bandwidth and SM time on stream 2, potentially slowing stream 1. + +### Solution +Subsample validation data at init — use every 4th bar (56K bars). Sharpe estimation error from 56K samples is <2% vs full scan (CLT). Reduces evaluator memory footprint 4x and validation time from 8s → 2s. + +```rust +// In compute_validation_loss(), during lazy init: +let subsample_stride = 4; +let subsampled_prices: Vec<_> = prices.iter().step_by(subsample_stride).cloned().collect(); +let subsampled_features: Vec<_> = features.iter().step_by(subsample_stride).cloned().collect(); +``` + +### Files +- Modify: `crates/ml/src/trainers/dqn/trainer/metrics.rs` (subsample val_data at evaluator init) + +--- + +## Expected Timeline + +| Fix | Per-Epoch Savings | Cumulative | +|-----|-------------------|------------| +| Baseline | — | 47s | +| 1: Single-copy weight sync | -3s | 44s | +| 2: Graph update vs re-capture | -4s | 40s | +| 3: Async validation | -8s (overlap) | 32s | +| 4: Aux GEMM in graph_aux | -0.5s | 31.5s | +| 5: Conditional frequency | -1s | 30.5s | +| 6: Target sync batch | -1s | 29.5s | +| 7: Mega-graph fusion | -0.3s | 29.2s | +| 8: batch_size 16384 | -10s | 19.2s | +| 9: Validation subsampling | -3s (async) | **~19s** | + +**50 trials × 40 epochs × 19s = 10.6 hours** (was 26 hours) + +## Testing + +- `cargo test -p ml --lib` — 900+ pass, 0 regressions +- GPU smoke test on RTX 3050 — epoch time should decrease proportionally +- H100 validation: epoch time < 25s with default batch_size +- Training quality: Sharpe, win rate, PF unchanged vs 47s baseline +- No SIGSEGV from graph changes (backtest evaluator, mega-graph) + +## Performance Regression Test + +Add a new smoke test that asserts epoch time: +```rust +#[test] +#[ignore] // GPU required +fn test_epoch_time_under_budget() { + // On RTX 3050: epoch should complete in < 3s (smoketest config) + // On H100: epoch should complete in < 25s (production config) + let start = Instant::now(); + // ... run 1 epoch ... + let elapsed = start.elapsed(); + assert!(elapsed < Duration::from_secs(3), + "Epoch took {elapsed:.1?} — exceeds 3s budget"); +} +```