d6f0538ac0
Three-phase plan to eliminate all GPU→CPU roundtrips from training: - Phase 1: Seal training loop (persistent GPU epoch state, async monitoring) - Phase 2: Vectorized CUDA backtest kernel for hyperopt evaluation - Phase 3: General-purpose GPU backtester replacing CPU SIMD path Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
435 lines
20 KiB
Markdown
435 lines
20 KiB
Markdown
# CUDA Backtest & GPU-Resident Training — Design Spec
|
||
|
||
> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||
|
||
**Goal:** Eliminate all GPU→CPU roundtrips from the DQN/PPO training loop and port the walk-forward backtesting engine to CUDA, enabling fully GPU-resident training and evaluation.
|
||
|
||
**Architecture:** Three-phase approach — (1) seal remaining CPU roundtrips in the training loop, (2) build a vectorized GPU backtest environment for hyperopt evaluation, (3) extend to a general-purpose GPU backtester replacing the CPU SIMD path.
|
||
|
||
**Tech Stack:** Rust, cudarc 0.17, NVRTC, Candle (tensor ops + model forward), CUDA C (custom kernels)
|
||
|
||
---
|
||
|
||
## Context & Motivation
|
||
|
||
The DQN training loop currently has ~80% GPU residency:
|
||
|
||
- `GpuExperienceCollector` — full episode kernel (action selection, portfolio sim, barriers, fill sim, reward, TD error)
|
||
- `collect_experiences_gpu()` → `insert_batch_tensors()` — zero-roundtrip DtoD path to GPU replay buffer
|
||
- `GpuReplayBuffer` — GPU-resident PER sampling (proportional + rank-based)
|
||
- `GpuTrainingGuard` — on-device NaN/loss-clip/grad-collapse checks
|
||
|
||
**Remaining CPU roundtrips to eliminate:**
|
||
|
||
1. **Training step readbacks**: `to_scalar` for loss, `to_vec1` for Q-values, gradient norms
|
||
2. **Monitoring downloads**: rewards_cpu/actions_cpu from experience kernel (per kernel launch)
|
||
3. **Epoch-boundary state**: vol EMA, portfolio reset/compound, DSR normalizer reset — force `cudaStreamSync`
|
||
4. **Hyperopt evaluation**: trained model weights downloaded → CPU walk-forward backtest → metric uploaded
|
||
5. **Standalone evaluation**: `evaluate_baseline` binary runs CPU backtesting with SIMD AVX2
|
||
|
||
---
|
||
|
||
## Phase 1: Seal the Training Loop
|
||
|
||
Close remaining GPU→CPU gaps in `crates/ml/src/trainers/dqn/trainer.rs` and CUDA pipeline.
|
||
|
||
### 1.1 Eliminate train_step readbacks
|
||
|
||
**Loss scalar** (`trainer.rs:4797`):
|
||
- `GpuTrainingGuard` already checks loss on-device via pinned host memory
|
||
- Defer `to_scalar` readback: accumulate losses on GPU, single async readback at epoch end
|
||
- Replace per-step loss with GPU-side running mean (EMA in training guard kernel)
|
||
|
||
**Q-value monitoring** (`trainer.rs:4834,5495`):
|
||
- `gpu_qvalue_stats` kernel already computes mean/std/min/max on device
|
||
- Drop the CPU `to_vec1` → `to_vec2` fallback paths entirely when `GpuTrainingGuard` is active
|
||
- Q-value divergence check: already has GPU kernel path — make it exclusive on CUDA
|
||
|
||
**Gradient norm** (`trainer.rs:5419`):
|
||
- Candle computes grad norm during `optimizer.step()` — stays on GPU
|
||
- Current `to_scalar` readback is for logging only
|
||
- Fix: write to pinned host memory via `cuMemHostAlloc`, read asynchronously at epoch end
|
||
|
||
### 1.2 Batch monitoring downloads
|
||
|
||
**Current**: `collect_experiences_gpu()` downloads `rewards_cpu` and `actions_cpu` after every kernel launch for monitoring (mean reward, Sharpe, action diversity).
|
||
|
||
**Fix**: Accumulate monitoring stats on GPU across all kernel launches within an epoch. Single `memcpy_dtoh` of summary struct at epoch end:
|
||
```c
|
||
struct EpochMonitoringSummary {
|
||
float mean_reward;
|
||
float reward_std;
|
||
float sharpe_estimate; // mean/std * sqrt(steps)
|
||
int action_counts[5]; // per-exposure-level counts
|
||
float max_reward;
|
||
float min_reward;
|
||
int total_experiences;
|
||
};
|
||
```
|
||
|
||
Add a small `monitoring_reduction_kernel` that reduces per-experience rewards/actions into this summary. Launch once at epoch end. Download 48 bytes instead of `2 * N_episodes * timesteps * 4` bytes per kernel launch.
|
||
|
||
### 1.3 GPU-persistent epoch-boundary state
|
||
|
||
Move epoch-boundary state management from CPU to persistent GPU buffers.
|
||
|
||
**Persistent `CudaSlice<f32>` buffers on `GpuExperienceCollector`:**
|
||
|
||
| Buffer | Size | Purpose |
|
||
|--------|------|---------|
|
||
| `epoch_state` | 8 × f32 | vol_ema, median_vol, portfolio_value, portfolio_position, portfolio_cash, dsr_mean, dsr_var, step_count |
|
||
|
||
**Kernel changes:**
|
||
- Last step of experience kernel writes final state to `epoch_state` buffer
|
||
- Next epoch's kernel launch reads `epoch_state` as initial state
|
||
- New kernel arg `reset_flags: u32` (bitfield): bit 0 = reset portfolio (DSR mode), bit 1 = reset DSR normalizer, bit 2 = reset vol EMA
|
||
|
||
**Result**: Zero `cudaStreamSynchronize` between epochs. The only CPU→GPU communication is updating kernel config args (epsilon decay, learning rate) which are cheap scalar copies.
|
||
|
||
### 1.4 Remove CPU fallback codepaths (CUDA builds)
|
||
|
||
When compiled with `feature = "cuda"` and running on a CUDA device:
|
||
- Remove `#[cfg(not(feature = "cuda"))]` branches from the hot path in `train_epoch()`
|
||
- The CPU fallback in the experience collection loop (`if !gpu_experiences_collected`) should be unreachable when GPU collector is initialized
|
||
- Add `debug_assert!` guards confirming GPU path was taken
|
||
- Keep CPU paths for `feature = "cpu-only"` builds and test harness
|
||
|
||
### 1.5 Expected outcome
|
||
|
||
| Metric | Before | After |
|
||
|--------|--------|-------|
|
||
| `cudaStreamSync` per epoch | ~8-16 (per batch readback) | 1 (epoch-end monitoring) |
|
||
| CPU roundtrips per experience batch | 2 (rewards + actions download) | 0 |
|
||
| CPU roundtrips per train step | 3 (loss + Q-stats + grad norm) | 0 |
|
||
| Epoch boundary sync | 1 (portfolio/vol state) | 0 |
|
||
|
||
---
|
||
|
||
## Phase 2: CUDA Backtest Kernel for Hyperopt Evaluation
|
||
|
||
### 2.1 Problem statement
|
||
|
||
Hyperopt evaluation loop:
|
||
```
|
||
for trial in 0..N_trials:
|
||
train model on GPU (epochs)
|
||
for fold in walk_forward_windows:
|
||
download model weights to CPU # SYNC
|
||
run CPU backtest on test window # SLOW
|
||
compute Sharpe/PnL on CPU # SLOW
|
||
upload aggregated metric to optimizer # SYNC
|
||
```
|
||
|
||
On H100 with 20 hyperopt trials × 8 walk-forward folds × ~100K bars per window:
|
||
- CPU backtest: ~2-5s per fold × 8 folds = 16-40s per trial
|
||
- GPU training: ~30s per trial
|
||
- **Evaluation is 30-60% of total hyperopt time**
|
||
|
||
### 2.2 Architecture: Vectorized GPU Environment
|
||
|
||
Pattern: NVIDIA Isaac Gym / Google Brax style — parallelize across environments (walk-forward windows), sequential within each.
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────────┐
|
||
│ GPU Memory │
|
||
│ │
|
||
│ ┌──────────────────────────────────────────────┐ │
|
||
│ │ Market Data [N_windows × max_len × feat_dim] │ │
|
||
│ │ (uploaded once, read-only, SoA layout) │ │
|
||
│ └──────────────────────────────────────────────┘ │
|
||
│ │
|
||
│ for step in 0..max_steps: │
|
||
│ ┌────────────────────────────────────────────┐ │
|
||
│ │ 1. Gather states [cudarc kernel] │ │
|
||
│ │ states[w] = features[w][step] ++ port[w]│ │
|
||
│ └──────────────┬─────────────────────────────┘ │
|
||
│ ▼ │
|
||
│ ┌────────────────────────────────────────────┐ │
|
||
│ │ 2. Batch forward [Candle, on-device] │ │
|
||
│ │ q_values = model.forward(states_batch) │ │
|
||
│ └──────────────┬─────────────────────────────┘ │
|
||
│ ▼ │
|
||
│ ┌────────────────────────────────────────────┐ │
|
||
│ │ 3. Action select [cudarc kernel] │ │
|
||
│ │ actions = argmax(q_values, dim=-1) │ │
|
||
│ └──────────────┬─────────────────────────────┘ │
|
||
│ ▼ │
|
||
│ ┌────────────────────────────────────────────┐ │
|
||
│ │ 4. Env step [cudarc kernel] │ │
|
||
│ │ For each window w (parallel): │ │
|
||
│ │ execute_trade(actions[w], port[w]) │ │
|
||
│ │ compute_reward(port[w], prices[w]) │ │
|
||
│ │ update portfolio state │ │
|
||
│ │ check done (window exhausted) │ │
|
||
│ └──────────────┬─────────────────────────────┘ │
|
||
│ ▼ │
|
||
│ end for │
|
||
│ │
|
||
│ ┌──────────────────────────────────────────────┐ │
|
||
│ │ 5. Metrics reduction [cudarc kernel] │ │
|
||
│ │ Per window: Sharpe, total_pnl, max_dd │ │
|
||
│ └──────────────┬───────────────────────────────┘ │
|
||
│ ▼ │
|
||
│ Single memcpy_dtoh: [N_windows × 3] floats │
|
||
└─────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
### 2.3 Kernel designs
|
||
|
||
#### `backtest_env_kernel.cu` — Vectorized environment step
|
||
|
||
```c
|
||
// One thread per walk-forward window
|
||
// Each thread steps sequentially through its window
|
||
__global__ void backtest_env_step(
|
||
// Market data (read-only, SoA)
|
||
const float* __restrict__ features, // [N_windows, max_len, feat_dim]
|
||
const float* __restrict__ prices, // [N_windows, max_len, 4] (open, high, low, close)
|
||
const int* __restrict__ window_lens, // [N_windows] actual length per window
|
||
|
||
// Actions from model (read-only)
|
||
const int* __restrict__ actions, // [N_windows] current step action
|
||
|
||
// Portfolio state (read-write, persistent across steps)
|
||
float* portfolio_state, // [N_windows, PORTFOLIO_STATE_SIZE]
|
||
// Layout per window: [value, position, cash, entry_price, max_equity, step_pnl, cum_return, step_count]
|
||
|
||
// Step rewards (write)
|
||
float* step_rewards, // [N_windows]
|
||
int* done_flags, // [N_windows]
|
||
|
||
// Config
|
||
float max_position,
|
||
float spread_cost,
|
||
float tx_cost_bps,
|
||
int current_step
|
||
) {
|
||
int w = blockIdx.x * blockDim.x + threadIdx.x;
|
||
if (w >= N_WINDOWS) return;
|
||
if (done_flags[w]) return; // already done
|
||
if (current_step >= window_lens[w]) { done_flags[w] = 1; return; }
|
||
|
||
// Read current price
|
||
int price_idx = w * max_len * 4 + current_step * 4;
|
||
float close = prices[price_idx + 3];
|
||
|
||
// Read portfolio state
|
||
int ps_idx = w * PORTFOLIO_STATE_SIZE;
|
||
float value = portfolio_state[ps_idx + 0];
|
||
float position = portfolio_state[ps_idx + 1];
|
||
float cash = portfolio_state[ps_idx + 2];
|
||
// ... (execute trade, compute reward, update state)
|
||
|
||
// Write back
|
||
portfolio_state[ps_idx + 0] = new_value;
|
||
step_rewards[w] = step_return;
|
||
}
|
||
```
|
||
|
||
#### `backtest_metrics_kernel.cu` — Per-window reduction
|
||
|
||
```c
|
||
// Reduction kernel: compute Sharpe, total PnL, max drawdown per window
|
||
// Uses warp-level primitives for efficient reduction
|
||
__global__ void compute_backtest_metrics(
|
||
const float* step_returns, // [N_windows, max_len]
|
||
const int* window_lens, // [N_windows]
|
||
float* metrics_out, // [N_windows, 3]: (sharpe, total_pnl, max_drawdown)
|
||
int max_len
|
||
) {
|
||
int w = blockIdx.x;
|
||
// Thread-parallel reduction over step_returns[w][0..window_lens[w]]
|
||
// Compute: mean, variance (Welford's online), cumsum for drawdown
|
||
// Write: sharpe = mean/sqrt(var) * sqrt(252), total_pnl, max_dd
|
||
}
|
||
```
|
||
|
||
### 2.4 Rust orchestrator: `GpuBacktestEvaluator`
|
||
|
||
```rust
|
||
pub struct GpuBacktestEvaluator {
|
||
stream: Arc<CudaStream>,
|
||
env_kernel: CudaFunction,
|
||
metrics_kernel: CudaFunction,
|
||
// Persistent GPU buffers
|
||
features_buf: CudaSlice<f32>, // [N_windows, max_len, feat_dim]
|
||
prices_buf: CudaSlice<f32>, // [N_windows, max_len, 4]
|
||
window_lens_buf: CudaSlice<i32>, // [N_windows]
|
||
portfolio_buf: CudaSlice<f32>, // [N_windows, PORTFOLIO_STATE_SIZE]
|
||
rewards_buf: CudaSlice<f32>, // [N_windows, max_len]
|
||
done_buf: CudaSlice<i32>, // [N_windows]
|
||
actions_buf: CudaSlice<i32>, // [N_windows]
|
||
metrics_buf: CudaSlice<f32>, // [N_windows, 3]
|
||
// Config
|
||
n_windows: usize,
|
||
max_window_len: usize,
|
||
feature_dim: usize,
|
||
}
|
||
|
||
impl GpuBacktestEvaluator {
|
||
/// Upload walk-forward test windows to GPU (called once per hyperopt trial).
|
||
pub fn upload_windows(
|
||
windows: &[WalkForwardWindow],
|
||
feature_extractor: &impl FeatureExtractor,
|
||
device: &Device,
|
||
) -> Result<Self, MLError>;
|
||
|
||
/// Run full evaluation: step loop with model forward + env kernel.
|
||
/// Returns per-window metrics without leaving GPU until final readback.
|
||
pub fn evaluate(
|
||
&mut self,
|
||
model: &dyn ModelForward, // Candle model with .forward(Tensor) -> Tensor
|
||
device: &Device,
|
||
) -> Result<Vec<WindowMetrics>, MLError>;
|
||
}
|
||
|
||
pub struct WindowMetrics {
|
||
pub sharpe: f32,
|
||
pub total_pnl: f32,
|
||
pub max_drawdown: f32,
|
||
}
|
||
```
|
||
|
||
### 2.5 Integration with hyperopt adapters
|
||
|
||
Each hyperopt adapter's `evaluate()` method gains a GPU path:
|
||
|
||
```rust
|
||
// In hyperopt/adapters/dqn.rs (and ppo.rs, tft.rs, etc.)
|
||
fn evaluate_walk_forward(&self, model: &TrainedModel, windows: &[WalkForwardWindow]) -> f64 {
|
||
#[cfg(feature = "cuda")]
|
||
if self.device.is_cuda() {
|
||
// GPU path: zero roundtrips
|
||
let mut evaluator = GpuBacktestEvaluator::upload_windows(windows, &self.feature_extractor, &self.device)?;
|
||
let metrics = evaluator.evaluate(model, &self.device)?;
|
||
return aggregate_sharpe(&metrics);
|
||
}
|
||
// CPU fallback: existing walk_forward_backtest()
|
||
self.cpu_evaluate_walk_forward(model, windows)
|
||
}
|
||
```
|
||
|
||
### 2.6 Memory budget
|
||
|
||
On H100 80GB with 8 walk-forward windows × 100K bars × 48-dim features:
|
||
|
||
| Buffer | Size |
|
||
|--------|------|
|
||
| Features | 8 × 100K × 48 × 4B = 147 MB |
|
||
| Prices | 8 × 100K × 4 × 4B = 12 MB |
|
||
| Portfolio state | 8 × 8 × 4B = 256 B |
|
||
| Step rewards | 8 × 100K × 4B = 3 MB |
|
||
| Actions/done/lens | < 1 MB |
|
||
| Metrics output | 96 B |
|
||
| **Total** | **~163 MB** (~0.2% of H100 VRAM) |
|
||
|
||
Trivial. Could run 256+ parallel windows if needed for ensemble cross-validation.
|
||
|
||
### 2.7 Expected speedup
|
||
|
||
| Component | CPU (current) | GPU (target) | Speedup |
|
||
|-----------|--------------|-------------|---------|
|
||
| Feature gather per step | 1-5 μs | <0.1 μs (coalesced read) | 10-50× |
|
||
| Model forward (batch of 8) | 100-500 μs | 20-50 μs (already on GPU) | 5-10× |
|
||
| Env step (8 windows) | 8-40 μs | 0.5-2 μs (parallel threads) | 10-20× |
|
||
| Metrics reduction | 5-20 ms | 0.1-0.5 ms | 20-40× |
|
||
| Data transfer overhead | 2-5 ms (weight download) | 0 (stays on GPU) | ∞ |
|
||
| **Per-trial eval (8 folds × 100K bars)** | **16-40s** | **1-3s** | **8-15×** |
|
||
|
||
---
|
||
|
||
## Phase 3: General-Purpose GPU Backtester
|
||
|
||
Extends Phase 2 to serve as a drop-in replacement for `crates/backtesting/strategy_runner.rs`.
|
||
|
||
### 3.1 Additional capabilities
|
||
|
||
- **Ensemble inference**: Batch forward through K models, GPU-side confidence aggregation (weighted vote)
|
||
- **Full metrics suite**: Sharpe, Sortino, Calmar, VaR (percentile), CVaR (tail mean), win rate, avg trade return, profit factor — all via GPU reduction kernels
|
||
- **Slippage modeling**: `VolumeImpactSlippage` (Almgren-Chriss `sqrtf()`) computed in env kernel
|
||
- **Triple barrier episodes**: Port existing logic from `dqn_experience_kernel.cu` — already GPU-proven
|
||
- **Position sizing**: Kelly criterion scaling (already in `dqn_experience_kernel.cu`)
|
||
|
||
### 3.2 New kernel: `backtest_full_metrics_kernel.cu`
|
||
|
||
Extends `backtest_metrics_kernel.cu` with:
|
||
- Sortino ratio (downside deviation only)
|
||
- Rolling max drawdown with recovery time tracking
|
||
- VaR/CVaR via parallel sort + percentile extraction
|
||
- Win rate / profit factor from per-trade P&L buffer
|
||
- Annualization with configurable trading days (252 default)
|
||
|
||
### 3.3 Integration with evaluate_baseline binary
|
||
|
||
```rust
|
||
// In bin targets: evaluate_baseline, evaluate_supervised
|
||
fn run_evaluation(model_path: &Path, test_data: &[OHLCVBar], config: &EvalConfig) -> EvalReport {
|
||
#[cfg(feature = "cuda")]
|
||
if let Ok(device) = Device::new_cuda(0) {
|
||
let evaluator = GpuBacktestEvaluator::new_full(test_data, config, &device)?;
|
||
return evaluator.evaluate_full(model, &device)?;
|
||
}
|
||
// CPU fallback: existing AdaptiveStrategyRunner with SIMD
|
||
run_cpu_evaluation(model_path, test_data, config)
|
||
}
|
||
```
|
||
|
||
### 3.4 Compatibility
|
||
|
||
- CPU path preserved for non-CUDA builds and CI testing
|
||
- f32 GPU results validated against f64 CPU results (max 0.1% relative error for Sharpe, 0.01% for PnL)
|
||
- `Decimal` precision maintained in CPU path; GPU uses f32 (sufficient for relative model comparison)
|
||
|
||
---
|
||
|
||
## File Map
|
||
|
||
### Phase 1 (modified files)
|
||
- `crates/ml/src/trainers/dqn/trainer.rs` — remove CPU readback paths, add GPU-persistent epoch state
|
||
- `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs` — persistent epoch_state buffer, monitoring reduction
|
||
- `crates/ml/src/cuda_pipeline/dqn_experience_kernel.cu` — epoch state read/write, monitoring accumulation
|
||
- `crates/ml/src/cuda_pipeline/gpu_training_guard.rs` — async pinned-memory readback for grad norm
|
||
|
||
### Phase 2 (new files)
|
||
- `crates/ml/src/cuda_pipeline/backtest_env_kernel.cu` — vectorized environment step kernel
|
||
- `crates/ml/src/cuda_pipeline/backtest_metrics_kernel.cu` — per-window Sharpe/PnL/drawdown reduction
|
||
- `crates/ml/src/cuda_pipeline/gpu_backtest_env.rs` — Rust wrapper for env kernel
|
||
- `crates/ml/src/cuda_pipeline/gpu_backtest_metrics.rs` — Rust wrapper for metrics kernel
|
||
- `crates/ml/src/cuda_pipeline/gpu_backtest_evaluator.rs` — orchestrator (upload → step loop → metrics)
|
||
- `crates/ml/src/hyperopt/gpu_evaluator.rs` — adapter integration
|
||
|
||
### Phase 2 (modified files)
|
||
- `crates/ml/src/cuda_pipeline/mod.rs` — re-export new modules
|
||
- `crates/ml/src/hyperopt/adapters/dqn.rs` — GPU eval path
|
||
- `crates/ml/src/hyperopt/adapters/ppo.rs` — GPU eval path
|
||
- `crates/ml/src/hyperopt/adapters/tft.rs` — GPU eval path (and remaining supervised adapters)
|
||
|
||
### Phase 3 (new files)
|
||
- `crates/ml/src/cuda_pipeline/backtest_full_metrics_kernel.cu` — extended metrics (Sortino, VaR, etc.)
|
||
- `crates/ml/src/cuda_pipeline/gpu_backtest_full_metrics.rs` — Rust wrapper for full metrics
|
||
|
||
### Phase 3 (modified files)
|
||
- `crates/ml/src/cuda_pipeline/gpu_backtest_evaluator.rs` — ensemble support, full metrics
|
||
- Evaluate binary entry points
|
||
|
||
---
|
||
|
||
## Risks & Mitigations
|
||
|
||
| Risk | Mitigation |
|
||
|------|-----------|
|
||
| Candle tensor ↔ cudarc CudaSlice interop | Both use same CUDA context via `candle_core::cuda_backend::cudarc`; proven pattern in existing `gpu_experience_collector.rs` |
|
||
| Numerical divergence GPU vs CPU | Validation tests comparing f32 GPU vs f64 CPU with documented tolerance bounds |
|
||
| Kernel compilation time (NVRTC) | Cache compiled PTX in `GpuBacktestEvaluator::new()`, reuse across trials |
|
||
| Variable-length windows | Pad to max length, use `window_lens` array + early `done_flag` exit per thread |
|
||
| Model architecture diversity | Use Candle `.forward()` for all models — only env kernels are custom CUDA |
|
||
|
||
---
|
||
|
||
## Success Criteria
|
||
|
||
1. **Phase 1**: Zero `cudaStreamSynchronize` during DQN training epoch (only at epoch boundary for monitoring summary download)
|
||
2. **Phase 2**: Hyperopt evaluation runs fully on GPU; per-trial eval time drops from 16-40s to 1-3s on H100
|
||
3. **Phase 3**: `evaluate_baseline` binary uses GPU backtester when CUDA available, produces results within 0.1% of CPU path
|
||
4. **All phases**: Existing CPU paths unchanged, no regressions in 2758+ lib tests
|