foxhunt/docs/plans/2026-02-28-gpu-pipeline-design.md

GPU Pipeline Design: Custom CUDA Kernels for DQN/PPO Hyperopt

Date: 2026-02-28 Status: Approved (casting fixes implemented, CUDA kernels pending)

Problem

Pipeline #208 confirmed GPU works but only hits 12% utilization on L4 (24GB). Root cause: experience collection hot loop (trainer.rs:1317-1605) runs ~300K per-bar iterations on CPU doing portfolio tracking, barrier checks, reward calculation, and wasteful Decimal-to-float conversions via heap-allocated Strings.

Phase 0: Casting Fixes (DONE)

Eliminated all .to_string().parse::<f32/f64>() patterns across 19 files (36 instances). Replaced with num_traits::ToPrimitive methods (.to_f32(), .to_f64()).

Key changes:

• feature_vector_to_state: price.to_string().parse::<f32>() → price.to_f32().unwrap_or(0.0)
• Reward output: reward_decimal.to_string().parse::<f32>() → reward_decimal.to_f32().unwrap_or(0.0)
• Barrier scaling: scaled.to_string().parse::<f32>() → scaled.to_f64().unwrap_or(0.0)
• risk_integration.rs: blocking_read() → .read().await (no sync-in-async)
• 32 test files: same pattern fixes

Decimal stays as the canonical financial type. Conversions happen at GPU boundaries via ToPrimitive.

Phase 1: Pre-upload Data to GPU (next)

Upload all training data as f32 GPU tensors at epoch start. Eliminates per-bar Vec<f32> allocations and f64→f32 conversions.

Before: [CPU f64 Vec] → per-bar copy → Tensor → GPU
After:  [GPU f32 tensor, pre-uploaded at epoch start]

Phase 2: GPU Portfolio Simulator

New CUDA kernel portfolio_sim_kernel replaces PortfolioTracker::execute_action(). Sequential per trial (inherent dependency), parallel across 5 hyperopt trials via CUDA streams.

__global__ void portfolio_sim_kernel(
    const float* prices,        // [num_bars, 4]
    const int* actions,         // [num_bars]
    float* portfolio_state,     // [5] per trial
    float* portfolio_features,  // [num_bars, 3] output
    float initial_capital, float avg_spread, float cash_reserve_pct,
    int num_bars
);

Phase 3: Fused Experience Collection Kernel

Single CUDA kernel replaces entire inner loop body (state construction + portfolio sim

• barrier tracking + reward calculation):

__global__ void fused_experience_kernel(
    const float* features,         // [num_bars, 51]
    const float* targets,          // [num_bars, 4]
    const int* actions,            // [num_bars]
    float* states_out,             // [num_bars, 54]
    float* next_states_out,        // [num_bars, 54]
    float* rewards_out,            // [num_bars]
    int8_t* barrier_labels_out,    // [num_bars]
    float* portfolio_state,        // [num_trials, 5]
    float* barrier_state,          // [num_trials, 4]
    float* reward_stats,           // [num_trials, 3]
    float initial_capital, float avg_spread,
    int num_bars, int trial_id
);

Phase 4: GPU Backtest

CUDA kernel for EvaluationEngine::process_bar(): position tracking in registers, P&L per bar, aggregate metrics via atomic reduction.

VRAM Budget (L4 24GB)

Buffer	Size (5 trials × 300K bars)
Features + Targets	330 MB
States + Actions + Rewards	556 MB
Replay buffer	220 MB
DQN networks (5 × online+target)	~50 MB
Total	~1.2 GB

Implementation

• New module: crates/ml/src/cuda/ (parallel to liquid/cuda/)
• Kernel compilation: NVRTC at runtime (same as liquid/cuda/mod.rs)
• cudarc: already transitive dep via candle-core
• Feature gate: cuda-pipeline, CPU path always compiles

Expected Performance

Metric	Before (Phase 0)	After (Phase 3)
GPU utilization	~15-20%	70-85%
Per-trial speed	~40 min	~8-12 min
5-trial hyperopt	~40 min	~15-20 min

PPO Compatibility

PPO's hot loop is already f32-native (no Decimal). PPO benefits from Phase 1 (pre-upload) and Phase 3 (batched tensor construction replacing per-step Tensor::from_vec(state.clone(), ...)).