# GPU Max Performance Phase 2 Design (L4 → H100)

**Goal**: Maximize GPU utilization from ~40% (post-Phase 1) to 80%+ across all GPU tiers (L4 24GB, L40S 48GB, A100 40/80GB, H100 80GB). Covers both training throughput and inference latency.

**Prior work (Phase 1)**: BF16 mixed precision (DQN+PPO), dynamic hidden_dim_base, data caching via Arc, batch experience storage, PPO GAE CPU path. All 5 wired end-to-end, verified by quality audit.

**Framework**: Candle v0.9.1 (cuBLAS + cuDNN, CUDA streams via cudarc 0.17.3).

---

## 1. Dynamic Batch Size Per GPU

**Problem**: DQN hard-caps batch_size at 230 (RTX 3050 Ti legacy). PPO defaults to 64. Neither adapts to available VRAM.

**Design**: At trainer init, query `HardwareBudget::detect()` and compute device-specific batch size. Formula: `(gpu_vram_mb - model_overhead_mb) / mb_per_sample`, clamped to `[64, 8192]`.

| GPU | VRAM | DQN batch | PPO batch |
|---|---|---|---|
| RTX 3050 Ti | 4GB | 230 | 64 |
| L4 | 24GB | 2048 | 512 |
| L40S | 48GB | 4096 | 1024 |
| A100 80GB | 80GB | 8192 | 2048 |
| H100 | 80GB | 8192 | 2048 |

**Files**: `trainers/dqn/trainer.rs`, `trainers/ppo.rs`, `hyperopt/traits.rs`

**Impact**: 8-20% throughput gain on H100 (larger batches = better GPU utilization).

---

## 2. Tensor Core Dimension Alignment

**Problem**: `state_dim = 54` (51 market + 3 portfolio). Not a multiple of 8 — tensor cores on Ampere/Hopper can't fire optimally for the first Linear layer.

**Design**: Pad state_dim from 54 to 56 by appending 2 zero features during tensor construction in the CUDA pipeline's `upload()` and `get_batch()` methods. Hidden dims already default to powers of 2 (256, 128, 64). Enforce rounding to nearest multiple of 8 in `HardwareBudget::max_hidden_dim_base()`.

**Files**: `cuda_pipeline/mod.rs`, `hyperopt/traits.rs`

**Impact**: 2-5% compute throughput (ensures cuBLAS selects tensor core paths).

---

## 3. Inference Without Gradient Tape

**Problem**: `evaluate_baseline.rs` runs forward passes but Candle still builds the computation graph. Wastes ~2x VRAM on activation caches that are never used.

**Design**: Add `Tensor::detach()` on forward pass outputs in evaluation and inference code. Candle has no `torch.no_grad()` equivalent, but detaching outputs prevents gradient accumulation. Key: don't call `.backward()` during inference — the VRAM savings come from not storing intermediate activations for a backward pass that never happens.

**Files**: `examples/evaluate_baseline.rs`, `inference.rs`, `ppo/trainable_adapter.rs`

**Impact**: ~2x VRAM savings during inference, enabling larger batch sizes for evaluation.

---

## 4. CUDA Stream Double-Buffering

**Problem**: Data upload is synchronous. GPU sits idle while CPU prepares the next batch.

**Design**: Use existing `cuda_dev.cuda_stream()` (already accessed in 3 trainer files) to create a secondary stream for data staging.

```
Stream A (compute): forward + backward on current batch
Stream B (staging):  async-copy next batch from host → device
Synchronize at batch boundary, swap streams
```

Wrap this pattern in a `DoubleBufferedLoader` struct that holds two pre-allocated GPU buffers and alternates between them.

**Files**: `cuda_pipeline/mod.rs` (new `DoubleBufferedLoader`), `trainers/dqn/trainer.rs`, `trainers/ppo.rs`

**Impact**: 5-15% training throughput (overlap I/O with compute).

---

## 5. Epoch Data Prefetch

**Problem**: Feature extraction and normalization happen synchronously before each epoch. CPU work blocks GPU.

**Design**: Spawn a background `tokio::spawn` task that computes next epoch's features/normalization stats while the current epoch trains on GPU. Use `tokio::sync::oneshot` channel to pass pre-computed data to the next epoch.

```
Epoch N:   GPU trains  ─────────────────────>
           CPU prefetch (epoch N+1 features) ──>
Epoch N+1: GPU trains (data ready)  ─────────>
```

**Files**: `trainers/dqn/trainer.rs`, `trainers/ppo.rs`

**Impact**: 5-12% training throughput (hidden CPU-side latency behind GPU compute).

---

## 6. INT8 Quantized Inference

**Problem**: 28 quantization files exist in `crates/ml/` (INT8/INT4, QAT, quantized attention/GRN/LSTM). But `QuantizedTFT::from_fp32_model()` has empty scales and zero-points. The skeleton is built, the wiring is missing.

**Design**: Wire existing quantization infrastructure:

1. **Post-training quantization**: After training completes, compute per-layer min/max from weight tensors to derive scale and zero-point.
2. **Quantized checkpoint**: Store INT8 weights + scale/zero-point metadata alongside FP32 checkpoints.
3. **Quantized inference**: Load quantized model, use INT8 matmul for forward passes.
4. **Priority order**: TFT first (most complete skeleton), then DQN, then PPO.

**Files**: `transformers/quantization.rs`, `memory_optimization/quantization.rs`, `tft/hft_optimizations.rs`

**Impact**: 2-4x inference throughput + 4x memory savings for inference workloads.

---

## 7. Memory Pinning for Host-Device Transfers

**Problem**: CPU to GPU transfers use pageable memory (`Tensor::from_vec()` allocates from standard heap). DMA engine can't use the fast pinned-memory path.

**Design**: For the CUDA pipeline's large transfers (`upload()` method), use cudarc's `htod_copy()` which internally uses pinned memory. For small per-step tensors (portfolio features, 3-dim), pre-allocate a pinned buffer pool and reuse it across steps.

**Files**: `cuda_pipeline/mod.rs`

**Impact**: 2-5% host-device transfer latency reduction. Marginal for pre-uploaded data, meaningful for per-step portfolio features.

---

## 8. Custom CUDA Kernel Optimization

**Problem**: Two custom kernels (`dqn_experience_kernel.cu`, `ppo_experience_kernel.cu`) exist but haven't been optimized for L4/H100 SM counts.

**Design**: Audit and optimize:

1. **Block/grid sizing**: Auto-configure based on device SM count (L4: 58 SMs, L40S: 142, A100: 108, H100: 132).
2. **Shared memory**: Use shared memory for reduction operations in experience collection.
3. **Fused ops**: Identify sequential kernel launches that can be merged.
4. **Occupancy**: Target 75%+ occupancy by tuning threads-per-block.

Query SM count at init via `cudarc::driver::CudaDevice::attribute()`.

**Files**: `cuda_kernels/dqn_experience_kernel.cu`, `cuda_kernels/ppo_experience_kernel.cu`, `cuda_pipeline/mod.rs`

**Impact**: 5-10% for custom kernel paths. cuBLAS ops already optimized.

---

## Summary

| # | Optimization | Effort | Impact | Training | Inference |
|---|---|---|---|---|---|
| 1 | Dynamic batch size | Low | 8-20% | Y | Y |
| 2 | Dimension alignment | Low | 2-5% | Y | Y |
| 3 | No-grad inference | Low | 2x VRAM | N | Y |
| 4 | CUDA stream double-buffer | Medium | 5-15% | Y | N |
| 5 | Epoch data prefetch | Medium | 5-12% | Y | N |
| 6 | INT8 quantized inference | High | 2-4x | N | Y |
| 7 | Memory pinning | Medium | 2-5% | Y | Y |
| 8 | CUDA kernel optimization | High | 5-10% | Y | N |

**Combined theoretical gain**: 30-70% training throughput + 2-4x inference speedup.

**Target GPU utilization**: 60-80% sustained (from current ~40% post-Phase 1).

**Non-goals**: Multi-GPU (single GPU per pod, Kapsule quota=1), FP8 (unsupported by Candle).
