docs: add dynamic GPU detection design plan

Centralizes GPU device selection, capability detection, and per-model
batch size optimization to replace hardcoded RTX 3050 Ti limits.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

docs/plans/2026-02-23-dynamic-gpu-detection-design.md

@@ -0,0 +1,213 @@
+# Dynamic GPU Detection & Validation Design
+
+**Date**: 2026-02-23
+**Status**: Approved
+**Goal**: Auto-detect GPU capabilities and dynamically adjust training parameters (batch size, device selection) so the system scales from a 4GB dev GPU to production-class hardware without hardcoded limits.
+
+## Problem
+
+- 10 ML model adapters each inline `Device::cuda_if_available(0).unwrap_or(Device::Cpu)` — no centralization
+- Batch size hardcoded to 230 (PPO trainer, hyperopt campaign) assuming RTX 3050 Ti 4GB
+- `AutoBatchSizer` with `detect_gpu_memory()` exists but isn't wired to trainers
+- `DeviceConfig` enum exists in `liquid/candle_cfc.rs` but only used by Liquid CfC
+- Production GPUs (A100/H100/RTX 4090) would waste 90%+ of VRAM with the 230 cap
+- No startup validation — GPU failures surface mid-training instead of at service boot
+
+## Approach: Combined A+B+C
+
+Three focused structs, each doing one thing:
+
+### 1. `DeviceConfig` (relocated)
+
+**From**: `ml/src/liquid/candle_cfc.rs`
+**To**: `ml/src/gpu/mod.rs`
+
+Existing enum — resolves config to a `candle_core::Device`:
+
+```rust
+pub enum DeviceConfig {
+    Cpu,
+    Cuda(usize),
+    Auto,
+}
+
+impl DeviceConfig {
+    pub fn resolve(&self) -> Result<Device, MLError>;
+}
+```
+
+Liquid CfC re-imports from new location. No behavior change.
+
+### 2. `GpuCapabilities` (new)
+
+**File**: `ml/src/gpu/capabilities.rs`
+
+One-shot hardware detection. No logic beyond detection:
+
+```rust
+pub struct GpuCapabilities {
+    pub device_name: String,
+    pub total_vram_mb: f64,
+    pub free_vram_mb: f64,
+    pub is_cuda: bool,
+}
+
+impl GpuCapabilities {
+    /// Detect via nvidia-smi. Returns CPU fallback if unavailable.
+    pub fn detect() -> Self;
+    /// Explicit CPU-only mode.
+    pub fn cpu_fallback() -> Self;
+}
+```
+
+Reuses `detect_gpu_memory()` from `memory_optimization/auto_batch_size.rs` internally.
+
+### 3. `ModelMemoryEstimate` (new)
+
+**File**: `ml/src/gpu/memory_profile.rs`
+
+Simple data struct — each model provides one:
+
+```rust
+pub struct ModelMemoryEstimate {
+    pub param_count: usize,
+    pub activation_multiplier: f64,     // 1.0 simple, 2.0+ attention
+    pub supports_checkpointing: bool,
+}
+```
+
+### 4. `resolve_batch_size` (new function)
+
+**File**: `ml/src/gpu/memory_profile.rs`
+
+Stateless function, delegates to existing `AutoBatchSizer` math:
+
+```rust
+pub fn resolve_batch_size(
+    capabilities: &GpuCapabilities,
+    estimate: &ModelMemoryEstimate,
+    config: &BatchSizeConfig,
+) -> usize;
+```
+
+Returns auto-shrunk batch size + logs warning if shrunk.
+
+## Module Structure
+
+```
+ml/src/gpu/
+├── mod.rs              // DeviceConfig enum + re-exports
+├── capabilities.rs     // GpuCapabilities detection
+└── memory_profile.rs   // ModelMemoryEstimate + resolve_batch_size()
+```
+
+## Integration Points
+
+### Trainable Adapters (10 models)
+
+Replace inline device selection:
+```rust
+// Before (scattered in 8+ files):
+Device::cuda_if_available(0).unwrap_or(Device::Cpu)
+
+// After:
+DeviceConfig::Auto.resolve()?
+```
+
+Files: `ensemble/adapters/{dqn,ppo,tft,mamba2,liquid}.rs`, `trainers/ppo.rs`
+
+### PPO Trainer (`trainers/ppo.rs`)
+
+Replace hardcoded batch limit:
+```rust
+// Before:
+if use_gpu && hyperparams.batch_size > 230 { ... }
+
+// After:
+let caps = GpuCapabilities::detect();
+let max_batch = resolve_batch_size(&caps, &ppo_estimate, &config);
+if hyperparams.batch_size > max_batch {
+    warn!("Batch size {} exceeds GPU capacity, shrinking to {}", ...);
+    hyperparams.batch_size = max_batch;
+}
+```
+
+### Hyperopt Campaign (`hyperopt/campaign.rs`)
+
+Replace hardcoded 230:
+```rust
+// Before:
+max_batch_size: 230, // RTX 3050 Ti 4GB VRAM
+
+// After:
+let caps = GpuCapabilities::detect();
+let max_batch = resolve_batch_size(&caps, &model_estimate, &default_config);
+// max_batch_size is now hardware-appropriate
+```
+
+### Inference Engine (`inference.rs`)
+
+Replace string-based device preference:
+```rust
+// Before:
+device_preference: "cuda".to_string()
+
+// After:
+device_config: DeviceConfig::Auto
+```
+
+### Liquid CfC (`liquid/candle_cfc.rs`)
+
+Remove `DeviceConfig` definition, re-import:
+```rust
+// Before:
+pub enum DeviceConfig { Cpu, Cuda(usize), Auto }
+
+// After:
+pub use crate::gpu::DeviceConfig;
+```
+
+### gRPC Training Service
+
+`StartTrainingRequest.use_gpu` validated against `GpuCapabilities::detect()`:
+- GPU requested + available → proceed
+- GPU requested + unavailable → fail fast with clear error
+- GPU not requested → CPU mode
+
+## Per-Model Memory Estimates
+
+| Model | Est. Params | Activation Mult. | Checkpointing | Notes |
+|-------|------------|-------------------|---------------|-------|
+| DQN | ~50K | 1.0 | No | Simple MLP, low memory |
+| PPO | ~100K | 1.0 | No | Policy + Value networks |
+| TFT | ~2M | 2.5 | Yes | Multi-head attention |
+| Mamba2 | ~500K | 1.5 | No | SSM selective scan |
+| TGGN | ~200K | 1.5 | No | Graph attention |
+| TLOB | ~300K | 2.0 | No | LOB attention layers |
+| LNN/Liquid | ~100K | 1.0 | No | ODE-based, low memory |
+| KAN | ~150K | 1.0 | No | B-spline activations |
+| xLSTM | ~400K | 1.5 | No | sLSTM + mLSTM blocks |
+| Diffusion | ~1M | 2.0 | Yes | UNet + noise schedule |
+
+These are estimates — exact values will be derived from model architecture configs.
+
+## Fallback Behavior
+
+1. GPU detected → use GPU, compute per-model batch sizes
+2. nvidia-smi unavailable → fall back to CPU, use conservative batch sizes
+3. Batch size exceeds VRAM → auto-shrink + `warn!()` log
+4. VRAM insufficient for min batch (1) → error with actionable message
+
+## What This Does NOT Include
+
+- Multi-GPU support (single GPU only, extend later)
+- Runtime VRAM monitoring during training (one-shot detection at start)
+- Mixed precision (FP16/BF16) — future enhancement
+- Model-specific CUDA kernels (Liquid CfC already has these separately)
+
+## Testing
+
+- Unit tests with `GpuCapabilities::cpu_fallback()` and `with_manual_memory()`
+- Test auto-shrink behavior: request batch_size=512 on 4GB GPU → verify shrink
+- Test each model's `ModelMemoryEstimate` produces reasonable batch sizes
+- All tests run on CPU (no GPU required for CI)