2b2ff4ffa5
Wire BF16/FP16 mixed precision end-to-end for DQN and PPO with auto-detection from GPU name (Ampere+ → BF16, Volta/Turing → FP16). Add hidden_dim_base to hyperopt and wire through training/eval binaries. Reduce GPU sync points: make DQN NaN checks periodic (every 100 steps), replace PPO GAE GPU round-trip with pure CPU implementation. Cache training data across hyperopt trials for all 10 models via Arc. Batch DQN experience storage (128x fewer lock acquisitions). Correct VRAM constants and batch bounds for all 9 supervised model adapters. 28 files changed, +1207/-208 lines. 2418 tests pass. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
97 lines
4.5 KiB
Markdown
97 lines
4.5 KiB
Markdown
# GPU Blockers Validation — Design
|
|
|
|
**Date**: 2026-03-01
|
|
**Status**: Approved
|
|
**Goal**: Fix 5 blockers preventing GPU utilization from reaching 80%+ after initial dynamic-sizing changes
|
|
|
|
## Findings Summary
|
|
|
|
Post-implementation validation of the GPU dynamic-sizing changes (20 files, +353/-84) found 3 critical and 2 important blockers.
|
|
|
|
## Critical Fixes
|
|
|
|
### Fix 1: Wire BF16 Mixed Precision End-to-End
|
|
|
|
**Problem**: Detection works (`HardwareBudget::detect()` → `supports_bf16()`) but result stored in `_mixed_precision_detected` (unused). Chain broken at 3 points:
|
|
- `DQNConfig` has no `mixed_precision` field
|
|
- `DQNHyperparameters` has no `mixed_precision` field
|
|
- `DQNAgent::new()` hardcodes `mixed_precision: None`
|
|
|
|
**Fix**:
|
|
1. Add `mixed_precision: Option<MixedPrecisionConfig>` to `DQNConfig` (dqn/dqn.rs)
|
|
2. Add `mixed_precision: Option<MixedPrecisionConfig>` to `DQNHyperparameters` (trainers/dqn/config.rs)
|
|
3. In `trainer.rs`: rename `_mixed_precision_detected` → `mixed_precision_detected`, pass to DQNConfig
|
|
4. In `DQN::new()`: pass `config.mixed_precision` → `QNetworkConfig.mixed_precision`
|
|
5. In hyperopt `train_with_params()`: auto-detect and pass mixed precision
|
|
|
|
**Files**: `dqn/dqn.rs`, `trainers/dqn/config.rs`, `trainers/dqn/trainer.rs`, `dqn/agent.rs`, `hyperopt/adapters/dqn.rs`
|
|
|
|
### Fix 2: Wire `hidden_dim_base` Through Training + Eval Binaries
|
|
|
|
**Problem**: `train_baseline_rl.rs` hardcodes `hidden_dims: vec![128, 64]` in both `train_dqn_fold()` and `train_ppo_fold()`. Hyperopt discovers optimal `hidden_dim_base` but it's silently ignored at train time.
|
|
|
|
**Fix**:
|
|
1. In `train_dqn_fold()`: read `hidden_dim_base` from hyperopt JSON, compute `[base, base/2, base/4]`
|
|
2. In `train_ppo_fold()`: read `hidden_dim_base`, compute policy `[base, base/2]` and value `[base*2, base, base/2]`
|
|
3. In `evaluate_baseline.rs`: same treatment for eval DQN/PPO config construction
|
|
4. Fallback to `[128, 64]` when key absent (backward compat with old hyperopt results)
|
|
|
|
**Files**: `examples/train_baseline_rl.rs`, `examples/evaluate_baseline.rs`
|
|
|
|
### Fix 3: Reduce GPU Sync Points (5 per step → 1 periodic)
|
|
|
|
**Problem**: Four `.to_vec1()` NaN checks (BUG #14) + one `.to_scalar()` force GPU queue flush every training step. GPU cannot pipeline.
|
|
|
|
**Fix**:
|
|
1. Replace per-step NaN detection with periodic check (every 100 steps)
|
|
2. Use GPU-side `Tensor::any()` + `Tensor::isnan()` to check NaN without CPU transfer
|
|
3. Only pull `.to_vec1()` on the periodic check or when GPU-side NaN detected
|
|
4. Keep the `.to_scalar()` in `estimate_avg_q_value` but call it every 100 steps instead of every step
|
|
|
|
**Files**: `dqn/dqn.rs` (`compute_loss_internal`), `trainers/dqn/trainer.rs` (Q-value monitoring frequency)
|
|
|
|
## Important Fixes
|
|
|
|
### Fix 4: Cache Loaded Data Across Trials
|
|
|
|
**Problem**: Every hyperopt trial re-loads 36 `.dbn.zst` files, decompresses, extracts 51 features. GPU idle for minutes per trial boundary.
|
|
|
|
**Fix**:
|
|
1. In hyperopt runner, load data once into `Arc<Vec<TrainingData>>` before trial loop
|
|
2. Pass shared reference to each `train_with_params()` call
|
|
3. Add `train_with_preloaded_data()` variant that accepts pre-loaded data
|
|
|
|
**Files**: `hyperopt/adapters/dqn.rs`, `hyperopt/adapters/ppo.rs`
|
|
|
|
### Fix 5: Batch Experience Storage (Remove Per-Sample Lock)
|
|
|
|
**Problem**: `agent.write().await` acquired per sample (100K+ per epoch). Serializes experience collection.
|
|
|
|
**Fix**:
|
|
1. Collect actions + experiences into `Vec` during batch processing
|
|
2. Single `agent.write().await` to store entire batch at once
|
|
3. Move `track_action()` into batched path
|
|
|
|
**Files**: `trainers/dqn/trainer.rs` (experience collection loop)
|
|
|
|
## Files Changed Summary
|
|
|
|
| File | Fix | Change |
|
|
|------|-----|--------|
|
|
| `dqn/dqn.rs` | 1,3 | Add `mixed_precision` to DQNConfig, periodic NaN check |
|
|
| `trainers/dqn/config.rs` | 1 | Add `mixed_precision` to DQNHyperparameters |
|
|
| `trainers/dqn/trainer.rs` | 1,5 | Wire mixed precision, batch experience storage |
|
|
| `dqn/agent.rs` | 1 | Pass `config.mixed_precision` to QNetworkConfig |
|
|
| `hyperopt/adapters/dqn.rs` | 1,4 | Auto-detect AMP, pre-loaded data path |
|
|
| `hyperopt/adapters/ppo.rs` | 4 | Pre-loaded data path |
|
|
| `examples/train_baseline_rl.rs` | 2 | Read `hidden_dim_base` from JSON |
|
|
| `examples/evaluate_baseline.rs` | 2 | Read `hidden_dim_base` from JSON |
|
|
|
|
## Expected Outcome
|
|
|
|
With all 5 fixes:
|
|
- TENSOR_ACTIVE: 0% → 40-60% (BF16 actually engaged)
|
|
- GPU_UTIL: 12% → 60-80% (fewer sync stalls, larger models)
|
|
- Trial startup: minutes → seconds (cached data)
|
|
- Training step throughput: ~5x (1 sync per 100 steps vs 5 per step)
|