foxhunt/docs/plans/2026-03-01-gpu-blockers-validation-design.md

GPU Blockers Validation — Design

Date: 2026-03-01 Status: Approved Goal: Fix 5 blockers preventing GPU utilization from reaching 80%+ after initial dynamic-sizing changes

Findings Summary

Post-implementation validation of the GPU dynamic-sizing changes (20 files, +353/-84) found 3 critical and 2 important blockers.

Critical Fixes

Fix 1: Wire BF16 Mixed Precision End-to-End

Problem: Detection works (HardwareBudget::detect() → supports_bf16()) but result stored in _mixed_precision_detected (unused). Chain broken at 3 points:

• DQNConfig has no mixed_precision field
• DQNHyperparameters has no mixed_precision field
• DQNAgent::new() hardcodes mixed_precision: None

Fix:

1. Add mixed_precision: Option<MixedPrecisionConfig> to DQNConfig (dqn/dqn.rs)
2. Add mixed_precision: Option<MixedPrecisionConfig> to DQNHyperparameters (trainers/dqn/config.rs)
3. In trainer.rs: rename _mixed_precision_detected → mixed_precision_detected, pass to DQNConfig
4. In DQN::new(): pass config.mixed_precision → QNetworkConfig.mixed_precision
5. In hyperopt train_with_params(): auto-detect and pass mixed precision

Files: dqn/dqn.rs, trainers/dqn/config.rs, trainers/dqn/trainer.rs, dqn/agent.rs, hyperopt/adapters/dqn.rs

Fix 2: Wire hidden_dim_base Through Training + Eval Binaries

Problem: train_baseline_rl.rs hardcodes hidden_dims: vec![128, 64] in both train_dqn_fold() and train_ppo_fold(). Hyperopt discovers optimal hidden_dim_base but it's silently ignored at train time.

Fix:

1. In train_dqn_fold(): read hidden_dim_base from hyperopt JSON, compute [base, base/2, base/4]
2. In train_ppo_fold(): read hidden_dim_base, compute policy [base, base/2] and value [base*2, base, base/2]
3. In evaluate_baseline.rs: same treatment for eval DQN/PPO config construction
4. Fallback to [128, 64] when key absent (backward compat with old hyperopt results)

Files: examples/train_baseline_rl.rs, examples/evaluate_baseline.rs

Fix 3: Reduce GPU Sync Points (5 per step → 1 periodic)

Problem: Four .to_vec1() NaN checks (BUG #14) + one .to_scalar() force GPU queue flush every training step. GPU cannot pipeline.

Fix:

1. Replace per-step NaN detection with periodic check (every 100 steps)
2. Use GPU-side Tensor::any() + Tensor::isnan() to check NaN without CPU transfer
3. Only pull .to_vec1() on the periodic check or when GPU-side NaN detected
4. Keep the .to_scalar() in estimate_avg_q_value but call it every 100 steps instead of every step

Files: dqn/dqn.rs (compute_loss_internal), trainers/dqn/trainer.rs (Q-value monitoring frequency)

Important Fixes

Fix 4: Cache Loaded Data Across Trials

Problem: Every hyperopt trial re-loads 36 .dbn.zst files, decompresses, extracts 51 features. GPU idle for minutes per trial boundary.

Fix:

1. In hyperopt runner, load data once into Arc<Vec<TrainingData>> before trial loop
2. Pass shared reference to each train_with_params() call
3. Add train_with_preloaded_data() variant that accepts pre-loaded data

Files: hyperopt/adapters/dqn.rs, hyperopt/adapters/ppo.rs

Fix 5: Batch Experience Storage (Remove Per-Sample Lock)

Problem: agent.write().await acquired per sample (100K+ per epoch). Serializes experience collection.

Fix:

1. Collect actions + experiences into Vec during batch processing
2. Single agent.write().await to store entire batch at once
3. Move track_action() into batched path

Files: trainers/dqn/trainer.rs (experience collection loop)

Files Changed Summary

File	Fix	Change
dqn/dqn.rs	1,3	Add mixed_precision to DQNConfig, periodic NaN check
trainers/dqn/config.rs	1	Add mixed_precision to DQNHyperparameters
trainers/dqn/trainer.rs	1,5	Wire mixed precision, batch experience storage
dqn/agent.rs	1	Pass config.mixed_precision to QNetworkConfig
hyperopt/adapters/dqn.rs	1,4	Auto-detect AMP, pre-loaded data path
hyperopt/adapters/ppo.rs	4	Pre-loaded data path
examples/train_baseline_rl.rs	2	Read hidden_dim_base from JSON
examples/evaluate_baseline.rs	2	Read hidden_dim_base from JSON

Expected Outcome

With all 5 fixes:

• TENSOR_ACTIVE: 0% → 40-60% (BF16 actually engaged)
• GPU_UTIL: 12% → 60-80% (fewer sync stalls, larger models)
• Trial startup: minutes → seconds (cached data)
• Training step throughput: ~5x (1 sync per 100 steps vs 5 per step)