# Batch Tuning Quick Reference **Status**: Implementation Complete (Integration Pending) --- ## What is Batch Tuning? Automated multi-model hyperparameter optimization that: - Tunes 2-6 models sequentially (DQN, PPO, MAMBA_2, TFT, TLOB, LIQUID) - Resolves dependencies automatically (e.g., TFT requires MAMBA_2) - Exports best hyperparameters to `ml/config/best_hyperparameters.yaml` - Generates consolidated report comparing all models --- ## TLI Commands (After Integration) ### Start Batch Job ```bash # Basic usage - 2 models tli tune batch start --models DQN,PPO --trials 50 # All 4 trainable models (6-8 hours) tli tune batch start --models DQN,PPO,MAMBA_2,TFT --trials 50 # Custom YAML export path tli tune batch start --models DQN,PPO --trials 20 --yaml-export /custom/path.yaml # Disable auto-export tli tune batch start --models DQN,PPO --trials 10 --no-auto-export ``` ### Check Status ```bash tli tune batch status --batch-id ``` ### Get Report ```bash # Print to terminal tli tune batch report --batch-id # Save to file tli tune batch report --batch-id > report.txt ``` ### Export YAML Manually ```bash tli tune batch export --batch-id --output best_params.yaml ``` ### Stop Running Job ```bash tli tune batch stop --batch-id --reason "Sufficient trials completed" ``` --- ## Model Dependencies | Model | Depends On | Reason | |-------|------------|--------| | DQN | - | Independent | | PPO | - | Independent | | MAMBA_2 | - | Independent | | TFT | MAMBA_2 | Uses MAMBA-2 features/embeddings | | TLOB | - | Independent (inference-only) | | LIQUID | - | Independent | **Execution Order Example**: ``` Input: ["TFT", "DQN", "MAMBA_2", "PPO"] Output: ["DQN", "PPO", "MAMBA_2", "TFT"] ↑ ↑ ↑ Independent Must run Depends on (parallel OK) before TFT MAMBA_2 ``` --- ## Time Estimates (RTX 3050 Ti) ### Per Model (50 trials) - **DQN**: 2-3 hours - **PPO**: 2-3 hours - **MAMBA_2**: 3-5 hours (memory-intensive) - **TFT**: 4-6 hours (large model) - **LIQUID**: 1-2 hours (lightweight) ### Batch Jobs | Models | Trials/Model | Total Time | |--------|--------------|------------| | DQN + PPO | 50 | 4-6 hours | | DQN + PPO | 20 | 2-3 hours | | ALL 4 (DQN, PPO, MAMBA_2, TFT) | 50 | 12-18 hours | | ALL 4 | 20 | 5-8 hours | **Recommendation**: Start with 10-20 trials for initial testing --- ## YAML Export Format **File**: `ml/config/best_hyperparameters.yaml` ```yaml # Best Hyperparameters from Batch Tuning # Batch ID: 550e8400-e29b-41d4-a716-446655440000 # Generated: 2025-10-15T14:30:00Z models: DQN: hyperparameters: learning_rate: 0.001 batch_size: 128 replay_buffer_size: 100000 gamma: 0.99 metrics: sharpe_ratio: 1.850000 training_loss: 0.042000 PPO: hyperparameters: learning_rate: 0.0005 batch_size: 256 clip_ratio: 0.2 gae_lambda: 0.95 metrics: sharpe_ratio: 2.100000 training_loss: 0.038000 MAMBA_2: hyperparameters: learning_rate: 0.0001 batch_size: 32 hidden_dim: 256 state_size: 16 metrics: sharpe_ratio: 2.200000 training_loss: 0.035000 TFT: hyperparameters: learning_rate: 0.0001 batch_size: 64 hidden_dim: 128 num_heads: 8 metrics: sharpe_ratio: 2.350000 training_loss: 0.032000 ``` --- ## Consolidated Report Sample ``` ╔════════════════════════════════════════════════════════════════╗ ║ BATCH TUNING CONSOLIDATED REPORT ║ ╚════════════════════════════════════════════════════════════════╝ Batch ID: 550e8400-e29b-41d4-a716-446655440000 Status: Completed Started: 2025-10-15 06:00:00 UTC Completed: 2025-10-15 14:30:00 UTC Duration: 510 minutes (8.5 hours) Models Tuned: 4 Trials per Model: 50 ═══════════════════════════════════════════════════════════════ MODEL COMPARISON ═══════════════════════════════════════════════════════════════ ┌──────────┬──────────────┬────────────────┬────────────────┐ │ Model │ Sharpe Ratio │ Training Loss │ Duration (min) │ ├──────────┼──────────────┼────────────────┼────────────────┤ │ DQN │ 1.8500 │ 0.042000 │ 120 │ │ PPO │ 2.1000 │ 0.038000 │ 135 │ │ MAMBA_2 │ 2.2000 │ 0.035000 │ 180 │ │ TFT │ 2.3500 │ 0.032000 │ 210 │ └──────────┴──────────────┴────────────────┴────────────────┘ 🏆 RECOMMENDATION Best Overall Model: TFT (Sharpe Ratio: 2.3500) Use these hyperparameters for production deployment. 📄 YAML exported to: ml/config/best_hyperparameters.yaml ``` --- ## Architecture ``` TLI Client │ │ tli tune batch start --models DQN,PPO ▼ API Gateway (port 50051) │ │ BatchStartTuningJobs gRPC ▼ ML Training Service (port 50054) │ └─ BatchTuningManager │ ├─ Dependency Resolver │ (TFT → MAMBA_2) │ ├─ Sequential Executor │ │ │ ├─ DQN: TuningManager (50 trials) │ ├─ PPO: TuningManager (50 trials) │ ├─ MAMBA_2: TuningManager (50 trials) │ └─ TFT: TuningManager (50 trials) │ ├─ YAML Exporter │ (ml/config/best_hyperparameters.yaml) │ └─ Report Generator (comparison + recommendation) ``` --- ## Implementation Status ### Completed ✅ - [x] Proto definition (3 gRPC methods) - [x] BatchTuningManager (550+ lines) - [x] Dependency resolution (topological sort) - [x] YAML auto-export - [x] Consolidated reporting - [x] 10 TDD tests ### Pending 🔲 - [ ] gRPC handlers (Agent 165) - [ ] TLI commands (Agent 164) - [ ] Proto code regeneration - [ ] E2E test (2 models, 10 trials) --- ## Files ### Core Implementation - `services/ml_training_service/src/batch_tuning_manager.rs` (550+ lines) - `services/ml_training_service/proto/ml_training.proto` (75 new lines) - `services/ml_training_service/tests/batch_tuning_tests.rs` (450+ lines) ### TLI Integration (TODO) - `tli/src/commands/tune_batch.rs` (NEW) - `tli/proto/ml_training.proto` (regenerate from service proto) ### Documentation - `AGENT_163_BATCH_TUNING_TDD.md` (comprehensive guide) - `BATCH_TUNING_QUICK_REFERENCE.md` (this file) --- ## Usage Tips ### 1. Start Small ```bash # Test with 2 models, 10 trials (1-2 hours) tli tune batch start --models DQN,PPO --trials 10 ``` ### 2. Monitor Progress ```bash # Check status every 15 minutes watch -n 900 tli tune batch status --batch-id ``` ### 3. Analyze Results ```bash # Get report after completion tli tune batch report --batch-id # Inspect YAML cat ml/config/best_hyperparameters.yaml ``` ### 4. Use Best Params ```bash # Copy best params to training config cp ml/config/best_hyperparameters.yaml ml/config/production_params.yaml # Start production training with optimized params tli train start --model TFT --config production_params.yaml ``` --- ## Troubleshooting ### Batch Job Stuck ```bash # Check ML Training Service logs docker-compose logs -f ml_training_service # Check individual tuning job tli tune status --job-id ``` ### YAML Not Exported ```bash # Export manually tli tune batch export --batch-id --output best_params.yaml ``` ### Dependency Error ```bash # If TFT starts before MAMBA_2 (should never happen): # 1. Check BatchTuningManager.resolve_model_dependencies() # 2. File bug report with batch_id ``` --- ## Performance Optimization ### Reduce Trial Count ```bash # Use 20-30 trials for faster results (trade-off: may miss optimal params) tli tune batch start --models DQN,PPO --trials 20 ``` ### Selective Model Tuning ```bash # Only tune models you actually need tli tune batch start --models PPO # Single model (not batch, use `tli tune start`) tli tune batch start --models DQN,PPO # Two models (batch) ``` ### Resume Failed Batch ```bash # If batch fails at MAMBA_2, manually start remaining models: tli tune start --model MAMBA_2 --trials 50 tli tune start --model TFT --trials 50 # Then manually combine results into YAML ``` --- ## Best Practices 1. **Start with 10-20 trials** for initial testing 2. **Monitor GPU temperature** during long batch jobs (nvidia-smi) 3. **Save batch_id** for later reference 4. **Review consolidated report** before deploying best params 5. **Validate best params** with backtesting before production --- ## FAQ **Q: Can I run multiple batch jobs in parallel?** A: No, GPU memory limitations. Queue second batch after first completes. **Q: What if one model fails?** A: Batch continues with remaining models. Final status: PartiallyCompleted. **Q: Can I change execution order?** A: No, order is determined by dependency resolution. Edit MODEL_DEPENDENCIES in code if needed. **Q: How to stop a batch job?** A: `tli tune batch stop --batch-id --reason "User request"` **Q: Where are checkpoints stored?** A: `/tmp/tuning_jobs////` **Q: How to retry a failed model?** A: Use single-model tuning: `tli tune start --model --trials 50` --- ## Next Steps 1. **Test with 2 models** (DQN, PPO, 10 trials, ~2 hours) 2. **Review YAML export** format and accuracy 3. **Validate consolidated report** recommendations 4. **Scale to 4 models** (50 trials, 12-18 hours) after validation 5. **Deploy best params** to production training config --- **Quick Reference Version**: 1.0 (2025-10-15)