foxhunt/BATCH_TUNING_QUICK_REFERENCE.md

# Batch Tuning Quick Reference

**Status**: Implementation Complete (Integration Pending)

---

## What is Batch Tuning?

Automated multi-model hyperparameter optimization that:
- Tunes 2-6 models sequentially (DQN, PPO, MAMBA_2, TFT, TLOB, LIQUID)
- Resolves dependencies automatically (e.g., TFT requires MAMBA_2)
- Exports best hyperparameters to `ml/config/best_hyperparameters.yaml`
- Generates consolidated report comparing all models

---

## TLI Commands (After Integration)

### Start Batch Job
```bash
# Basic usage - 2 models
tli tune batch start --models DQN,PPO --trials 50

# All 4 trainable models (6-8 hours)
tli tune batch start --models DQN,PPO,MAMBA_2,TFT --trials 50

# Custom YAML export path
tli tune batch start --models DQN,PPO --trials 20 --yaml-export /custom/path.yaml

# Disable auto-export
tli tune batch start --models DQN,PPO --trials 10 --no-auto-export
```

### Check Status
```bash
tli tune batch status --batch-id <uuid>
```

### Get Report
```bash
# Print to terminal
tli tune batch report --batch-id <uuid>

# Save to file
tli tune batch report --batch-id <uuid> > report.txt
```

### Export YAML Manually
```bash
tli tune batch export --batch-id <uuid> --output best_params.yaml
```

### Stop Running Job
```bash
tli tune batch stop --batch-id <uuid> --reason "Sufficient trials completed"
```

---

## Model Dependencies

| Model | Depends On | Reason |
|-------|------------|--------|
| DQN | - | Independent |
| PPO | - | Independent |
| MAMBA_2 | - | Independent |
| TFT | MAMBA_2 | Uses MAMBA-2 features/embeddings |
| TLOB | - | Independent (inference-only) |
| LIQUID | - | Independent |

**Execution Order Example**:
```
Input:  ["TFT", "DQN", "MAMBA_2", "PPO"]
Output: ["DQN", "PPO", "MAMBA_2", "TFT"]
        ↑             ↑             ↑
    Independent    Must run    Depends on
    (parallel OK)   before TFT   MAMBA_2
```

---

## Time Estimates (RTX 3050 Ti)

### Per Model (50 trials)
- **DQN**: 2-3 hours
- **PPO**: 2-3 hours
- **MAMBA_2**: 3-5 hours (memory-intensive)
- **TFT**: 4-6 hours (large model)
- **LIQUID**: 1-2 hours (lightweight)

### Batch Jobs
| Models | Trials/Model | Total Time |
|--------|--------------|------------|
| DQN + PPO | 50 | 4-6 hours |
| DQN + PPO | 20 | 2-3 hours |
| ALL 4 (DQN, PPO, MAMBA_2, TFT) | 50 | 12-18 hours |
| ALL 4 | 20 | 5-8 hours |

**Recommendation**: Start with 10-20 trials for initial testing

---

## YAML Export Format

**File**: `ml/config/best_hyperparameters.yaml`

```yaml
# Best Hyperparameters from Batch Tuning
# Batch ID: 550e8400-e29b-41d4-a716-446655440000
# Generated: 2025-10-15T14:30:00Z

models:
  DQN:
    hyperparameters:
      learning_rate: 0.001
      batch_size: 128
      replay_buffer_size: 100000
      gamma: 0.99
    metrics:
      sharpe_ratio: 1.850000
      training_loss: 0.042000

  PPO:
    hyperparameters:
      learning_rate: 0.0005
      batch_size: 256
      clip_ratio: 0.2
      gae_lambda: 0.95
    metrics:
      sharpe_ratio: 2.100000
      training_loss: 0.038000

  MAMBA_2:
    hyperparameters:
      learning_rate: 0.0001
      batch_size: 32
      hidden_dim: 256
      state_size: 16
    metrics:
      sharpe_ratio: 2.200000
      training_loss: 0.035000

  TFT:
    hyperparameters:
      learning_rate: 0.0001
      batch_size: 64
      hidden_dim: 128
      num_heads: 8
    metrics:
      sharpe_ratio: 2.350000
      training_loss: 0.032000
```

---

## Consolidated Report Sample

```
╔════════════════════════════════════════════════════════════════╗
║       BATCH TUNING CONSOLIDATED REPORT                        ║
╚════════════════════════════════════════════════════════════════╝

Batch ID: 550e8400-e29b-41d4-a716-446655440000
Status: Completed
Started: 2025-10-15 06:00:00 UTC
Completed: 2025-10-15 14:30:00 UTC
Duration: 510 minutes (8.5 hours)

Models Tuned: 4
Trials per Model: 50

═══════════════════════════════════════════════════════════════
                   MODEL COMPARISON
═══════════════════════════════════════════════════════════════

┌──────────┬──────────────┬────────────────┬────────────────┐
│  Model   │ Sharpe Ratio │ Training Loss  │ Duration (min) │
├──────────┼──────────────┼────────────────┼────────────────┤
│ DQN      │       1.8500 │       0.042000 │            120 │
│ PPO      │       2.1000 │       0.038000 │            135 │
│ MAMBA_2  │       2.2000 │       0.035000 │            180 │
│ TFT      │       2.3500 │       0.032000 │            210 │
└──────────┴──────────────┴────────────────┴────────────────┘

🏆 RECOMMENDATION
   Best Overall Model: TFT (Sharpe Ratio: 2.3500)
   Use these hyperparameters for production deployment.

📄 YAML exported to: ml/config/best_hyperparameters.yaml
```

---

## Architecture

```
TLI Client
    │
    │ tli tune batch start --models DQN,PPO
    ▼
API Gateway (port 50051)
    │
    │ BatchStartTuningJobs gRPC
    ▼
ML Training Service (port 50054)
    │
    └─ BatchTuningManager
           │
           ├─ Dependency Resolver
           │  (TFT → MAMBA_2)
           │
           ├─ Sequential Executor
           │     │
           │     ├─ DQN: TuningManager (50 trials)
           │     ├─ PPO: TuningManager (50 trials)
           │     ├─ MAMBA_2: TuningManager (50 trials)
           │     └─ TFT: TuningManager (50 trials)
           │
           ├─ YAML Exporter
           │  (ml/config/best_hyperparameters.yaml)
           │
           └─ Report Generator
              (comparison + recommendation)
```

---

## Implementation Status

### Completed ✅
- [x] Proto definition (3 gRPC methods)
- [x] BatchTuningManager (550+ lines)
- [x] Dependency resolution (topological sort)
- [x] YAML auto-export
- [x] Consolidated reporting
- [x] 10 TDD tests

### Pending 🔲
- [ ] gRPC handlers (Agent 165)
- [ ] TLI commands (Agent 164)
- [ ] Proto code regeneration
- [ ] E2E test (2 models, 10 trials)

---

## Files

### Core Implementation
- `services/ml_training_service/src/batch_tuning_manager.rs` (550+ lines)
- `services/ml_training_service/proto/ml_training.proto` (75 new lines)
- `services/ml_training_service/tests/batch_tuning_tests.rs` (450+ lines)

### TLI Integration (TODO)
- `tli/src/commands/tune_batch.rs` (NEW)
- `tli/proto/ml_training.proto` (regenerate from service proto)

### Documentation
- `AGENT_163_BATCH_TUNING_TDD.md` (comprehensive guide)
- `BATCH_TUNING_QUICK_REFERENCE.md` (this file)

---

## Usage Tips

### 1. Start Small
```bash
# Test with 2 models, 10 trials (1-2 hours)
tli tune batch start --models DQN,PPO --trials 10
```

### 2. Monitor Progress
```bash
# Check status every 15 minutes
watch -n 900 tli tune batch status --batch-id <uuid>
```

### 3. Analyze Results
```bash
# Get report after completion
tli tune batch report --batch-id <uuid>

# Inspect YAML
cat ml/config/best_hyperparameters.yaml
```

### 4. Use Best Params
```bash
# Copy best params to training config
cp ml/config/best_hyperparameters.yaml ml/config/production_params.yaml

# Start production training with optimized params
tli train start --model TFT --config production_params.yaml
```

---

## Troubleshooting

### Batch Job Stuck
```bash
# Check ML Training Service logs
docker-compose logs -f ml_training_service

# Check individual tuning job
tli tune status --job-id <model-job-uuid>
```

### YAML Not Exported
```bash
# Export manually
tli tune batch export --batch-id <uuid> --output best_params.yaml
```

### Dependency Error
```bash
# If TFT starts before MAMBA_2 (should never happen):
# 1. Check BatchTuningManager.resolve_model_dependencies()
# 2. File bug report with batch_id
```

---

## Performance Optimization

### Reduce Trial Count
```bash
# Use 20-30 trials for faster results (trade-off: may miss optimal params)
tli tune batch start --models DQN,PPO --trials 20
```

### Selective Model Tuning
```bash
# Only tune models you actually need
tli tune batch start --models PPO  # Single model (not batch, use `tli tune start`)
tli tune batch start --models DQN,PPO  # Two models (batch)
```

### Resume Failed Batch
```bash
# If batch fails at MAMBA_2, manually start remaining models:
tli tune start --model MAMBA_2 --trials 50
tli tune start --model TFT --trials 50

# Then manually combine results into YAML
```

---

## Best Practices

1. **Start with 10-20 trials** for initial testing
2. **Monitor GPU temperature** during long batch jobs (nvidia-smi)
3. **Save batch_id** for later reference
4. **Review consolidated report** before deploying best params
5. **Validate best params** with backtesting before production

---

## FAQ

**Q: Can I run multiple batch jobs in parallel?**
A: No, GPU memory limitations. Queue second batch after first completes.

**Q: What if one model fails?**
A: Batch continues with remaining models. Final status: PartiallyCompleted.

**Q: Can I change execution order?**
A: No, order is determined by dependency resolution. Edit MODEL_DEPENDENCIES in code if needed.

**Q: How to stop a batch job?**
A: `tli tune batch stop --batch-id <uuid> --reason "User request"`

**Q: Where are checkpoints stored?**
A: `/tmp/tuning_jobs/<batch_id>/<model_type>/<job_id>/`

**Q: How to retry a failed model?**
A: Use single-model tuning: `tli tune start --model <MODEL> --trials 50`

---

## Next Steps

1. **Test with 2 models** (DQN, PPO, 10 trials, ~2 hours)
2. **Review YAML export** format and accuracy
3. **Validate consolidated report** recommendations
4. **Scale to 4 models** (50 trials, 12-18 hours) after validation
5. **Deploy best params** to production training config

---

**Quick Reference Version**: 1.0 (2025-10-15)