foxhunt/BATCH_TUNING_QUICK_REFERENCE.md

Batch Tuning Quick Reference

Status: Implementation Complete (Integration Pending)

---

What is Batch Tuning?

Automated multi-model hyperparameter optimization that:

• Tunes 2-6 models sequentially (DQN, PPO, MAMBA_2, TFT, TLOB, LIQUID)
• Resolves dependencies automatically (e.g., TFT requires MAMBA_2)
• Exports best hyperparameters to ml/config/best_hyperparameters.yaml
• Generates consolidated report comparing all models

---

TLI Commands (After Integration)

Start Batch Job

# Basic usage - 2 models
tli tune batch start --models DQN,PPO --trials 50

# All 4 trainable models (6-8 hours)
tli tune batch start --models DQN,PPO,MAMBA_2,TFT --trials 50

# Custom YAML export path
tli tune batch start --models DQN,PPO --trials 20 --yaml-export /custom/path.yaml

# Disable auto-export
tli tune batch start --models DQN,PPO --trials 10 --no-auto-export

Check Status

tli tune batch status --batch-id <uuid>

Get Report

# Print to terminal
tli tune batch report --batch-id <uuid>

# Save to file
tli tune batch report --batch-id <uuid> > report.txt

Export YAML Manually

tli tune batch export --batch-id <uuid> --output best_params.yaml

Stop Running Job

tli tune batch stop --batch-id <uuid> --reason "Sufficient trials completed"

---

Model Dependencies

Model	Depends On	Reason
DQN	-	Independent
PPO	-	Independent
MAMBA_2	-	Independent
TFT	MAMBA_2	Uses MAMBA-2 features/embeddings
TLOB	-	Independent (inference-only)
LIQUID	-	Independent

Execution Order Example:

Input:  ["TFT", "DQN", "MAMBA_2", "PPO"]
Output: ["DQN", "PPO", "MAMBA_2", "TFT"]
        ↑             ↑             ↑
    Independent    Must run    Depends on
    (parallel OK)   before TFT   MAMBA_2

---

Time Estimates (RTX 3050 Ti)

Per Model (50 trials)

• DQN: 2-3 hours
• PPO: 2-3 hours
• MAMBA_2: 3-5 hours (memory-intensive)
• TFT: 4-6 hours (large model)
• LIQUID: 1-2 hours (lightweight)

Batch Jobs

Models	Trials/Model	Total Time
DQN + PPO	50	4-6 hours
DQN + PPO	20	2-3 hours
ALL 4 (DQN, PPO, MAMBA_2, TFT)	50	12-18 hours
ALL 4	20	5-8 hours

Recommendation: Start with 10-20 trials for initial testing

---

YAML Export Format

File: ml/config/best_hyperparameters.yaml

# Best Hyperparameters from Batch Tuning
# Batch ID: 550e8400-e29b-41d4-a716-446655440000
# Generated: 2025-10-15T14:30:00Z

models:
  DQN:
    hyperparameters:
      learning_rate: 0.001
      batch_size: 128
      replay_buffer_size: 100000
      gamma: 0.99
    metrics:
      sharpe_ratio: 1.850000
      training_loss: 0.042000

  PPO:
    hyperparameters:
      learning_rate: 0.0005
      batch_size: 256
      clip_ratio: 0.2
      gae_lambda: 0.95
    metrics:
      sharpe_ratio: 2.100000
      training_loss: 0.038000

  MAMBA_2:
    hyperparameters:
      learning_rate: 0.0001
      batch_size: 32
      hidden_dim: 256
      state_size: 16
    metrics:
      sharpe_ratio: 2.200000
      training_loss: 0.035000

  TFT:
    hyperparameters:
      learning_rate: 0.0001
      batch_size: 64
      hidden_dim: 128
      num_heads: 8
    metrics:
      sharpe_ratio: 2.350000
      training_loss: 0.032000

---

Consolidated Report Sample

╔════════════════════════════════════════════════════════════════╗
║       BATCH TUNING CONSOLIDATED REPORT                        ║
╚════════════════════════════════════════════════════════════════╝

Batch ID: 550e8400-e29b-41d4-a716-446655440000
Status: Completed
Started: 2025-10-15 06:00:00 UTC
Completed: 2025-10-15 14:30:00 UTC
Duration: 510 minutes (8.5 hours)

Models Tuned: 4
Trials per Model: 50

═══════════════════════════════════════════════════════════════
                   MODEL COMPARISON
═══════════════════════════════════════════════════════════════

┌──────────┬──────────────┬────────────────┬────────────────┐
│  Model   │ Sharpe Ratio │ Training Loss  │ Duration (min) │
├──────────┼──────────────┼────────────────┼────────────────┤
│ DQN      │       1.8500 │       0.042000 │            120 │
│ PPO      │       2.1000 │       0.038000 │            135 │
│ MAMBA_2  │       2.2000 │       0.035000 │            180 │
│ TFT      │       2.3500 │       0.032000 │            210 │
└──────────┴──────────────┴────────────────┴────────────────┘

🏆 RECOMMENDATION
   Best Overall Model: TFT (Sharpe Ratio: 2.3500)
   Use these hyperparameters for production deployment.

📄 YAML exported to: ml/config/best_hyperparameters.yaml

---

Architecture

TLI Client
    │
    │ tli tune batch start --models DQN,PPO
    ▼
API Gateway (port 50051)
    │
    │ BatchStartTuningJobs gRPC
    ▼
ML Training Service (port 50054)
    │
    └─ BatchTuningManager
           │
           ├─ Dependency Resolver
           │  (TFT → MAMBA_2)
           │
           ├─ Sequential Executor
           │     │
           │     ├─ DQN: TuningManager (50 trials)
           │     ├─ PPO: TuningManager (50 trials)
           │     ├─ MAMBA_2: TuningManager (50 trials)
           │     └─ TFT: TuningManager (50 trials)
           │
           ├─ YAML Exporter
           │  (ml/config/best_hyperparameters.yaml)
           │
           └─ Report Generator
              (comparison + recommendation)

---

Implementation Status

Completed ✅

• Proto definition (3 gRPC methods)
• BatchTuningManager (550+ lines)
• Dependency resolution (topological sort)
• YAML auto-export
• Consolidated reporting
• 10 TDD tests

Pending 🔲

• gRPC handlers (Agent 165)
• TLI commands (Agent 164)
• Proto code regeneration
• E2E test (2 models, 10 trials)

---

Files

Core Implementation

• services/ml_training_service/src/batch_tuning_manager.rs (550+ lines)
• services/ml_training_service/proto/ml_training.proto (75 new lines)
• services/ml_training_service/tests/batch_tuning_tests.rs (450+ lines)

TLI Integration (TODO)

• tli/src/commands/tune_batch.rs (NEW)
• tli/proto/ml_training.proto (regenerate from service proto)

Documentation

• AGENT_163_BATCH_TUNING_TDD.md (comprehensive guide)
• BATCH_TUNING_QUICK_REFERENCE.md (this file)

---

Usage Tips

1. Start Small

# Test with 2 models, 10 trials (1-2 hours)
tli tune batch start --models DQN,PPO --trials 10

2. Monitor Progress

# Check status every 15 minutes
watch -n 900 tli tune batch status --batch-id <uuid>

3. Analyze Results

# Get report after completion
tli tune batch report --batch-id <uuid>

# Inspect YAML
cat ml/config/best_hyperparameters.yaml

4. Use Best Params

# Copy best params to training config
cp ml/config/best_hyperparameters.yaml ml/config/production_params.yaml

# Start production training with optimized params
tli train start --model TFT --config production_params.yaml

---

Troubleshooting

Batch Job Stuck

# Check ML Training Service logs
docker-compose logs -f ml_training_service

# Check individual tuning job
tli tune status --job-id <model-job-uuid>

YAML Not Exported

# Export manually
tli tune batch export --batch-id <uuid> --output best_params.yaml

Dependency Error

# If TFT starts before MAMBA_2 (should never happen):
# 1. Check BatchTuningManager.resolve_model_dependencies()
# 2. File bug report with batch_id

---

Performance Optimization

Reduce Trial Count

# Use 20-30 trials for faster results (trade-off: may miss optimal params)
tli tune batch start --models DQN,PPO --trials 20

Selective Model Tuning

# Only tune models you actually need
tli tune batch start --models PPO  # Single model (not batch, use `tli tune start`)
tli tune batch start --models DQN,PPO  # Two models (batch)

Resume Failed Batch

# If batch fails at MAMBA_2, manually start remaining models:
tli tune start --model MAMBA_2 --trials 50
tli tune start --model TFT --trials 50

# Then manually combine results into YAML

---

Best Practices

1. Start with 10-20 trials for initial testing
2. Monitor GPU temperature during long batch jobs (nvidia-smi)
3. Save batch_id for later reference
4. Review consolidated report before deploying best params
5. Validate best params with backtesting before production

---

FAQ

Q: Can I run multiple batch jobs in parallel? A: No, GPU memory limitations. Queue second batch after first completes.

Q: What if one model fails? A: Batch continues with remaining models. Final status: PartiallyCompleted.

Q: Can I change execution order? A: No, order is determined by dependency resolution. Edit MODEL_DEPENDENCIES in code if needed.

Q: How to stop a batch job? A: tli tune batch stop --batch-id <uuid> --reason "User request"

Q: Where are checkpoints stored? A: /tmp/tuning_jobs/<batch_id>/<model_type>/<job_id>/

Q: How to retry a failed model? A: Use single-model tuning: tli tune start --model <MODEL> --trials 50

---

Next Steps

1. Test with 2 models (DQN, PPO, 10 trials, ~2 hours)
2. Review YAML export format and accuracy
3. Validate consolidated report recommendations
4. Scale to 4 models (50 trials, 12-18 hours) after validation
5. Deploy best params to production training config

---

Quick Reference Version: 1.0 (2025-10-15)