foxhunt/docs/guides/QUICK_START_TUNING.md

Quick Start: Hyperparameter Tuning

Time to Complete: 4-8 hours (50 trials) Prerequisites: Trained baseline model, ML Training Service running Goal: Find optimal hyperparameters for 10-20% performance improvement

---

What is Hyperparameter Tuning?

Problem: Default hyperparameters are rarely optimal

• Learning rate too high → unstable training
• Batch size too small → slow convergence
• Hidden layers wrong size → underfitting/overfitting

Solution: Automated search (Optuna) to find best configuration

• Objective: Maximize Sharpe ratio (risk-adjusted returns)
• Method: Bayesian optimization (smart search, not brute force)
• Time: 5-10 minutes per trial × 50 trials = 4-8 hours

Expected Improvement:

• Baseline Sharpe: 1.5
• Tuned Sharpe: 1.7-2.0 (10-30% improvement)

---

Step 1: Prerequisites (5 minutes)

Services Running

# Check services
docker-compose ps

# Should be running:
# - postgres (Optuna study storage)
# - ml_training_service
# - api_gateway

Baseline Model

# List trained models
tli checkpoints list --model DQN

# You should have at least one checkpoint
# If not, train baseline first: see QUICK_START_TRAINING.md

---

Step 2: Review Tuning Configuration (2 minutes)

Check Search Space

cat tuning_config.yaml

Example DQN Configuration:

dqn:
  learning_rate:
    type: loguniform
    low: 1.0e-5
    high: 1.0e-2

  batch_size:
    type: categorical
    choices: [32, 64, 128, 256]

  gamma:
    type: uniform
    low: 0.95
    high: 0.999

  hidden_size:
    type: categorical
    choices: [128, 256, 512]

  num_layers:
    type: int
    low: 2
    high: 4

Understand Parameters

Parameter	Range	Impact
learning_rate	1e-5 to 1e-2	Training speed/stability
batch_size	32-256	Memory usage, convergence
gamma	0.95-0.999	Future reward discount
hidden_size	128-512	Model capacity
num_layers	2-4	Model depth

---

Step 3: Start Tuning Job (1 minute)

Basic Tuning

tli tune start --model DQN --trials 50

Advanced Tuning (Recommended)

tli tune start \
  --model DQN \
  --trials 50 \
  --watch \
  --symbol ES.FUT \
  --epochs 100

Options:

• --trials: Number of hyperparameter combinations to test
• --watch: Stream progress updates in real-time
• --symbol: Training symbol (default: ES.FUT)
• --epochs: Epochs per trial (default: 100)

Expected Output

Tuning job started: job-id=a1b2c3d4-e5f6-7890-abcd-ef1234567890
Study: dqn-tuning-20251014-153045
Trials: 0/50 complete
Best Sharpe: N/A (waiting for first trial)
ETA: 4-8 hours

Use 'tli tune status --job-id a1b2c3d4...' to check progress

---

Step 4: Monitor Progress (Active Monitoring)

Check Status

tli tune status --job-id <job-id>

Output:

Study: dqn-tuning-20251014-153045
Status: RUNNING
Trials: 12/50 complete (24%)
Duration: 1h 23m (elapsed)
ETA: 4h 37m (remaining)

Current Best Trial:
  Trial #7
  Sharpe Ratio: 1.82
  Parameters:
    learning_rate: 0.000234
    batch_size: 128
    gamma: 0.985
    hidden_size: 256
    num_layers: 3

Watch Live Updates

tli tune status --job-id <job-id> --watch

Live Output:

Trial 13/50: Sharpe 1.65 | LR=0.0005 BS=64 Gamma=0.99 HS=128 Layers=2
Trial 14/50: Sharpe 1.78 | LR=0.0002 BS=128 Gamma=0.985 HS=256 Layers=3
Trial 15/50: Sharpe 1.45 | LR=0.001 BS=32 Gamma=0.95 HS=512 Layers=4
...

---

Step 5: Analyze Results (10 minutes)

Get Best Hyperparameters

tli tune best --job-id <job-id>

Output:

{
  "study": "dqn-tuning-20251014-153045",
  "best_trial": 7,
  "best_value": 1.82,
  "best_params": {
    "learning_rate": 0.000234,
    "batch_size": 128,
    "gamma": 0.985,
    "hidden_size": 256,
    "num_layers": 3
  },
  "improvement": {
    "baseline_sharpe": 1.50,
    "tuned_sharpe": 1.82,
    "improvement_pct": 21.3
  },
  "training_metrics": {
    "final_loss": 0.0234,
    "total_reward": 18450.5,
    "win_rate": 0.612
  }
}

Compare with Baseline

# Baseline model
tli checkpoints info --checkpoint-id <baseline-checkpoint>

# Tuned model
tli checkpoints info --checkpoint-id <tuned-checkpoint>

Comparison:

Metric	Baseline	Tuned	Improvement
Sharpe Ratio	1.50	1.82	+21.3%
Win Rate	56.2%	61.2%	+5.0%
Max Drawdown	14.8%	11.2%	-24.3%

---

Step 6: Retrain with Best Hyperparameters (2-3 days)

Create Custom Config

cat > dqn_tuned_config.yaml << EOF
model: DQN
symbol: ES.FUT
epochs: 200
hyperparameters:
  learning_rate: 0.000234
  batch_size: 128
  gamma: 0.985
  hidden_size: 256
  num_layers: 3
EOF

Train Optimized Model

tli train start --config dqn_tuned_config.yaml

Monitor Training

tli train status --job-id <train-job-id> --watch

---

Step 7: Validate Tuned Model (1 hour)

Run Backtest

tli backtest run \
  --strategy dqn_strategy \
  --symbol ES.FUT \
  --start 2024-01-01 \
  --end 2024-12-31 \
  --checkpoint-id <tuned-checkpoint-id>

Expected Results

Backtest Complete:
  Strategy: dqn_strategy (tuned)
  Period: 2024-01-01 to 2024-12-31

  Performance:
    Sharpe Ratio: 1.85
    Win Rate: 61.8%
    Max Drawdown: 10.8%
    Total PnL: $165,230
    Trades: 1,342

  Improvement over Baseline:
    Sharpe: +23.3%
    Win Rate: +5.6%
    Drawdown: -27.0%
    PnL: +31.5%

---

Advanced Tuning Strategies

Multi-Model Tuning

# Tune all models in parallel
tli tune start --model DQN --trials 50 &
tli tune start --model PPO --trials 50 &
tli tune start --model MAMBA2 --trials 50 &
tli tune start --model TFT --trials 50 &

# Wait for all jobs to complete (12-24 hours)

Multi-Symbol Tuning

# Find hyperparameters that work across symbols
tli tune start \
  --model DQN \
  --trials 50 \
  --symbols ES.FUT,NQ.FUT,ZN.FUT,6E.FUT

# This tests generalization across markets

Warm Start (Continue Tuning)

# If tuning interrupted or want more trials
tli tune start \
  --model DQN \
  --trials 50 \
  --study-name dqn-tuning-20251014-153045  # Reuse existing study

# Optuna will resume from last trial

---

Troubleshooting

Trial Failures

# Check logs
docker-compose logs -f ml_training_service

# Common causes:
# - OOM (reduce batch_size range in config)
# - NaN loss (reduce learning_rate upper bound)
# - Timeout (increase epochs per trial)

Slow Tuning

# Speed up by reducing epochs per trial
tli tune start --model DQN --trials 50 --epochs 50

# Trade-off: Faster tuning but less accurate Sharpe estimates

Poor Results (No Improvement)

# Expand search space in tuning_config.yaml
learning_rate:
  low: 1.0e-6    # Was 1.0e-5
  high: 5.0e-2   # Was 1.0e-2

# Try more trials
tli tune start --model DQN --trials 100  # Was 50

Out of Memory

# Reduce batch_size range
batch_size:
  choices: [16, 32, 64]  # Was [32, 64, 128, 256]

# Or reduce hidden_size range
hidden_size:
  choices: [64, 128, 256]  # Was [128, 256, 512]

---

Best Practices

Trial Count

• Quick test: 10-20 trials (1-2 hours)
• Standard: 50 trials (4-8 hours)
• Thorough: 100 trials (8-16 hours)
• Research: 200+ trials (16-32 hours)

Early Stopping

# Optuna MedianPruner automatically stops poor trials
# Saves 30-50% time by killing obviously bad hyperparameters

# Check pruned trials
tli tune status --job-id <job-id> --show-pruned

Study Persistence

# All studies saved to PostgreSQL (JournalStorage)
# Can resume anytime, even after service restart

# List all studies
tli tune list

# Resume specific study
tli tune start --study-name <study-name> --trials 50

---

Next Steps

Ensemble Tuning

# After tuning individual models, optimize ensemble weights
# See: /home/jgrusewski/Work/foxhunt/ENSEMBLE_WEIGHT_OPTIMIZATION_QUICKSTART.md

tli ensemble optimize \
  --models DQN,PPO,MAMBA2,TFT \
  --trials 100

Production Deployment

# Deploy tuned model to paper trading
# See: /home/jgrusewski/Work/foxhunt/PAPER_TRADING_DEPLOYMENT_PLAN.md

# Expected: Sharpe > 1.8 in live conditions

---

Key Resources

Tuning Documentation

• Optuna Tuning Integration Report - Full implementation (26.8K)
• MAMBA-2 Tuning Report - Model-specific tuning
• Tuning Quickstart Guide - Quick reference

ML Training

• ML Training Roadmap - Overall training plan
• Quick Start: Training - Train baseline model

---

Success Metrics

Tuning Success

• ✅ 50 trials complete without failures
• ✅ Best Sharpe > baseline + 10%
• ✅ Improvement consistent across validation periods

Model Quality

• ✅ Tuned Sharpe ratio > 1.8
• ✅ Win rate > 60%
• ✅ Max drawdown < 12%

Production Ready

• ✅ Backtest validates tuning results
• ✅ Paper trading confirms improvement
• ✅ Consistent performance for 2-4 weeks

---

Estimated Time:

• Configuration: 5 minutes
• Tuning job: 4-8 hours
• Analysis: 10 minutes
• Retrain: 2-3 days
• Validation: 1 hour

Total: ~3-4 days from start to validated tuned model

Next Guide: Quick Start: Ensemble Deployment