foxhunt/services/ml_training_service/AGENT_49_EXECUTION_GUIDE.md

Agent 49: Hyperparameter Optimization Execution Guide

Date: 2025-10-14 Agent: Agent 49 Task: Run hyperparameter search for all 4 models (DQN, PPO, MAMBA-2, TFT) Method: Grid search + Bayesian optimization (TPE Sampler)

---

📋 Overview

This guide provides step-by-step instructions for executing comprehensive hyperparameter optimization across all 4 ML models in the Foxhunt trading system.

Models to Optimize

1. DQN (Deep Q-Network) - Reinforcement Learning
2. PPO (Proximal Policy Optimization) - Policy Gradient RL
3. MAMBA-2 (State Space Model) - Sequential Modeling
4. TFT (Temporal Fusion Transformer) - Time Series Forecasting

Search Spaces (Agent 49 Specifications)

DQN

• Learning rate: [1e-5, 1e-4, 1e-3]
• Batch size: [64, 128, 256]
• Gamma: [0.95, 0.99, 0.999]
• Grid combinations: 3^3 = 27

PPO

• Learning rate: [3e-5, 1e-4, 3e-4]
• Entropy coefficient: [0.01, 0.05, 0.1]
• Clip range: [0.1, 0.2, 0.3]
• Grid combinations: 3^3 = 27

MAMBA-2

• Learning rate: [1e-5, 1e-4, 1e-3]
• State size: [16, 32, 64]
• Layers: [4, 6, 8]
• Grid combinations: 3^3 = 27

TFT

• Learning rate: [1e-5, 1e-4, 1e-3]
• Attention heads: [4, 8, 16]
• Hidden dimension: [128, 256, 512]
• Grid combinations: 3^3 = 27

---

🔧 Prerequisites

1. Infrastructure Running

# Verify ML Training Service is running
docker-compose ps | grep ml_training

# Expected output:
# foxhunt-ml-training-service   Up (healthy)   0.0.0.0:50054->50053/tcp

2. Test Data Available

# Validate test data
cd /home/jgrusewski/Work/foxhunt/services/ml_training_service
python3 validate_test_data.py /home/jgrusewski/Work/foxhunt/test_data/real/parquet/BTC-USD_30day_2024-09.parquet

Expected output:

✓ File loaded successfully
  - Rows: 43,200+ (30 days × 1 minute bars)
  - Columns: 6+ (OHLCV + metadata)
✓ All required columns present
✓ No null values
Training Suitability: EXCELLENT

✅ Data validation PASSED - Ready for hyperparameter optimization

3. Python Dependencies

# Verify Python environment
python3 -c "import optuna, grpc, yaml, pandas; print('✓ All dependencies installed')"

If missing dependencies:

pip install optuna==3.6.1 grpcio==1.60.0 pyyaml pandas pyarrow pynvml

---

🚀 Execution Steps

Step 1: Navigate to Service Directory

cd /home/jgrusewski/Work/foxhunt/services/ml_training_service

Step 2: Validate Configuration

# Verify optimized configuration exists
ls -lh tuning_config_optimized.yaml

# Preview configuration
head -50 tuning_config_optimized.yaml

Step 3: Run Quick Test (Single Trial)

# Test with 1 trial for DQN (quick validation, ~5-10 minutes)
python3 run_hyperparameter_optimization.py \
    --num-trials 1 \
    --config tuning_config_optimized.yaml \
    --data-path /home/jgrusewski/Work/foxhunt/test_data/real/parquet/BTC-USD_30day_2024-09.parquet \
    --output-dir ./hyperparameter_results_test \
    --use-gpu

Expected output:

[2025-10-14 HH:MM:SS] [INFO] Hyperparameter Optimization Runner initialized
[2025-10-14 HH:MM:SS] [INFO]   Trials per model: 1
[2025-10-14 HH:MM:SS] [INFO]   Models to optimize: DQN, PPO, MAMBA_2, TFT
[2025-10-14 HH:MM:SS] [INFO]   GPU enabled: True
...

Step 4: Run Full Optimization (Production)

# Full optimization: 50 trials per model (estimated: 4-8 hours total)
python3 run_hyperparameter_optimization.py \
    --num-trials 50 \
    --config tuning_config_optimized.yaml \
    --data-path /home/jgrusewski/Work/foxhunt/test_data/real/parquet/BTC-USD_30day_2024-09.parquet \
    --output-dir ./hyperparameter_results \
    --use-gpu \
    2>&1 | tee hyperparameter_optimization.log

Parameters:

• --num-trials 50: 50 optimization trials per model (recommended for production)
• --config tuning_config_optimized.yaml: Agent 49 optimized search spaces
• --data-path: BTC/USD 30-day data (43K+ rows)
• --output-dir: Results directory
• --use-gpu: Enable GPU acceleration (RTX 3050 Ti)
• 2>&1 | tee: Log output to file for later review

Estimated Duration:

• DQN: 60-90 minutes (50 trials)
• PPO: 60-90 minutes (50 trials)
• MAMBA-2: 45-60 minutes (50 trials, faster convergence)
• TFT: 60-90 minutes (50 trials)
• Total: 4-6 hours (sequential execution for GPU safety)

---

📊 Monitoring Progress

Real-time Monitoring

# Monitor execution log
tail -f hyperparameter_optimization.log

# Monitor GPU utilization
watch -n 1 nvidia-smi

# Monitor ML service logs
docker-compose logs -f ml_training_service

Progress Indicators

[INFO] Starting hyperparameter optimization for DQN
[INFO] Trial 1/50: {'learning_rate': 0.0001, 'batch_size': 128, 'gamma': 0.99}
[INFO] Trial 1: Training succeeded - Sharpe=1.25, Loss=0.042, Duration=120s
[INFO] Trial 2/50: {'learning_rate': 0.001, 'batch_size': 64, 'gamma': 0.999}
...
[INFO] DQN optimization completed in 75.3 minutes
[INFO] Best Sharpe ratio: 1.85

---

📈 Results Analysis

Output Structure

hyperparameter_results/
├── hyperparameter_optimization_report.txt    # Human-readable summary
├── aggregate_results.json                    # Machine-readable aggregate
├── results/
│   ├── DQN_results.json                     # DQN best params + metrics
│   ├── PPO_results.json                     # PPO best params + metrics
│   ├── MAMBA_2_results.json                 # MAMBA-2 best params + metrics
│   └── TFT_results.json                     # TFT best params + metrics
└── studies/
    ├── study_DQN_<job_id>.log              # Optuna study (crash recovery)
    ├── study_PPO_<job_id>.log
    ├── study_MAMBA_2_<job_id>.log
    └── study_TFT_<job_id>.log

Reading Results

1. Summary Report (Human-Readable)

cat hyperparameter_results/hyperparameter_optimization_report.txt

Example output:

================================================================================
HYPERPARAMETER OPTIMIZATION SUMMARY REPORT
Agent 49 - Foxhunt HFT Trading System
================================================================================

Execution Time: 285.4 minutes
Trials per Model: 50
GPU Enabled: True
Timestamp: 2025-10-14 23:45:32

================================================================================
MODEL-BY-MODEL RESULTS
================================================================================

### DQN ###
Status: SUCCESS
Best Sharpe Ratio: 1.85
Performance Improvement: 185.00%
Completed Trials: 47
Pruned Trials: 2
Failed Trials: 1
Duration: 75.3 minutes
Best Hyperparameters:
  - batch_size: 128
  - gamma: 0.99
  - learning_rate: 0.0001

### PPO ###
Status: SUCCESS
Best Sharpe Ratio: 1.72
Performance Improvement: 172.00%
...

2. Aggregate Results (JSON)

cat hyperparameter_results/aggregate_results.json | jq .

Example output:

{
  "DQN": {
    "model_type": "DQN",
    "best_sharpe": 1.85,
    "best_params": {
      "learning_rate": 0.0001,
      "batch_size": 128,
      "gamma": 0.99
    },
    "improvement_percent": 185.0,
    "num_completed_trials": 47,
    "duration_seconds": 4518
  },
  ...
}

3. Model-Specific Results

cat hyperparameter_results/results/DQN_results.json | jq .

---

🎯 Success Criteria

Optimization Success

• ✅ All 4 models complete optimization without crashes
• ✅ At least 90% of trials complete successfully per model
• ✅ Best Sharpe ratio > 1.0 for at least 3 models
• ✅ Results reproducible from saved studies

Performance Improvements

• Baseline: Sharpe ratio 0.0 (random/default params)
• Target: Sharpe ratio > 1.5 (good trading performance)
• Excellent: Sharpe ratio > 2.0 (exceptional)

Expected Results

Model	Expected Best Sharpe	Improvement	Status
DQN	1.5 - 2.0	150-200%	To be measured
PPO	1.4 - 1.9	140-190%	To be measured
MAMBA-2	1.6 - 2.1	160-210%	To be measured
TFT	1.5 - 2.0	150-200%	To be measured

---

🔄 Resuming After Interruption

If optimization is interrupted (power loss, crash, etc.):

# Optuna JournalStorage automatically saves progress
# Simply re-run the command - it will resume from last completed trial

python3 run_hyperparameter_optimization.py \
    --num-trials 50 \
    --config tuning_config_optimized.yaml \
    --data-path /home/jgrusewski/Work/foxhunt/test_data/real/parquet/BTC-USD_30day_2024-09.parquet \
    --output-dir ./hyperparameter_results \
    --use-gpu

Note: Optuna will detect existing study files in ./hyperparameter_results/studies/ and resume automatically.

---

🐛 Troubleshooting

Issue: ML Training Service Not Responding

# Check service health
curl http://localhost:8095/health

# Restart if needed
docker-compose restart ml_training_service

# Wait 30 seconds for startup
sleep 30

Issue: GPU Out of Memory

# Check GPU memory
nvidia-smi

# Kill any other GPU processes
kill -9 <PID>

# Or reduce batch size in tuning_config_optimized.yaml
# Change batch_size choices to [32, 64] instead of [64, 128, 256]

Issue: Data Loading Errors

# Validate data file
python3 validate_test_data.py <data_path>

# Check file permissions
ls -lh /home/jgrusewski/Work/foxhunt/test_data/real/parquet/BTC-USD_30day_2024-09.parquet

Issue: Slow Progress

# Reduce trials for faster results
python3 run_hyperparameter_optimization.py \
    --num-trials 20 \
    --config tuning_config_optimized.yaml \
    ...

---

📝 Post-Optimization Actions

1. Review Results

# Read summary report
cat hyperparameter_results/hyperparameter_optimization_report.txt

# Compare best parameters across models
jq '.[] | {model: .model_type, sharpe: .best_sharpe, params: .best_params}' \
    hyperparameter_results/aggregate_results.json

2. Deploy Best Parameters

# Update model configurations with best hyperparameters
# Example: Update DQN config
cat hyperparameter_results/results/DQN_results.json | \
    jq '.best_params' > /path/to/dqn_production_config.json

3. Generate Visualizations (Optional)

# Python script to visualize optimization history
import optuna
from optuna.storages import JournalStorage, JournalFileStorage

# Load study
storage = JournalStorage(JournalFileStorage("hyperparameter_results/studies/study_DQN_<job_id>.log"))
study = optuna.load_study(study_name="study_<job_id>", storage=storage)

# Plot optimization history
optuna.visualization.plot_optimization_history(study).show()

# Plot parameter importances
optuna.visualization.plot_param_importances(study).show()

# Plot contour (learning_rate vs batch_size)
optuna.visualization.plot_contour(study, params=['learning_rate', 'batch_size']).show()

4. Archive Results

# Create timestamped archive
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
tar -czf hyperparameter_results_${TIMESTAMP}.tar.gz hyperparameter_results/
mv hyperparameter_results_${TIMESTAMP}.tar.gz /path/to/archive/

---

📚 Additional Resources

Configuration Files

• tuning_config_optimized.yaml - Agent 49 optimized search spaces
• tuning_config.yaml - Original (broader) search spaces

Scripts

• run_hyperparameter_optimization.py - Main orchestration script
• hyperparameter_tuner.py - Optuna subprocess executor
• validate_test_data.py - Data validation utility

Documentation

• CLAUDE.md - System architecture overview
• TESTING_PLAN.md - ML testing strategy
• Optuna Docs - https://optuna.readthedocs.io/

---

✅ Expected Final Report

After successful execution, you should see:

================================================================================
HYPERPARAMETER OPTIMIZATION SUMMARY REPORT
Agent 49 - Foxhunt HFT Trading System
================================================================================

Execution Time: 285.4 minutes
Trials per Model: 50
GPU Enabled: True

MODEL-BY-MODEL RESULTS:
  ✅ DQN: Sharpe 1.85 (Excellent)
  ✅ PPO: Sharpe 1.72 (Good)
  ✅ MAMBA-2: Sharpe 1.96 (Excellent)
  ✅ TFT: Sharpe 1.78 (Good)

AGGREGATE STATISTICS:
  Successful Optimizations: 4/4 (100%)
  Average Best Sharpe: 1.83
  Best Performing Model: MAMBA-2 (Sharpe: 1.96)

PRODUCTION RECOMMENDATIONS:
  ✅ All models ready for deployment
  ✅ MAMBA-2 recommended as primary model
  ✅ DQN recommended as secondary (RL diversity)

================================================================================
END OF REPORT
================================================================================

---

Last Updated: 2025-10-14 Agent: Agent 49 Status: Ready for execution