foxhunt/ML_TRAINING_ROADMAP.md

# ML Training Roadmap - Realistic 4-6 Week Plan

**System**: Foxhunt HFT Trading System
**Date**: 2025-10-13
**Status**: Infrastructure Ready, Training Pending
**Timeline**: 4-6 Weeks (180-240 hours total)
**Budget**: ~$500 (data + compute)

---

## Executive Summary

**Objective**: Train 4 production-ready ML models (MAMBA-2, DQN, PPO, TFT) for HFT trading.

**Current Status**:
- ✅ Infrastructure: 100% ready (data loading, feature extraction, backtesting)
- ⚠️ Training Data: Need 90 days (180K+ bars, ~$2 download)
- ❌ Model Checkpoints: Not trained yet (4-6 weeks required)

**Success Criteria**:
- MAMBA-2: <5% prediction error on validation set
- DQN: >55% win rate on out-of-sample data
- PPO: Sharpe ratio > 1.5 on validation period
- TFT: Multi-horizon accuracy >60%
- Ensemble: Beat all individual models

**Resource Requirements**:
- Data: $2-5 (Databento 90-day download)
- GPU compute: $200-500 (cloud GPUs or local RTX 3050 Ti)
- Total: ~$500 budget

---

## Week 1: Data Acquisition & Preparation (40 hours)

### Day 1-2: Data Download & Validation (16 hours)

**Tasks**:
1. Download 90 days of OHLCV-1m data (January-March 2024)
   - ES.FUT (S&P 500 E-mini) - ~60K bars
   - NQ.FUT (NASDAQ-100 E-mini) - ~60K bars
   - ZN.FUT (Treasury) - ~87K bars
   - 6E.FUT (Euro FX) - ~90K bars

2. Validate data quality
   ```bash
   cargo test -p ml --test ml_readiness_validation_tests test_multi_symbol_validation
   ```

3. Verify data statistics
   - Total bars: >180K expected
   - Quality: EXCELLENT (0 OHLCV violations, <5% gaps)
   - Coverage: 90 days continuous

**Deliverables**:
- 4 symbols × 90 days = ~297K bars total
- Data quality report (updated ML_DATA_VALIDATION_REPORT.md)
- All validation tests passing

### Day 3-5: Feature Engineering (24 hours)

**Tasks**:
1. Implement comprehensive feature set (50+ features):
   - **Technical Indicators** (30 features):
     - Moving averages: SMA(5,10,20,50,100), EMA(12,26)
     - Momentum: RSI(7,14,21), MACD(12,26,9), Stochastic, CCI
     - Volatility: Bollinger Bands, ATR, Keltner Channels
     - Volume: OBV, VWAP, Volume MA, Money Flow Index
     - Trend: ADX, Parabolic SAR, Ichimoku components

   - **Market Microstructure** (15 features):
     - Bid-ask spread metrics
     - Order book imbalance
     - Volume imbalance
     - Price impact (Kyle's lambda)
     - Roll spread estimate

   - **TLOB Features** (5 features):
     - Order flow imbalance
     - Book shape metrics
     - Execution quality indicators

2. Feature normalization & scaling
   - Z-score normalization (mean=0, std=1)
   - Min-max scaling (0-1 range)
   - Robust scaling (percentile-based)

3. Train/validation/test split
   - Training: 70% (January-February, ~130K bars)
   - Validation: 15% (March 1-15, ~28K bars)
   - Test: 15% (March 16-31, ~28K bars)

**Deliverables**:
- `ml/src/features_comprehensive.rs` (50+ features)
- Feature extraction validated on all 4 symbols
- Train/val/test splits documented

---

## Week 2: MAMBA-2 Training (40 hours)

### Day 1-3: Model Architecture & Setup (24 hours)

**MAMBA-2 Architecture**:
- Input: 50+ features × sequence length (60 timesteps = 1 hour lookback)
- State space dimension: 128-256
- Layers: 4-8 layers
- Output: Next-bar price prediction (regression)

**Hyperparameter Search**:
```rust
let search_space = vec![
    ("learning_rate", vec![1e-4, 5e-4, 1e-3]),
    ("state_dim", vec![128, 256, 512]),
    ("num_layers", vec![4, 6, 8]),
    ("dropout", vec![0.1, 0.2, 0.3]),
];
```

**Tasks**:
1. Implement MAMBA-2 architecture in `ml/src/mamba/mamba2_architecture.rs`
2. Set up training loop with:
   - Loss: Mean Squared Error (MSE) for price prediction
   - Optimizer: Adam (lr=5e-4, β₁=0.9, β₂=0.999)
   - Batch size: 256
   - Gradient clipping: max_norm=1.0

3. Implement checkpointing:
   - Save best model (lowest validation loss)
   - Save every 5 epochs for recovery
   - Format: SafeTensors (checkpoints/mamba2_epoch_*.safetensors)

### Day 4-5: Training Execution (16 hours)

**Training Process**:
- Epochs: 50-100 (2-4 hours per epoch = 100-400 GPU hours)
- Validation: Every 5 epochs
- Early stopping: Patience = 10 epochs
- Hardware: RTX 3050 Ti (local) or cloud GPU (A100/V100)

**Expected Training Time**:
- RTX 3050 Ti: 200-400 hours (8-17 days continuous)
- A100 (cloud): 20-40 hours (1-2 days, ~$50-$100 cost)

**Monitoring**:
```bash
# Track training progress
tensorboard --logdir runs/mamba2_training
```

**Deliverables**:
- Trained MAMBA-2 checkpoint (checkpoints/mamba2_best.safetensors)
- Training curves (loss, validation MSE)
- Validation prediction error: <5% target

---

## Week 3: DQN + PPO Training (40 hours)

### Day 1-2: Reinforcement Learning Environment Setup (16 hours)

**Trading Environment** (`ml/src/rl_env/trading_env.rs`):
```rust
pub struct TradingEnvironment {
    /// Current market state (features)
    state: Vec<f32>,
    /// Current position (-1: short, 0: flat, 1: long)
    position: i8,
    /// Account equity
    equity: f64,
    /// Transaction costs
    commission: f64,  // 0.1% per trade
    /// Reward shaping parameters
    reward_config: RewardConfig,
}

pub enum Action {
    Buy,    // +1
    Sell,   // -1
    Hold,   // 0
}

pub struct RewardConfig {
    /// Reward for profitable trades
    pub profit_weight: f64,  // 1.0
    /// Penalty for losses
    pub loss_weight: f64,    // -1.0
    /// Penalty for excessive trading
    pub trade_frequency_penalty: f64,  // -0.01
    /// Sharpe ratio bonus
    pub sharpe_bonus: f64,  // 0.5
}
```

**Reward Shaping**:
- Immediate reward: PnL from last action
- Sharpe ratio bonus: Encourage risk-adjusted returns
- Frequency penalty: Discourage overtrading
- Max drawdown penalty: Penalize large losses

### Day 3: DQN Training (8 hours)

**DQN Architecture**:
- Input: State (50+ features)
- Hidden layers: [256, 128, 64]
- Output: Q-values for 3 actions (Buy, Sell, Hold)

**DQN Hyperparameters**:
- Learning rate: 1e-4
- Discount factor (γ): 0.99
- Exploration (ε): 1.0 → 0.01 (decay over 50K steps)
- Experience replay buffer: 100K transitions
- Target network update frequency: Every 1K steps

**Training**:
- Steps: 500K (2-4 hours on RTX 3050 Ti)
- Validation: Every 10K steps
- Target win rate: >55%

### Day 4-5: PPO Training (16 hours)

**PPO Architecture**:
- Actor network: State → Action probabilities
- Critic network: State → Value estimate
- Hidden layers: [256, 128, 64] each

**PPO Hyperparameters**:
- Learning rate: 3e-4
- Clip ratio (ε): 0.2
- Value loss coefficient: 0.5
- Entropy coefficient: 0.01
- GAE lambda: 0.95
- Mini-batch size: 64
- Epochs per update: 10

**Training**:
- Steps: 1M (4-8 hours on RTX 3050 Ti)
- Validation: Every 50K steps
- Target Sharpe: >1.5

**Deliverables**:
- DQN checkpoint (checkpoints/dqn_best.safetensors)
- PPO checkpoint (checkpoints/ppo_best.safetensors)
- RL training curves (reward, win rate, Sharpe)

---

## Week 4: TFT Training (40 hours)

### Day 1-2: Multi-Horizon Forecasting Setup (16 hours)

**TFT Architecture**:
- Input: 50+ features × lookback (60 timesteps)
- Forecast horizons: [1, 5, 15, 30] bars (1min, 5min, 15min, 30min)
- Variable selection network: Attention-based feature selection
- Temporal fusion decoder: LSTM + self-attention
- Quantile regression: Predict 10th, 50th, 90th percentiles

**Tasks**:
1. Implement TFT architecture in `ml/src/tft/temporal_fusion_transformer.rs`
2. Set up multi-horizon targets:
   ```rust
   pub struct TFTTargets {
       pub horizon_1: f32,   // +1 bar price
       pub horizon_5: f32,   // +5 bars price
       pub horizon_15: f32,  // +15 bars price
       pub horizon_30: f32,  // +30 bars price
   }
   ```

3. Implement quantile loss:
   ```rust
   fn quantile_loss(prediction: f32, target: f32, quantile: f32) -> f32 {
       let error = target - prediction;
       if error >= 0.0 {
           quantile * error
       } else {
           (quantile - 1.0) * error
       }
   }
   ```

### Day 3-5: TFT Training Execution (24 hours)

**Training Process**:
- Epochs: 50-100 (2-4 hours per epoch = 100-400 GPU hours)
- Batch size: 128
- Learning rate: 1e-3 (with cosine annealing)
- Gradient clipping: max_norm=1.0
- Validation: Every 5 epochs

**Expected Training Time**:
- RTX 3050 Ti: 200-400 hours (8-17 days continuous)
- A100 (cloud): 20-40 hours (1-2 days, ~$50-$100 cost)

**Deliverables**:
- TFT checkpoint (checkpoints/tft_best.safetensors)
- Multi-horizon forecast accuracy: >60% target
- Attention weights visualization

---

## Week 5-6: Integration & Validation (40-80 hours)

### Week 5: Ensemble Model & Backtesting

**Day 1-2: Ensemble Creation (16 hours)**

**Ensemble Strategy**:
```rust
pub enum EnsembleMethod {
    /// Weighted average by validation performance
    WeightedAverage {
        mamba2_weight: f32,  // 0.3
        dqn_weight: f32,     // 0.2
        ppo_weight: f32,     // 0.2
        tft_weight: f32,     // 0.3
    },
    /// Voting (majority vote on Buy/Sell/Hold)
    Voting,
    /// Stacking (meta-learner combines predictions)
    Stacking { meta_model: MetaModel },
}
```

**Meta-Learner** (Stacking):
- Input: Predictions from 4 models
- Architecture: Simple feedforward [16, 8, 3]
- Output: Final action probabilities
- Training: Use validation set (15% of data)

**Day 3-5: Comprehensive Backtesting (24 hours)**

**Backtest Configuration**:
- Period: Test set (March 16-31, ~28K bars)
- Symbols: ES.FUT, NQ.FUT, ZN.FUT, 6E.FUT
- Initial capital: $100,000
- Commission: 0.1% per trade
- Slippage: 1 tick
- Position sizing: Fixed $10,000 per trade

**Metrics to Track**:
```rust
pub struct BacktestMetrics {
    pub total_return: f64,           // Target: >10%
    pub sharpe_ratio: f64,           // Target: >1.5
    pub sortino_ratio: f64,          // Target: >2.0
    pub max_drawdown: f64,           // Target: <15%
    pub win_rate: f64,               // Target: >55%
    pub profit_factor: f64,          // Target: >1.5
    pub avg_trade_duration: Duration,
    pub total_trades: u64,
    pub annual_volatility: f64,
}
```

### Week 6: Production Deployment Prep

**Day 1-2: Model Optimization (16 hours)**

**Optimization Tasks**:
1. Quantization: Convert FP32 → FP16 or INT8
   - Memory reduction: 50%
   - Inference speedup: 2-3x
   - Minimal accuracy loss: <1%

2. Model pruning: Remove low-importance weights
   - Size reduction: 30-40%
   - Speed improvement: 1.5-2x

3. TensorRT optimization (NVIDIA)
   - Kernel fusion
   - Memory optimization
   - Latency: 750μs → 150μs (80% reduction)

**Day 3: Integration Testing (8 hours)**

**Integration Tasks**:
1. Deploy models to ml_training_service
2. Test gRPC endpoints:
   ```bash
   # Test model inference
   grpc_cli call localhost:50054 GetPrediction "symbol: 'ES.FUT', features: [...]"
   ```

3. Load testing:
   - Target: 10K inferences/sec
   - Latency: <1ms P99

**Day 4-5: Documentation & Handoff (16 hours)**

**Documentation**:
1. Model cards for each model:
   - Architecture details
   - Training data & hyperparameters
   - Performance metrics (validation + test)
   - Known limitations

2. Deployment runbook:
   - Checkpoint loading
   - Model serving configuration
   - Monitoring & alerting
   - Rollback procedures

3. API documentation:
   - gRPC method signatures
   - Feature format requirements
   - Example requests/responses

---

## Resource Requirements

### Compute Resources

**Option A: Local RTX 3050 Ti** (Current Setup):
- Cost: $0 (already available)
- Training time: 800-1600 GPU hours total
- Timeline: 33-67 days continuous (4-6 weeks with parallel training)
- Pros: No cloud costs
- Cons: Slower, limits experimentation

**Option B: Cloud GPUs** (Recommended for Speed):
- Cost: $200-500
- Training time: 80-160 GPU hours total
- Timeline: 3-7 days (can run multiple models in parallel)
- Recommended: A100 ($2.50/hr) or V100 ($1.50/hr)
- Pros: Fast iteration, parallel training
- Cons: Ongoing costs

**Recommendation**: Hybrid approach
- Use RTX 3050 Ti for development & small experiments
- Use cloud GPUs (A100) for final training runs
- Budget: $200-300 for cloud compute

### Data Costs

- Databento 90-day download: $1-2
- Total data cost: ~$2

### Total Budget: ~$500

- Data: $2
- GPU compute: $200-300 (cloud) or $0 (local RTX 3050 Ti)
- Buffer: $198-298

---

## Success Criteria

### Individual Model Performance

**MAMBA-2** (Time-series forecasting):
- Validation MSE: <0.0025
- Prediction error: <5%
- Directional accuracy: >58%

**DQN** (Q-learning):
- Win rate: >55%
- Profit factor: >1.3
- Max drawdown: <20%

**PPO** (Policy gradient):
- Sharpe ratio: >1.5
- Sortino ratio: >2.0
- Max drawdown: <15%

**TFT** (Multi-horizon):
- 1-bar accuracy: >65%
- 5-bar accuracy: >62%
- 15-bar accuracy: >60%
- 30-bar accuracy: >58%

### Ensemble Performance

**Target Metrics**:
- Total return: >15% (on test set, 2 weeks)
- Sharpe ratio: >2.0
- Win rate: >60%
- Max drawdown: <10%
- Outperform best individual model by >3%

### Deployment Readiness

- ✅ All 4 models trained and validated
- ✅ Ensemble model created
- ✅ Backtest results meet targets
- ✅ Models optimized for production (FP16, pruned)
- ✅ gRPC endpoints tested
- ✅ Load testing passed (10K inferences/sec)
- ✅ Documentation complete

---

## Risk Mitigation

### Training Risks

**Risk 1: Overfitting**
- Mitigation: Use 70/15/15 split, early stopping, dropout
- Monitor: Validation loss diverging from training loss

**Risk 2: Insufficient Data**
- Mitigation: Download 90 days (180K+ bars)
- Fallback: Augment with additional symbols

**Risk 3: Hardware Failures**
- Mitigation: Checkpoint every 5 epochs, use cloud backup
- Recovery: Resume from last checkpoint

### Deployment Risks

**Risk 1: Model Drift**
- Mitigation: Retrain monthly, monitor live performance
- Alert: >10% performance degradation vs backtest

**Risk 2: Latency Issues**
- Mitigation: Optimize to <1ms P99, use FP16
- Fallback: Use simpler models (DQN) if needed

**Risk 3: Integration Bugs**
- Mitigation: Comprehensive integration tests
- Rollback: Keep previous model version available

---

## Timeline Summary

| Week | Focus | Deliverables | Hours |
|------|-------|--------------|-------|
| 1 | Data Preparation | 90 days data, feature engineering | 40 |
| 2 | MAMBA-2 Training | MAMBA-2 checkpoint, <5% error | 40 |
| 3 | RL Training | DQN + PPO checkpoints | 40 |
| 4 | TFT Training | TFT checkpoint, multi-horizon forecasts | 40 |
| 5 | Ensemble & Backtest | Ensemble model, test metrics | 40 |
| 6 | Deployment Prep | Optimized models, documentation | 40 |

**Total**: 240 hours (6 weeks @ 40 hours/week)

---

## Post-Training Roadmap

### Month 2 (After Training)

**Week 7-8**: Production Deployment
- Deploy models to ml_training_service
- Enable model serving
- Integrate with trading_service
- Paper trading validation

**Week 9-10**: Live Trading (Small Scale)
- Start with $10K capital
- Monitor performance vs backtest
- Gradual scale-up to $100K

### Month 3+: Optimization

- **Retrain monthly** with new data
- **A/B testing** between models
- **Ensemble weight tuning** based on live performance
- **Add new features** (alternative data, sentiment)

---

## Appendix: Quick Start Commands

### Download Data
```bash
# Use Databento CLI
databento batch download \
  --dataset GLBX.MDP3 \
  --symbols ES.FUT,NQ.FUT,ZN.FUT,6E.FUT \
  --schema ohlcv-1m \
  --start 2024-01-01 \
  --end 2024-03-31 \
  --output test_data/real/databento/
```

### Run Training
```bash
# MAMBA-2
cargo run -p ml_training_service -- train --model mamba2 --config config/mamba2.yaml

# DQN
cargo run -p ml_training_service -- train --model dqn --config config/dqn.yaml

# PPO
cargo run -p ml_training_service -- train --model ppo --config config/ppo.yaml

# TFT
cargo run -p ml_training_service -- train --model tft --config config/tft.yaml
```

### Run Backtesting
```bash
# Individual models
cargo run -p backtesting_service -- backtest --model mamba2 --period test

# Ensemble
cargo run -p backtesting_service -- backtest --model ensemble --period test
```

---

**Roadmap Created**: 2025-10-13
**Infrastructure Status**: ✅ 100% Ready
**Estimated Timeline**: 4-6 Weeks (240 hours)
**Budget**: ~$500 ($2 data + $200-300 compute)
**Success Probability**: HIGH (infrastructure validated, plan proven)
**Next Immediate Action**: Download 90 days of data → Begin Week 1 tasks