foxhunt/docs/archive/backtesting/BACKTESTING_ML_QUICK_REFERENCE.md

Backtesting Service ML Integration - Quick Reference

Key Files

File	Purpose	Lines	Status
services/backtesting_service/src/ml_strategy_engine.rs	ML strategy framework	540	✅ READY
services/backtesting_service/src/performance.rs	Performance metrics	665	✅ READY
services/backtesting_service/src/strategy_engine.rs	Strategy execution	723	✅ READY
services/backtesting_service/tests/ml_strategy_backtest_test.rs	ML tests	508	✅ 8/8 PASSING
services/backtesting_service/tests/ml_backtest_integration_test.rs	Integration tests	293	❌ 4/4 FAILING
services/backtesting_service/tests/report_generation.rs	Report tests	473	✅ 12/12 PASSING

Feature Extraction

ML Feature Extractor (7 features)

// Location: ml_strategy_engine.rs lines 62-173
// Lookback: 20 periods (configurable)
// Output: 7 normalized features [-1, 1]

Features:
1. Price momentum (returns)
2. Short-term MA ratio (5-period)
3. Price volatility (9-bar std dev)
4. Volume ratio
5. Volume MA ratio (5-period)
6. Hour-of-day (normalized)
7. Day-of-week (normalized)

Unified Feature Extractor (16 features)

// Location: data crate (reused in backtesting)
// From strategy_engine.rs line 310

Features:
- 5 OHLCV (Open, High, Low, Close, Volume)
- 10+ technical indicators:
  - RSI, MACD, Bollinger Bands
  - ATR, EMA, SMA
  - + more

Performance Metrics (20+)

// Location: performance.rs lines 12-58

Returns:
- total_return, annualized_return, profit_factor

Risk:
- sharpe_ratio, sortino_ratio, calmar_ratio
- max_drawdown, volatility
- var_95 (Value at Risk)
- expected_shortfall (CVaR)

Trade Stats:
- total_trades, winning_trades, losing_trades
- win_rate, avg_win, avg_loss
- largest_win, largest_loss

ML Strategy Components

MLPoweredStrategy

// Lines 176-304
pub struct MLPoweredStrategy {
    name: String,
    strategy: Arc<SharedMLStrategy>,
    feature_extractor: MLFeatureExtractor,
    model_performance: HashMap<String, MLModelPerformance>,
    confidence_based_sizing: bool,
    min_confidence_threshold: f64,
}

Key methods:
- get_ensemble_prediction() → Vec<MLPrediction>
- calculate_ensemble_vote() → (f64, f64)
- validate_predictions() → tracks accuracy
- get_performance_summary() → HashMap<String, MLModelPerformance>

MLStrategyEngine

// Lines 389-539
pub struct MLStrategyEngine {
    base_engine: StrategyEngine,
    ml_strategies: HashMap<String, MLPoweredStrategy>,
    global_model_performance: HashMap<String, MLModelPerformance>,
}

Key methods:
- execute_ml_backtest() → (Vec<BacktestTrade>, HashMap<performance>)
- get_global_model_performance() → HashMap
- generate_performance_report() → String

Ensemble Voting

// Lines 247-266
// Weighted by confidence, normalized

let weighted_prediction = predictions.iter()
    .map(|p| p.prediction_value * p.confidence)
    .sum::<f64>() / total_confidence;

let average_confidence = predictions.iter()
    .map(|p| p.confidence).sum::<f64>() / predictions.len() as f64;

Confidence Filtering

// Default: 0.6 (60% threshold)
// Located: ml_strategy_engine.rs line 211

if confidence >= min_confidence_threshold {
    // Generate trade signal
} else {
    // Skip this prediction
}

Available Strategies

Rule-Based (Implemented)

1. moving_average_crossover
2. buy_and_hold
3. news_aware_strategy

ML (Framework Ready, No Trained Models)

1. ml_momentum (20-period lookback)
2. ml_ensemble (50-period lookback + voting)

Test Coverage

Passing Tests (20/24)

ML Strategy Tests (8/8 - PASSING)

• Prediction generation
• Ensemble voting
• Trade generation
• Confidence filtering
• Multi-symbol execution
• Performance metrics
• Feature extraction
• Performance tracking

Report Generation Tests (12/12 - PASSING)

• Save/load results
• Metrics aggregation
• Drawdown identification
• Equity curve generation
• Export formats
• Concurrent operations

Failing Tests (4/4 - TDD RED PHASE)

ML Integration Tests (0/4 - NOT IMPLEMENTED)

• Full backtest execution
• ML vs rule-based comparison
• Confidence threshold impact
• Target metrics validation

Data Flow

Real DBN Files
    ↓
DbnDataSource.load_ohlcv_bars()
    ↓
MarketData (OHLCV)
    ↓
MLFeatureExtractor (7 features)
    ↓
SharedMLStrategy (ensemble predictions)
    ↓
Confidence Filtering (threshold: 0.6)
    ↓
Ensemble Vote (weighted)
    ↓
Trade Signals (Buy/Sell with sizing)
    ↓
Portfolio Execution (commission + slippage)
    ↓
BacktestTrade (filled trades)
    ↓
PerformanceAnalyzer (20+ metrics)
    ↓
Results (JSON export)

Critical Gaps

Component	Status	Impact
Model Loading	❌ NOT IMPLEMENTED	Cannot load trained checkpoints
Inference Engine	⚠️ PARTIAL	Using simulator, not real models
Batch Predictions	❌ NOT IMPLEMENTED	Sequential only (~100 bars/sec)
Model Registry	❌ NOT IMPLEMENTED	Hard-coded strategies only
Strategy Comparison	⚠️ SKELETON	Test structure, no implementation

To Use Trained ML Models

Required (1-2 weeks):

1. Implement checkpoint loader
2. Connect inference engine
3. Add batch prediction support
4. Write integration tests

Current: Cannot use trained models yet. Framework ready, missing model loading.

Real Data Available

• ES.FUT (E-mini S&P 500) - 1m OHLCV
• NQ.FUT (Nasdaq futures) - 1m OHLCV
• ZN.FUT (10-year Treasury) - 1d OHLCV
• 6E.FUT (Euro FX) - 1d OHLCV
• CL.FUT (Crude Oil) - available

Configuration

Performance Config (defaults):

equity_curve_resolution: 1000
risk_free_rate: 0.02 (2% annual)

Strategy Config (defaults):

commission_rate: 0.001 (0.1%)
slippage_rate: 0.0005 (0.05%)

Performance Targets (from CLAUDE.md)

• Win Rate: >55%
• Sharpe Ratio: >1.5
• Max Drawdown: <20%

Next Steps

1. Load MAMBA-2/DQN/PPO/TFT checkpoints
2. Implement model inference wrapper
3. Connect to ML strategy engine
4. Test with real backtests
5. Optimize batch predictions

---

See: /home/jgrusewski/Work/foxhunt/ML_BACKTESTING_INTEGRATION_ANALYSIS.md for full analysis