foxhunt/BACKTEST_DQN_USAGE_GUIDE.md

DQN Backtest Validation Script - Usage Guide

Created: 2025-11-11 Script: /home/jgrusewski/Work/foxhunt/ml/examples/backtest_dqn.rs Purpose: Validate trained DQN checkpoints against production criteria

---

Overview

The backtest_dqn script provides comprehensive validation of DQN model checkpoints by:

• Running backtests on held-out validation data
• Calculating key performance metrics (Sharpe ratio, win rate, drawdown)
• Comparing against baseline models (optional)
• Validating against production readiness criteria
• Generating reports in multiple formats (console, JSON, markdown)

---

Success Criteria (Default)

Metric	Threshold	Description
Sharpe Ratio	≥ 2.0	Risk-adjusted return (annualized)
Win Rate	≥ 55%	Percentage of profitable trades
Max Drawdown	≤ 20%	Maximum peak-to-trough decline

All three criteria must pass for a checkpoint to be deemed "Production Ready".

---

Quick Start

1. Basic Validation (Single Checkpoint)

Validate the best checkpoint from Wave 9-11 training:

cargo run -p ml --example backtest_dqn --release --features cuda -- \
  --checkpoint ml/trained_models/dqn_best_model.safetensors \
  --data test_data/ES_FUT_180d.parquet

Expected Output:

╔══════════════════════════════════════════════════════════════════════╗
║              DQN BACKTEST VALIDATION REPORT                          ║
╚══════════════════════════════════════════════════════════════════════╝

═══ Checkpoint ═══
  Primary: ml/trained_models/dqn_best_model.safetensors

═══ Performance Metrics ═══
  Total Return:       12.50%
  Sharpe Ratio:        2.35
  Max Drawdown:       15.20%
  Win Rate:           58.3%
  Total Trades:         42
  Avg Trade PnL:      123.45
  Final Equity:    112500.00

═══ Success Criteria ═══
  ✅ Sharpe Ratio ≥ 2.0: 2.35
  ✅ Win Rate ≥ 55.0%: 58.3%
  ✅ Max Drawdown ≤ 20.0%: 15.2%

═══ Verdict ═══
  ✅ PRODUCTION READY
     Checkpoint meets all success criteria

✅ EXIT CODE 0: Production ready

---

2. Baseline Comparison

Compare new checkpoint against baseline:

cargo run -p ml --example backtest_dqn --release --features cuda -- \
  --checkpoint ml/trained_models/dqn_epoch_5.safetensors \
  --baseline ml/trained_models/dqn_baseline.safetensors \
  --data test_data/ES_FUT_180d.parquet

Additional Output:

═══ Baseline Comparison ═══
  Sharpe Ratio:        1.85 →     2.35 (+0.50)
  Total Return:        8.20% →   12.50% (+4.30%)
  Max Drawdown:       18.50% →   15.20% (+3.30%)
  Win Rate:           52.0% →    58.3% (+6.3%)

---

3. JSON Export (CI/CD Integration)

Export results to JSON for automated validation:

cargo run -p ml --example backtest_dqn --release --features cuda -- \
  --checkpoint ml/trained_models/dqn_epoch_5.safetensors \
  --data test_data/ES_FUT_180d.parquet \
  --output-json backtest_results.json

JSON Structure:

{
  "checkpoint_name": "ml/trained_models/dqn_epoch_5.safetensors",
  "baseline_name": null,
  "metrics": {
    "total_return_pct": 12.50,
    "sharpe_ratio": 2.35,
    "max_drawdown_pct": 15.20,
    "win_rate": 58.3,
    "total_trades": 42,
    "avg_trade_pnl": 123.45,
    "final_equity": 112500.00,
    "max_equity": 115200.00
  },
  "baseline_metrics": null,
  "success_criteria": {
    "min_sharpe": 2.0,
    "min_win_rate": 55.0,
    "max_drawdown": 20.0,
    "sharpe_passed": true,
    "win_rate_passed": true,
    "drawdown_passed": true,
    "overall_passed": true
  },
  "verdict": "ProductionReady"
}

CI/CD Integration:

# Exit code 0 = Production ready, Exit code 1 = Failed validation
cargo run -p ml --example backtest_dqn --release --features cuda -- \
  --checkpoint $CHECKPOINT_PATH \
  --data $VALIDATION_DATA \
  --output-json results.json

if [ $? -eq 0 ]; then
  echo "✅ Checkpoint validated - ready for deployment"
else
  echo "❌ Checkpoint failed validation - retrain required"
  exit 1
fi

---

4. Markdown Report

Generate markdown report for documentation:

cargo run -p ml --example backtest_dqn --release --features cuda -- \
  --checkpoint ml/trained_models/dqn_epoch_5.safetensors \
  --baseline ml/trained_models/dqn_baseline.safetensors \
  --data test_data/ES_FUT_180d.parquet \
  --output-markdown backtest_report.md

Output File: backtest_report.md

---

CLI Reference

Required Arguments

Argument	Description	Example
--checkpoint <PATH>	Primary DQN checkpoint to validate	ml/trained_models/dqn_best_model.safetensors
--data <PATH>	Validation data (Parquet format)	test_data/ES_FUT_180d.parquet

Optional Arguments

Argument	Default	Description
--baseline <PATH>	None	Baseline checkpoint for comparison
--device <DEVICE>	auto	Device selection: cpu, cuda, or auto
--initial-capital <FLOAT>	100000.0	Initial capital for backtest ($)
--warmup-bars <INT>	50	Warmup bars to skip (feature history)
--output-json <PATH>	None	Export results to JSON file
--output-markdown <PATH>	None	Export report to markdown file
--verbose / -v	false	Enable DEBUG level logging
--min-sharpe <FLOAT>	2.0	Minimum Sharpe ratio threshold
--min-win-rate <FLOAT>	55.0	Minimum win rate threshold (%)
--max-drawdown <FLOAT>	20.0	Maximum drawdown threshold (%)

---

Use Cases

1. Wave 9-11 Production Validation

Validate the best checkpoint from Wave 9-11 (45-action training):

# As recommended in production test report (lines 294-296, 350-355)
cargo run -p ml --example backtest_dqn --release --features cuda -- \
  --checkpoint /tmp/ml_training/wave11_production_10epoch/dqn_best_model.safetensors \
  --data test_data/ES_FUT_unseen.parquet \
  --output-json wave11_validation.json \
  --output-markdown wave11_report.md

Success Criteria (from report):

• Sharpe > 2.0 ✅
• Win Rate > 55% ✅
• Drawdown < 20% ✅

---

2. Hyperopt Best Parameters Validation

Validate hyperopt-optimized checkpoint from Wave 7:

# Wave 7 best: Trial #6, Sharpe 4.311 (LR=3.14e-5, BS=222, Gamma=0.963)
cargo run -p ml --example backtest_dqn --release --features cuda -- \
  --checkpoint ml/trained_models/hyperopt_trial_6_best.safetensors \
  --baseline ml/trained_models/dqn_baseline.safetensors \
  --data test_data/ES_FUT_validation.parquet \
  --min-sharpe 4.0 \
  --min-win-rate 60.0 \
  --max-drawdown 15.0

Expected: Trial #6 should exceed all thresholds (Sharpe 4.311 >> 4.0)

---

3. 3-Action vs 45-Action Comparison

Compare baseline 3-action system against new 45-action system:

cargo run -p ml --example backtest_dqn --release --features cuda -- \
  --checkpoint ml/trained_models/dqn_45action_best.safetensors \
  --baseline ml/trained_models/dqn_3action_baseline.safetensors \
  --data test_data/ES_FUT_180d.parquet \
  --output-markdown action_space_comparison.md

Hypothesis (from Wave 9-11): 45-action system should show:

• Higher action diversity (100% vs ~60%)
• Net positive transaction cost rebates (LimitMaker orders)
• Better risk-adjusted returns (higher Sharpe)

---

4. Batch Validation (Multiple Checkpoints)

Validate all epoch checkpoints to find best:

#!/bin/bash
# validate_all_checkpoints.sh

for epoch in 10 20 30 40 50 60 70 80 90 100; do
  echo "Validating epoch $epoch..."
  cargo run -p ml --example backtest_dqn --release --features cuda -- \
    --checkpoint ml/trained_models/dqn_epoch_${epoch}.safetensors \
    --data test_data/ES_FUT_validation.parquet \
    --output-json results/epoch_${epoch}.json

  if [ $? -eq 0 ]; then
    echo "✅ Epoch $epoch: PASSED"
  else
    echo "❌ Epoch $epoch: FAILED"
  fi
done

# Parse JSON results to find best checkpoint
python3 scripts/find_best_checkpoint.py results/*.json

---

Architecture Details

Phase 1: Data Loading

1. Load Parquet file → OHLCV bars
2. Extract 128-dimensional features (Wave D)
3. Apply preprocessing (log returns + normalization + clipping)

Key Files:

• ml/src/data_loaders/parquet_utils.rs - Parquet loading
• ml/src/features/extraction.rs - 128-feature extraction
• ml/src/preprocessing.rs - Preprocessing pipeline

---

Phase 2: Model Loading

1. Create WorkingDQN configuration (128 input → [256, 128, 64] hidden → 3 actions)
2. Load SafeTensors checkpoint
3. Verify architecture matches training

Configuration:

• State dimension: 128 features
• Hidden layers: [256, 128, 64]
• Actions: 3 (BUY, HOLD, SELL)
• Epsilon: 0.0 (greedy evaluation)

---

Phase 3: Backtest Execution

1. Run greedy inference (epsilon=0) for each bar
2. Execute trades via EvaluationEngine
3. Record trade history (entry/exit prices, PnL)

Trade Logic:

• BUY: Open long position (close short if exists)
• SELL: Open short position (close long if exists)
• HOLD: Maintain current position
• Final bar: Close any open position

---

Phase 4: Metrics Calculation

Comprehensive performance metrics from PerformanceMetrics::from_trades():

Metric	Formula	Description
Total Return	(final_equity - initial_capital) / initial_capital * 100	% gain/loss
Sharpe Ratio	mean(returns) / std(returns) * sqrt(252)	Annualized risk-adjusted return
Max Drawdown	max((peak_equity - current_equity) / peak_equity * 100)	Worst decline from peak
Win Rate	winning_trades / total_trades * 100	% profitable trades
Avg Trade PnL	total_pnl / total_trades	Mean profit/loss per trade

Key Files:

• ml/src/evaluation/metrics.rs - Metrics calculation
• ml/src/evaluation/engine.rs - Trade execution

---

Phase 5: Validation & Reporting

1. Compare metrics against success criteria
2. Generate verdict (ProductionReady / Failed)
3. Output reports (console, JSON, markdown)
4. Exit with appropriate code (0=success, 1=failure)

---

Troubleshooting

Issue: "Checkpoint file not found"

Error:

Checkpoint file not found: ml/trained_models/dqn_epoch_5.safetensors
Suggestion: Train a model first using train_dqn example

Solution:

# Train a model first
cargo run -p ml --example train_dqn --release --features cuda -- --epochs 10

---

Issue: "Data file not found"

Error:

Data file not found: test_data/ES_FUT_unseen.parquet
Suggestion: Use test_data/ES_FUT_180d.parquet

Solution:

# Use existing validation data
--data test_data/ES_FUT_180d.parquet

---

Issue: "CUDA unavailable"

Error:

CUDA unavailable. Use --device cpu or --device auto

Solution:

# Force CPU execution
--device cpu

# Or use auto-fallback
--device auto

---

Issue: "Feature/bar mismatch"

Error:

Feature/bar mismatch: 1000 features, 1050 bars

Cause: Warmup period mismatch (preprocessing removes first 50 bars)

Solution: Internal issue - should not occur. If it does, file a bug report.

---

Issue: "All trades unprofitable"

Output:

❌ FAILED VALIDATION
     Reasons:
       • Win rate 30.0% < 55.0% (required)

Analysis:

• Model may be overfitted to training data
• Validation data may be out-of-distribution
• Hyperparameters may need tuning

Actions:

1. Check training/validation data similarity
2. Re-run hyperopt with more diverse data
3. Inspect action distribution (should be balanced)
4. Review Q-value statistics (check for collapse)

---

Integration with Training Pipeline

Recommended Workflow

# 1. Train model
cargo run -p ml --example train_dqn --release --features cuda -- \
  --epochs 100 \
  --output-dir /tmp/ml_training/production

# 2. Validate best checkpoint
cargo run -p ml --example backtest_dqn --release --features cuda -- \
  --checkpoint /tmp/ml_training/production/dqn_best_model.safetensors \
  --data test_data/ES_FUT_validation.parquet \
  --output-json validation_results.json

# 3. If validated, deploy to production
if [ $? -eq 0 ]; then
  cp /tmp/ml_training/production/dqn_best_model.safetensors \
     ml/trained_models/dqn_production.safetensors
  echo "✅ Deployed to production"
fi

---

Hyperopt Integration

Use backtest validation as hyperopt objective function:

# scripts/python/hyperopt_with_backtest.py
def objective(trial):
    # 1. Train with trial parameters
    checkpoint = train_dqn_trial(trial)

    # 2. Run backtest validation
    result = run_backtest_validation(checkpoint)

    # 3. Return Sharpe ratio as objective
    return result['metrics']['sharpe_ratio']

Advantage: Optimize directly for backtest performance instead of training rewards.

---

Exit Codes

Code	Meaning	Description
0	Success	Checkpoint meets all success criteria (Production Ready)
1	Failure	Checkpoint failed one or more success criteria

CI/CD Usage:

# In GitLab CI / GitHub Actions
cargo run -p ml --example backtest_dqn --release --features cuda -- \
  --checkpoint $CHECKPOINT \
  --data $VALIDATION_DATA \
  --output-json results.json

# Exit code determines pipeline success/failure

---

Performance Benchmarks

Expected Runtime

Data Size	Device	Duration	Throughput
1,000 bars	CPU	~2s	500 bars/sec
1,000 bars	CUDA	~1s	1,000 bars/sec
10,000 bars	CPU	~15s	667 bars/sec
10,000 bars	CUDA	~8s	1,250 bars/sec

Bottlenecks:

• Data loading: ~0.7ms per bar (DBN legacy, 10ms Parquet)
• Feature extraction: ~1ms per bar (225 features)
• Inference: ~200μs per bar (DQN forward pass)
• Metrics calculation: <1ms total

---

Output Examples

Console Output (Failed Validation)

╔══════════════════════════════════════════════════════════════════════╗
║              DQN BACKTEST VALIDATION REPORT                          ║
╚══════════════════════════════════════════════════════════════════════╝

═══ Checkpoint ═══
  Primary: ml/trained_models/dqn_epoch_50.safetensors

═══ Performance Metrics ═══
  Total Return:        5.20%
  Sharpe Ratio:        1.45
  Max Drawdown:       22.30%
  Win Rate:           48.5%
  Total Trades:         35
  Avg Trade PnL:       67.89
  Final Equity:    105200.00

═══ Success Criteria ═══
  ❌ Sharpe Ratio ≥ 2.0: 1.45
  ❌ Win Rate ≥ 55.0%: 48.5%
  ❌ Max Drawdown ≤ 20.0%: 22.3%

═══ Verdict ═══
  ❌ FAILED VALIDATION
     Reasons:
       • Sharpe ratio 1.45 < 2.00 (required)
       • Win rate 48.5% < 55.0% (required)
       • Max drawdown 22.3% > 20.0% (limit)

❌ EXIT CODE 1: Validation failed

---

Future Enhancements

Planned Features (Post-Wave 11)

1. Multi-checkpoint comparison: Validate multiple checkpoints in one run
2. Time-series cross-validation: Rolling window validation
3. Custom success criteria: User-defined validation rules
4. Detailed trade log export: CSV export with entry/exit timestamps
5. Performance attribution: Breakdown by market regime
6. Risk metrics: Sortino ratio, Calmar ratio, Value at Risk
7. Execution simulation: Slippage and transaction costs

---

Related Documentation

• Training: ml/examples/train_dqn.rs - DQN training pipeline
• Evaluation: ml/examples/evaluate_dqn_main_orchestrator.rs - Comprehensive evaluation
• Hyperopt: ml/examples/hyperopt_dqn_demo.rs - Hyperparameter optimization
• Wave 9-11 Report: /tmp/WAVE9_11_PRODUCTION_CERTIFICATION.md - Production certification
• Wave 7 Report: WAVE7_P&L_VALIDATION_REPORT.md - Early stopping analysis

---

Summary

The backtest_dqn script provides production-grade validation for DQN checkpoints with:

• ✅ Comprehensive metrics (Sharpe, win rate, drawdown)
• ✅ Baseline comparison support
• ✅ Multiple output formats (console, JSON, markdown)
• ✅ CI/CD integration (exit codes)
• ✅ Configurable success criteria
• ✅ Fast execution (2-15s for 1K-10K bars)

Recommended Usage: Validate all production checkpoints before deployment to ensure they meet risk-adjusted return thresholds.