foxhunt/BACKTEST_DQN_USAGE_GUIDE.md

# DQN Backtest Validation Script - Usage Guide

**Created**: 2025-11-11
**Script**: `/home/jgrusewski/Work/foxhunt/ml/examples/backtest_dqn.rs`
**Purpose**: Validate trained DQN checkpoints against production criteria

---

## Overview

The `backtest_dqn` script provides comprehensive validation of DQN model checkpoints by:
- Running backtests on held-out validation data
- Calculating key performance metrics (Sharpe ratio, win rate, drawdown)
- Comparing against baseline models (optional)
- Validating against production readiness criteria
- Generating reports in multiple formats (console, JSON, markdown)

---

## Success Criteria (Default)

| Metric | Threshold | Description |
|--------|-----------|-------------|
| **Sharpe Ratio** | ≥ 2.0 | Risk-adjusted return (annualized) |
| **Win Rate** | ≥ 55% | Percentage of profitable trades |
| **Max Drawdown** | ≤ 20% | Maximum peak-to-trough decline |

All three criteria must pass for a checkpoint to be deemed "Production Ready".

---

## Quick Start

### 1. Basic Validation (Single Checkpoint)

Validate the best checkpoint from Wave 9-11 training:

```bash
cargo run -p ml --example backtest_dqn --release --features cuda -- \
  --checkpoint ml/trained_models/dqn_best_model.safetensors \
  --data test_data/ES_FUT_180d.parquet
```

**Expected Output**:
```
╔══════════════════════════════════════════════════════════════════════╗
║              DQN BACKTEST VALIDATION REPORT                          ║
╚══════════════════════════════════════════════════════════════════════╝

═══ Checkpoint ═══
  Primary: ml/trained_models/dqn_best_model.safetensors

═══ Performance Metrics ═══
  Total Return:       12.50%
  Sharpe Ratio:        2.35
  Max Drawdown:       15.20%
  Win Rate:           58.3%
  Total Trades:         42
  Avg Trade PnL:      123.45
  Final Equity:    112500.00

═══ Success Criteria ═══
  ✅ Sharpe Ratio ≥ 2.0: 2.35
  ✅ Win Rate ≥ 55.0%: 58.3%
  ✅ Max Drawdown ≤ 20.0%: 15.2%

═══ Verdict ═══
  ✅ PRODUCTION READY
     Checkpoint meets all success criteria

✅ EXIT CODE 0: Production ready
```

---

### 2. Baseline Comparison

Compare new checkpoint against baseline:

```bash
cargo run -p ml --example backtest_dqn --release --features cuda -- \
  --checkpoint ml/trained_models/dqn_epoch_5.safetensors \
  --baseline ml/trained_models/dqn_baseline.safetensors \
  --data test_data/ES_FUT_180d.parquet
```

**Additional Output**:
```
═══ Baseline Comparison ═══
  Sharpe Ratio:        1.85 →     2.35 (+0.50)
  Total Return:        8.20% →   12.50% (+4.30%)
  Max Drawdown:       18.50% →   15.20% (+3.30%)
  Win Rate:           52.0% →    58.3% (+6.3%)
```

---

### 3. JSON Export (CI/CD Integration)

Export results to JSON for automated validation:

```bash
cargo run -p ml --example backtest_dqn --release --features cuda -- \
  --checkpoint ml/trained_models/dqn_epoch_5.safetensors \
  --data test_data/ES_FUT_180d.parquet \
  --output-json backtest_results.json
```

**JSON Structure**:
```json
{
  "checkpoint_name": "ml/trained_models/dqn_epoch_5.safetensors",
  "baseline_name": null,
  "metrics": {
    "total_return_pct": 12.50,
    "sharpe_ratio": 2.35,
    "max_drawdown_pct": 15.20,
    "win_rate": 58.3,
    "total_trades": 42,
    "avg_trade_pnl": 123.45,
    "final_equity": 112500.00,
    "max_equity": 115200.00
  },
  "baseline_metrics": null,
  "success_criteria": {
    "min_sharpe": 2.0,
    "min_win_rate": 55.0,
    "max_drawdown": 20.0,
    "sharpe_passed": true,
    "win_rate_passed": true,
    "drawdown_passed": true,
    "overall_passed": true
  },
  "verdict": "ProductionReady"
}
```

**CI/CD Integration**:
```bash
# Exit code 0 = Production ready, Exit code 1 = Failed validation
cargo run -p ml --example backtest_dqn --release --features cuda -- \
  --checkpoint $CHECKPOINT_PATH \
  --data $VALIDATION_DATA \
  --output-json results.json

if [ $? -eq 0 ]; then
  echo "✅ Checkpoint validated - ready for deployment"
else
  echo "❌ Checkpoint failed validation - retrain required"
  exit 1
fi
```

---

### 4. Markdown Report

Generate markdown report for documentation:

```bash
cargo run -p ml --example backtest_dqn --release --features cuda -- \
  --checkpoint ml/trained_models/dqn_epoch_5.safetensors \
  --baseline ml/trained_models/dqn_baseline.safetensors \
  --data test_data/ES_FUT_180d.parquet \
  --output-markdown backtest_report.md
```

**Output File**: `backtest_report.md`

---

## CLI Reference

### Required Arguments

| Argument | Description | Example |
|----------|-------------|---------|
| `--checkpoint <PATH>` | Primary DQN checkpoint to validate | `ml/trained_models/dqn_best_model.safetensors` |
| `--data <PATH>` | Validation data (Parquet format) | `test_data/ES_FUT_180d.parquet` |

### Optional Arguments

| Argument | Default | Description |
|----------|---------|-------------|
| `--baseline <PATH>` | None | Baseline checkpoint for comparison |
| `--device <DEVICE>` | `auto` | Device selection: `cpu`, `cuda`, or `auto` |
| `--initial-capital <FLOAT>` | `100000.0` | Initial capital for backtest ($) |
| `--warmup-bars <INT>` | `50` | Warmup bars to skip (feature history) |
| `--output-json <PATH>` | None | Export results to JSON file |
| `--output-markdown <PATH>` | None | Export report to markdown file |
| `--verbose` / `-v` | `false` | Enable DEBUG level logging |
| `--min-sharpe <FLOAT>` | `2.0` | Minimum Sharpe ratio threshold |
| `--min-win-rate <FLOAT>` | `55.0` | Minimum win rate threshold (%) |
| `--max-drawdown <FLOAT>` | `20.0` | Maximum drawdown threshold (%) |

---

## Use Cases

### 1. Wave 9-11 Production Validation

Validate the best checkpoint from Wave 9-11 (45-action training):

```bash
# As recommended in production test report (lines 294-296, 350-355)
cargo run -p ml --example backtest_dqn --release --features cuda -- \
  --checkpoint /tmp/ml_training/wave11_production_10epoch/dqn_best_model.safetensors \
  --data test_data/ES_FUT_unseen.parquet \
  --output-json wave11_validation.json \
  --output-markdown wave11_report.md
```

**Success Criteria** (from report):
- Sharpe > 2.0 ✅
- Win Rate > 55% ✅
- Drawdown < 20% ✅

---

### 2. Hyperopt Best Parameters Validation

Validate hyperopt-optimized checkpoint from Wave 7:

```bash
# Wave 7 best: Trial #6, Sharpe 4.311 (LR=3.14e-5, BS=222, Gamma=0.963)
cargo run -p ml --example backtest_dqn --release --features cuda -- \
  --checkpoint ml/trained_models/hyperopt_trial_6_best.safetensors \
  --baseline ml/trained_models/dqn_baseline.safetensors \
  --data test_data/ES_FUT_validation.parquet \
  --min-sharpe 4.0 \
  --min-win-rate 60.0 \
  --max-drawdown 15.0
```

**Expected**: Trial #6 should exceed all thresholds (Sharpe 4.311 >> 4.0)

---

### 3. 3-Action vs 45-Action Comparison

Compare baseline 3-action system against new 45-action system:

```bash
cargo run -p ml --example backtest_dqn --release --features cuda -- \
  --checkpoint ml/trained_models/dqn_45action_best.safetensors \
  --baseline ml/trained_models/dqn_3action_baseline.safetensors \
  --data test_data/ES_FUT_180d.parquet \
  --output-markdown action_space_comparison.md
```

**Hypothesis** (from Wave 9-11): 45-action system should show:
- Higher action diversity (100% vs ~60%)
- Net positive transaction cost rebates (LimitMaker orders)
- Better risk-adjusted returns (higher Sharpe)

---

### 4. Batch Validation (Multiple Checkpoints)

Validate all epoch checkpoints to find best:

```bash
#!/bin/bash
# validate_all_checkpoints.sh

for epoch in 10 20 30 40 50 60 70 80 90 100; do
  echo "Validating epoch $epoch..."
  cargo run -p ml --example backtest_dqn --release --features cuda -- \
    --checkpoint ml/trained_models/dqn_epoch_${epoch}.safetensors \
    --data test_data/ES_FUT_validation.parquet \
    --output-json results/epoch_${epoch}.json

  if [ $? -eq 0 ]; then
    echo "✅ Epoch $epoch: PASSED"
  else
    echo "❌ Epoch $epoch: FAILED"
  fi
done

# Parse JSON results to find best checkpoint
python3 scripts/find_best_checkpoint.py results/*.json
```

---

## Architecture Details

### Phase 1: Data Loading

1. Load Parquet file → OHLCV bars
2. Extract 128-dimensional features (Wave D)
3. Apply preprocessing (log returns + normalization + clipping)

**Key Files**:
- `ml/src/data_loaders/parquet_utils.rs` - Parquet loading
- `ml/src/features/extraction.rs` - 128-feature extraction
- `ml/src/preprocessing.rs` - Preprocessing pipeline

---

### Phase 2: Model Loading

1. Create `WorkingDQN` configuration (128 input → [256, 128, 64] hidden → 3 actions)
2. Load SafeTensors checkpoint
3. Verify architecture matches training

**Configuration**:
- State dimension: 128 features
- Hidden layers: [256, 128, 64]
- Actions: 3 (BUY, HOLD, SELL)
- Epsilon: 0.0 (greedy evaluation)

---

### Phase 3: Backtest Execution

1. Run greedy inference (epsilon=0) for each bar
2. Execute trades via `EvaluationEngine`
3. Record trade history (entry/exit prices, PnL)

**Trade Logic**:
- **BUY**: Open long position (close short if exists)
- **SELL**: Open short position (close long if exists)
- **HOLD**: Maintain current position
- Final bar: Close any open position

---

### Phase 4: Metrics Calculation

Comprehensive performance metrics from `PerformanceMetrics::from_trades()`:

| Metric | Formula | Description |
|--------|---------|-------------|
| **Total Return** | `(final_equity - initial_capital) / initial_capital * 100` | % gain/loss |
| **Sharpe Ratio** | `mean(returns) / std(returns) * sqrt(252)` | Annualized risk-adjusted return |
| **Max Drawdown** | `max((peak_equity - current_equity) / peak_equity * 100)` | Worst decline from peak |
| **Win Rate** | `winning_trades / total_trades * 100` | % profitable trades |
| **Avg Trade PnL** | `total_pnl / total_trades` | Mean profit/loss per trade |

**Key Files**:
- `ml/src/evaluation/metrics.rs` - Metrics calculation
- `ml/src/evaluation/engine.rs` - Trade execution

---

### Phase 5: Validation & Reporting

1. Compare metrics against success criteria
2. Generate verdict (ProductionReady / Failed)
3. Output reports (console, JSON, markdown)
4. Exit with appropriate code (0=success, 1=failure)

---

## Troubleshooting

### Issue: "Checkpoint file not found"

**Error**:
```
Checkpoint file not found: ml/trained_models/dqn_epoch_5.safetensors
Suggestion: Train a model first using train_dqn example
```

**Solution**:
```bash
# Train a model first
cargo run -p ml --example train_dqn --release --features cuda -- --epochs 10
```

---

### Issue: "Data file not found"

**Error**:
```
Data file not found: test_data/ES_FUT_unseen.parquet
Suggestion: Use test_data/ES_FUT_180d.parquet
```

**Solution**:
```bash
# Use existing validation data
--data test_data/ES_FUT_180d.parquet
```

---

### Issue: "CUDA unavailable"

**Error**:
```
CUDA unavailable. Use --device cpu or --device auto
```

**Solution**:
```bash
# Force CPU execution
--device cpu

# Or use auto-fallback
--device auto
```

---

### Issue: "Feature/bar mismatch"

**Error**:
```
Feature/bar mismatch: 1000 features, 1050 bars
```

**Cause**: Warmup period mismatch (preprocessing removes first 50 bars)

**Solution**: Internal issue - should not occur. If it does, file a bug report.

---

### Issue: "All trades unprofitable"

**Output**:
```
❌ FAILED VALIDATION
     Reasons:
       • Win rate 30.0% < 55.0% (required)
```

**Analysis**:
- Model may be overfitted to training data
- Validation data may be out-of-distribution
- Hyperparameters may need tuning

**Actions**:
1. Check training/validation data similarity
2. Re-run hyperopt with more diverse data
3. Inspect action distribution (should be balanced)
4. Review Q-value statistics (check for collapse)

---

## Integration with Training Pipeline

### Recommended Workflow

```bash
# 1. Train model
cargo run -p ml --example train_dqn --release --features cuda -- \
  --epochs 100 \
  --output-dir /tmp/ml_training/production

# 2. Validate best checkpoint
cargo run -p ml --example backtest_dqn --release --features cuda -- \
  --checkpoint /tmp/ml_training/production/dqn_best_model.safetensors \
  --data test_data/ES_FUT_validation.parquet \
  --output-json validation_results.json

# 3. If validated, deploy to production
if [ $? -eq 0 ]; then
  cp /tmp/ml_training/production/dqn_best_model.safetensors \
     ml/trained_models/dqn_production.safetensors
  echo "✅ Deployed to production"
fi
```

---

## Hyperopt Integration

Use backtest validation as hyperopt objective function:

```python
# scripts/python/hyperopt_with_backtest.py
def objective(trial):
    # 1. Train with trial parameters
    checkpoint = train_dqn_trial(trial)

    # 2. Run backtest validation
    result = run_backtest_validation(checkpoint)

    # 3. Return Sharpe ratio as objective
    return result['metrics']['sharpe_ratio']
```

**Advantage**: Optimize directly for backtest performance instead of training rewards.

---

## Exit Codes

| Code | Meaning | Description |
|------|---------|-------------|
| **0** | Success | Checkpoint meets all success criteria (Production Ready) |
| **1** | Failure | Checkpoint failed one or more success criteria |

**CI/CD Usage**:
```bash
# In GitLab CI / GitHub Actions
cargo run -p ml --example backtest_dqn --release --features cuda -- \
  --checkpoint $CHECKPOINT \
  --data $VALIDATION_DATA \
  --output-json results.json

# Exit code determines pipeline success/failure
```

---

## Performance Benchmarks

### Expected Runtime

| Data Size | Device | Duration | Throughput |
|-----------|--------|----------|------------|
| 1,000 bars | CPU | ~2s | 500 bars/sec |
| 1,000 bars | CUDA | ~1s | 1,000 bars/sec |
| 10,000 bars | CPU | ~15s | 667 bars/sec |
| 10,000 bars | CUDA | ~8s | 1,250 bars/sec |

**Bottlenecks**:
- Data loading: ~0.7ms per bar (DBN legacy, 10ms Parquet)
- Feature extraction: ~1ms per bar (225 features)
- Inference: ~200μs per bar (DQN forward pass)
- Metrics calculation: <1ms total

---

## Output Examples

### Console Output (Failed Validation)

```
╔══════════════════════════════════════════════════════════════════════╗
║              DQN BACKTEST VALIDATION REPORT                          ║
╚══════════════════════════════════════════════════════════════════════╝

═══ Checkpoint ═══
  Primary: ml/trained_models/dqn_epoch_50.safetensors

═══ Performance Metrics ═══
  Total Return:        5.20%
  Sharpe Ratio:        1.45
  Max Drawdown:       22.30%
  Win Rate:           48.5%
  Total Trades:         35
  Avg Trade PnL:       67.89
  Final Equity:    105200.00

═══ Success Criteria ═══
  ❌ Sharpe Ratio ≥ 2.0: 1.45
  ❌ Win Rate ≥ 55.0%: 48.5%
  ❌ Max Drawdown ≤ 20.0%: 22.3%

═══ Verdict ═══
  ❌ FAILED VALIDATION
     Reasons:
       • Sharpe ratio 1.45 < 2.00 (required)
       • Win rate 48.5% < 55.0% (required)
       • Max drawdown 22.3% > 20.0% (limit)

❌ EXIT CODE 1: Validation failed
```

---

## Future Enhancements

### Planned Features (Post-Wave 11)

1. **Multi-checkpoint comparison**: Validate multiple checkpoints in one run
2. **Time-series cross-validation**: Rolling window validation
3. **Custom success criteria**: User-defined validation rules
4. **Detailed trade log export**: CSV export with entry/exit timestamps
5. **Performance attribution**: Breakdown by market regime
6. **Risk metrics**: Sortino ratio, Calmar ratio, Value at Risk
7. **Execution simulation**: Slippage and transaction costs

---

## Related Documentation

- **Training**: `ml/examples/train_dqn.rs` - DQN training pipeline
- **Evaluation**: `ml/examples/evaluate_dqn_main_orchestrator.rs` - Comprehensive evaluation
- **Hyperopt**: `ml/examples/hyperopt_dqn_demo.rs` - Hyperparameter optimization
- **Wave 9-11 Report**: `/tmp/WAVE9_11_PRODUCTION_CERTIFICATION.md` - Production certification
- **Wave 7 Report**: `WAVE7_P&L_VALIDATION_REPORT.md` - Early stopping analysis

---

## Summary

The `backtest_dqn` script provides production-grade validation for DQN checkpoints with:
- ✅ Comprehensive metrics (Sharpe, win rate, drawdown)
- ✅ Baseline comparison support
- ✅ Multiple output formats (console, JSON, markdown)
- ✅ CI/CD integration (exit codes)
- ✅ Configurable success criteria
- ✅ Fast execution (2-15s for 1K-10K bars)

**Recommended Usage**: Validate all production checkpoints before deployment to ensure they meet risk-adjusted return thresholds.