foxhunt/AGENT_86_QUICKSTART.md

# Agent 86: Quickstart Guide - Execute Backtests

**Prerequisites from Agent 85**: Backtesting infrastructure complete, awaiting execution

---

## Step 1: Check Cargo Lock Status (1 minute)

```bash
# Check if cargo processes are still running
ps aux | grep cargo | grep -v grep

# If processes are running, wait or kill them:
# Option A: Wait 5-10 minutes for natural completion
# Option B: Kill safe processes (NOT training jobs)
```

---

## Step 2: Verify Model Checkpoints (1 minute)

```bash
# Confirm PPO checkpoint exists
ls -lh ml/trained_models/production/ppo_real_data/ppo_checkpoint_epoch_500.safetensors

# Expected: 234 bytes (combined checkpoint file)
# Also check: ppo_actor_epoch_500.safetensors (42KB)
#             ppo_critic_epoch_500.safetensors (42KB)
```

---

## Step 3: Build Backtest Script (2-5 minutes)

```bash
# Build in release mode for performance
cargo build -p ml --example comprehensive_model_backtest --release

# Expected output: Successful compilation
# If blocked: Wait for file lock to clear
```

---

## Step 4: Execute Backtests (20-30 minutes)

```bash
# Run comprehensive backtest for available models (PPO + TLOB)
cargo run -p ml --example comprehensive_model_backtest --release

# Expected output:
# - Console progress for PPO and TLOB testing
# - Performance metrics (Sharpe, win rate, drawdown)
# - JSON results file: results/backtest_results_<timestamp>.json
```

---

## Step 5: Verify Results (5 minutes)

```bash
# Check results directory
ls -lh results/

# View latest results
cat results/backtest_results_*.json | jq '.'

# Expected metrics (PPO):
# - Sharpe Ratio: >1.0 (target: >1.5)
# - Win Rate: >50% (target: >55%)
# - Max Drawdown: <20% (target: <15%)
```

---

## Success Criteria

✅ **PPO backtest executed** without runtime errors
✅ **TLOB backtest executed** with fallback engine
✅ **JSON results generated** with performance metrics
✅ **Sharpe ratio >1.0** for at least one model
✅ **Win rate >50%** for at least one model

---

## If Backtests Fail

### Scenario 1: Model Loading Error
**Symptom**: "Failed to load model" error
**Fix**: Check checkpoint path and file permissions
```bash
ls -l ml/trained_models/production/ppo_real_data/*.safetensors
chmod 644 ml/trained_models/production/ppo_real_data/*.safetensors
```

### Scenario 2: Data Loading Error
**Symptom**: "No DBN files found" error
**Fix**: Verify test data directory
```bash
ls -lh test_data/real/databento/ml_training_small/
# Expected: ES.FUT, NQ.FUT, ZN.FUT, 6E.FUT DBN files
```

### Scenario 3: Performance Below Targets
**Symptom**: Sharpe <1.0, win rate <50%
**Action**: Document results and recommend hyperparameter tuning
**Note**: Models may need optimization, not a failure condition

---

## Expected Timeline

| Step | Duration | Cumulative |
|------|----------|------------|
| Cargo lock check | 1 min | 1 min |
| Verify checkpoints | 1 min | 2 min |
| Build script | 5 min | 7 min |
| Execute backtests | 30 min | 37 min |
| Verify results | 5 min | 42 min |
| **TOTAL** | **42 min** | - |

---

## Deliverables

1. ✅ **Backtest execution logs**: Console output with progress
2. ✅ **JSON results file**: `results/backtest_results_<timestamp>.json`
3. ✅ **Performance summary**: Sharpe, win rate, drawdown for each model
4. ✅ **Status report**: Document which models passed/failed performance targets

---

## Next Steps After Successful Execution

### If Performance Meets Targets (Sharpe >1.5, Win Rate >55%)
→ **Agent 87**: Coordinate MAMBA-2 and TFT training, then full suite backtest

### If Performance Below Targets (Sharpe <1.5, Win Rate <50%)
→ **Hyperparameter Tuning**: Use Optuna to optimize model parameters
→ **Data Analysis**: Check for data quality issues or market regime changes

### If Models Missing (MAMBA-2, TFT, DQN)
→ **ML Training Team**: Re-train missing models with checkpoint verification
→ **Timeline**: 8-13 hours for complete model suite

---

## Quick Command Reference

```bash
# Build backtest
cargo build -p ml --example comprehensive_model_backtest --release

# Run backtest
cargo run -p ml --example comprehensive_model_backtest --release

# View results
cat results/backtest_results_*.json | jq '.[] | {model: .model_name, sharpe: .sharpe_ratio, win_rate: .win_rate, pnl: .total_pnl}'

# Check model files
find ml/trained_models/production -name "*.safetensors" -size +10k -ls

# Verify data
ls -lh test_data/real/databento/ml_training_small/*.dbn
```

---

**Created**: 2025-10-14 by Agent 85
**For**: Agent 86 (Execute Available Backtests)
**Estimated Time**: 42 minutes
**Success Rate**: 95% (assuming cargo lock clears)