foxhunt/docs/codebase-cleanup/AGENT6_QUICK_SUMMARY.md

Agent 6: Walk-Forward Validation - Quick Summary

✅ Research Complete

Task: Replace fixed 80/20 train/val split with walk-forward validation in DQN hyperopt

---

🎯 Key Findings

1. Existing Implementation Found

• File: ml/src/backtesting/barrier_backtest.rs
• Algorithm: Rolling temporal windows with train/test split per window
• Features: Embargo period, aggregated metrics, stability score

2. Three 80/20 Split Locations

File: ml/src/trainers/dqn/trainer.rs

1. Line 2376: load_training_data_from_parquet()
2. Line 2660: train_with_normalization()
3. Line 2777: load_training_data() (DBN loading)

All three use same pattern:

let split_idx = (data.len() * 80) / 100;
let train_data = data[..split_idx].to_vec();
let val_data = data[split_idx..].to_vec();

---

📋 Implementation Plan

Phase 1: Create New Module (30 min)

File: ml/src/hyperopt/walk_forward.rs

pub struct WalkForwardValidator {
    num_folds: usize,           // Default: 5
    train_ratio: f64,           // Default: 0.8
    embargo_pct: f64,           // Default: 0.01 (1% gap)
}

impl WalkForwardValidator {
    pub fn split<T: Clone>(&self, data: &[T])
        -> Result<Vec<(Vec<T>, Vec<T>)>, MLError> {
        // Split into N folds
        // Each fold: [train][embargo][val]
        // Return Vec of (train, val) tuples
    }
}

Phase 2: Modify DQN Trainer (45 min)

File: ml/src/trainers/dqn/trainer.rs

1. Add fields to DQNTrainer:

   • enable_walk_forward: bool
   • walk_forward_folds: usize
   • walk_forward_embargo_pct: f64

2. Add method:

   pub fn with_walk_forward(mut self, num_folds: usize, embargo_pct: f64) -> Self
   

3. Modify 3 functions to check if self.enable_walk_forward:

   • load_training_data_from_parquet() (line 2376)
   • train_with_normalization() (line 2660)
   • load_training_data() (line 2777)

Phase 3: Enable in Hyperopt (15 min)

File: ml/src/hyperopt/adapters/dqn.rs

Enable by default in DQNTrainer::new():

let mut trainer = Self::with_buffer_max(...)?;
trainer = trainer.with_walk_forward(5, 0.01); // 5 folds, 1% embargo

---

🔧 Files to Modify

New Files (1)

• ml/src/hyperopt/walk_forward.rs

Modified Files (3)

• ml/src/hyperopt/mod.rs (add pub mod walk_forward;)
• ml/src/trainers/dqn/trainer.rs (3 functions)
• ml/src/hyperopt/adapters/dqn.rs (enable by default)

---

✨ Benefits

1. Anti-Overfitting: Validation always AFTER training (no temporal leakage)
2. Embargo Period: 1% gap prevents information leakage at boundaries
3. Robustness: 5 folds instead of 1 split
4. Stability Metric: Variance of Sharpe across folds
5. Production Alignment: Mirrors live trading (always predicting future)

---

🚀 Compilation Check

# After Phase 1
cargo check --package ml --lib

# After Phase 2
cargo test --package ml walk_forward

# After Phase 3
cargo test --package ml dqn::trainer

---

📊 Expected Results

Before (Fixed 80/20)

Training: 80% of data
Validation: 20% of data
Temporal Leakage: Possible
Robustness: Single split

After (Walk-Forward)

Training: Fold 0 (first 80% of fold 1)
Embargo: 1% gap
Validation: Fold 0 (remaining 19% of fold 1)
Temporal Leakage: Prevented
Robustness: 5-fold rotation available

---

🎯 Next Agent Task

Agent 7: Implement walk-forward validation based on analysis

Priority: Create walk_forward.rs module first (smallest, testable unit)

References: See full analysis in AGENT6_WALK_FORWARD_VALIDATION_ANALYSIS.md