Files

jgrusewski c3b5e124f0 chore: update .gitignore and add design plan docs

Ignore ML checkpoints, trained model safetensors, stray ml/ml/ dir,
and .claude/worktrees/. Clean up duplicate hive-mind-prompt entries.
Add 17 design/implementation plan docs from 2026-02-20 to 2026-02-22.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

2026-02-24 00:30:15 +01:00

72 KiB

Raw Blame History

Statistical Validation Stack Implementation Plan

For Claude: REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

Goal: Build an algorithm-agnostic statistical validation harness (walk-forward, DSR, PBO, permutation tests, per-regime analysis) that proves whether trading strategies are statistically real or overfit noise.

Architecture: Layered pipeline in ml/src/validation/ — five modules (walk_forward, statistical, regime_analysis, harness) composed by an orchestrator, all behind a ValidatableStrategy trait. Existing validation.rs becomes validation/financial.rs inside the new directory module.

Tech Stack: Rust, Candle ML, chrono (timestamps), rand/rand_chacha (permutation RNG). No new dependencies needed — all already in ml/Cargo.toml.

Safety gate for every task: SQLX_OFFLINE=true cargo check --manifest-path /home/jgrusewski/Work/foxhunt/ml/Cargo.toml must pass.

Task 1: Convert validation.rs to directory module

Convert the existing ml/src/validation.rs (financial validation) into a directory module so we can add new submodules alongside it.

Files:

Move: ml/src/validation.rs → ml/src/validation/financial.rs
Create: ml/src/validation/mod.rs
Delete orphan: ml/src/validation/numerical_tests.rs (unreferenced file in existing validation dir)

Step 1: Move validation.rs to validation/financial.rs

Move the existing file. The validation/ directory already exists (contains orphan numerical_tests.rs).

mv /home/jgrusewski/Work/foxhunt/ml/src/validation.rs /home/jgrusewski/Work/foxhunt/ml/src/validation/financial.rs

Step 2: Delete the orphan numerical_tests.rs

This file is unreferenced — it imports types that don't exist in the current module tree (MLModel, ModelType, etc.) and was never compiled.

rm /home/jgrusewski/Work/foxhunt/ml/src/validation/numerical_tests.rs

Step 3: Create validation/mod.rs

//! Statistical and Financial Validation Module
//!
//! Provides two categories of validation:
//! - Financial validation (price/volume/quantity sanity checks)
//! - Statistical validation (walk-forward, DSR, PBO, permutation tests)

pub mod financial;

// Re-export financial validation for backward compatibility
pub use financial::{
    validate_model_basic, validate_model_comprehensive, validate_type_conversions,
    FinancialValidationResult, ValidationResult,
};

Step 4: Verify compilation

Run: SQLX_OFFLINE=true cargo check --manifest-path /home/jgrusewski/Work/foxhunt/ml/Cargo.toml Expected: Compiles clean (0 errors)

Step 5: Run existing tests

Run: SQLX_OFFLINE=true cargo test --manifest-path /home/jgrusewski/Work/foxhunt/ml/Cargo.toml --lib 2>&1 | tail -5 Expected: All tests pass (same count as before: 1823+)

Step 6: Commit

git add ml/src/validation/mod.rs ml/src/validation/financial.rs
git add ml/src/validation.rs  # stages deletion
git add ml/src/validation/numerical_tests.rs  # stages deletion
git commit -m "refactor(validation): convert to directory module for validation stack"

Task 2: TimeSeriesData struct + ValidatableStrategy trait

Create the core data type and trait that all validation modules depend on.

Files:

Modify: ml/src/validation/mod.rs
Create: ml/src/validation/types.rs
Test: inline #[cfg(test)] in types.rs

Step 1: Write the failing test

Create ml/src/validation/types.rs with tests that reference structs/traits that don't exist yet:

//! Core types for the statistical validation stack.

use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};

use crate::MLError;

/// Time-series data for validation — features, prices, and returns aligned by timestamp.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct TimeSeriesData {
    /// Timestamps for each bar (must be monotonically increasing)
    pub timestamps: Vec<DateTime<Utc>>,
    /// Feature matrix: `features[i]` is the feature vector for bar `i`
    pub features: Vec<Vec<f32>>,
    /// Close prices for PnL computation
    pub prices: Vec<f64>,
    /// Log returns derived from prices: `returns[i] = ln(prices[i+1] / prices[i])`
    /// Length is `prices.len() - 1`
    pub returns: Vec<f64>,
}

impl TimeSeriesData {
    /// Create TimeSeriesData from raw components, computing returns automatically.
    ///
    /// # Errors
    /// Returns `MLError::InvalidInput` if lengths don't match or prices has < 2 elements.
    pub fn new(
        timestamps: Vec<DateTime<Utc>>,
        features: Vec<Vec<f32>>,
        prices: Vec<f64>,
    ) -> Result<Self, MLError> {
        if timestamps.len() != features.len() || timestamps.len() != prices.len() {
            return Err(MLError::InvalidInput(format!(
                "Length mismatch: timestamps={}, features={}, prices={}",
                timestamps.len(),
                features.len(),
                prices.len()
            )));
        }
        if prices.len() < 2 {
            return Err(MLError::InvalidInput(
                "Need at least 2 price points to compute returns".to_string(),
            ));
        }

        // Compute log returns
        let returns: Vec<f64> = prices
            .windows(2)
            .map(|w| {
                if w[0] > 0.0 {
                    (w[1] / w[0]).ln()
                } else {
                    0.0
                }
            })
            .collect();

        Ok(Self {
            timestamps,
            features,
            prices,
            returns,
        })
    }

    /// Number of bars in this dataset.
    pub fn len(&self) -> usize {
        self.timestamps.len()
    }

    /// Whether the dataset is empty.
    pub fn is_empty(&self) -> bool {
        self.timestamps.is_empty()
    }

    /// Slice this dataset to a given index range (inclusive start, exclusive end).
    ///
    /// # Errors
    /// Returns `MLError::InvalidInput` if range is out of bounds or too small.
    pub fn slice(&self, start: usize, end: usize) -> Result<Self, MLError> {
        if end > self.len() || start >= end {
            return Err(MLError::InvalidInput(format!(
                "Invalid slice range {}..{} for data of length {}",
                start,
                end,
                self.len()
            )));
        }
        if end - start < 2 {
            return Err(MLError::InvalidInput(
                "Slice must contain at least 2 bars".to_string(),
            ));
        }
        Self::new(
            self.timestamps.get(start..end).unwrap_or_default().to_vec(),
            self.features.get(start..end).unwrap_or_default().to_vec(),
            self.prices.get(start..end).unwrap_or_default().to_vec(),
        )
    }
}

/// Trait for any model/strategy that can be validated via walk-forward analysis.
///
/// Implement this for each algorithm (DQN, PPO, TFT, etc.) to enable
/// statistical validation through the `ValidationHarness`.
pub trait ValidatableStrategy: Send {
    /// Train the model on the given time-series data.
    fn train(&mut self, data: &TimeSeriesData) -> Result<(), MLError>;

    /// Evaluate the model on test data, returning daily PnL returns.
    ///
    /// The returned vector should have one entry per bar in `data`
    /// representing the strategy's return for that bar.
    fn evaluate(&self, data: &TimeSeriesData) -> Result<Vec<f64>, MLError>;

    /// Human-readable name for reporting.
    fn name(&self) -> &str;

    /// Reset model weights for fresh fold training.
    fn reset(&mut self) -> Result<(), MLError>;
}

#[cfg(test)]
mod tests {
    use super::*;
    use chrono::{Duration, Utc};

    fn make_test_data(n: usize) -> TimeSeriesData {
        let start = Utc::now();
        let timestamps: Vec<_> = (0..n)
            .map(|i| start + Duration::hours(i as i64))
            .collect();
        let features: Vec<Vec<f32>> = (0..n).map(|i| vec![i as f32; 4]).collect();
        let prices: Vec<f64> = (0..n).map(|i| 100.0 + i as f64 * 0.5).collect();
        TimeSeriesData::new(timestamps, features, prices).expect("valid test data")
    }

    #[test]
    fn test_time_series_data_new_computes_returns() {
        let data = make_test_data(10);
        assert_eq!(data.len(), 10);
        assert_eq!(data.returns.len(), 9); // n-1 returns
        // First return: ln(100.5 / 100.0)
        let expected = (100.5_f64 / 100.0).ln();
        assert!((data.returns[0] - expected).abs() < 1e-10);
    }

    #[test]
    fn test_time_series_data_length_mismatch_errors() {
        let start = Utc::now();
        let ts = vec![start, start + Duration::hours(1)];
        let features = vec![vec![1.0f32]; 3]; // wrong length
        let prices = vec![100.0, 101.0];
        let result = TimeSeriesData::new(ts, features, prices);
        assert!(result.is_err());
    }

    #[test]
    fn test_time_series_data_too_short_errors() {
        let start = Utc::now();
        let result = TimeSeriesData::new(vec![start], vec![vec![1.0]], vec![100.0]);
        assert!(result.is_err());
    }

    #[test]
    fn test_slice_valid_range() {
        let data = make_test_data(20);
        let sliced = data.slice(5, 15).expect("valid slice");
        assert_eq!(sliced.len(), 10);
        assert_eq!(sliced.returns.len(), 9);
    }

    #[test]
    fn test_slice_out_of_bounds_errors() {
        let data = make_test_data(10);
        assert!(data.slice(0, 20).is_err());
        assert!(data.slice(8, 8).is_err());
    }
}

Step 2: Register the module in mod.rs

Add to ml/src/validation/mod.rs:

pub mod types;

pub use types::{TimeSeriesData, ValidatableStrategy};

Step 3: Run test to verify it passes

Run: SQLX_OFFLINE=true cargo test --manifest-path /home/jgrusewski/Work/foxhunt/ml/Cargo.toml --lib validation::types -- --nocapture 2>&1 | tail -10 Expected: 5 tests pass

Step 4: Verify full compilation

Run: SQLX_OFFLINE=true cargo check --manifest-path /home/jgrusewski/Work/foxhunt/ml/Cargo.toml Expected: Compiles clean

Step 5: Commit

git add ml/src/validation/types.rs ml/src/validation/mod.rs
git commit -m "feat(validation): add TimeSeriesData struct and ValidatableStrategy trait"

Task 3: Walk-forward splitter

Implement the walk-forward cross-validation splitter with embargo periods. This replaces the stubbed walk_forward_split() that currently returns vec![].

Files:

Create: ml/src/validation/walk_forward.rs
Modify: ml/src/validation/mod.rs (add module declaration)

Step 1: Write walk_forward.rs with tests first, then implementation

//! Walk-forward cross-validation splitter with embargo periods.
//!
//! Produces non-overlapping folds that maintain strict temporal ordering:
//! `[train] [embargo] [test] → step → [train] [embargo] [test] → ...`
//!
//! The embargo gap prevents information leakage from features with lookback windows
//! (e.g., a 20-bar moving average would need embargo_bars >= 20).

use std::ops::Range;

use serde::{Deserialize, Serialize};

/// Configuration for walk-forward cross-validation.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct WalkForwardConfig {
    /// Number of bars in each training window.
    pub train_bars: usize,
    /// Number of bars in each test window.
    pub test_bars: usize,
    /// Number of bars in the embargo gap between train and test.
    /// Must be >= the maximum lookback window of any feature.
    pub embargo_bars: usize,
    /// How far to advance the window each fold (in bars).
    pub step_bars: usize,
    /// Minimum number of training bars required (folds with fewer are skipped).
    pub min_train_samples: usize,
}

impl Default for WalkForwardConfig {
    fn default() -> Self {
        Self {
            train_bars: 252 * 5,    // ~5 years of daily bars
            test_bars: 252,         // ~1 year
            embargo_bars: 20,       // 20-bar lookback gap
            step_bars: 126,         // ~6 months step
            min_train_samples: 252, // at least 1 year of training data
        }
    }
}

/// A single fold in walk-forward cross-validation.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Fold {
    /// Zero-based fold index.
    pub fold_index: usize,
    /// Index range for training data (into the original TimeSeriesData).
    pub train_range: Range<usize>,
    /// Index range for the embargo gap (excluded from both train and test).
    pub embargo_range: Range<usize>,
    /// Index range for test data.
    pub test_range: Range<usize>,
}

/// Generate walk-forward folds from a dataset of `num_bars` bars.
///
/// Returns an empty vec if the data is too short for even one valid fold.
pub fn walk_forward_split(num_bars: usize, config: &WalkForwardConfig) -> Vec<Fold> {
    let mut folds = Vec::new();
    let min_fold_size = config.train_bars + config.embargo_bars + config.test_bars;

    if num_bars < min_fold_size || config.train_bars < config.min_train_samples {
        return folds;
    }

    let mut fold_index = 0;
    let mut train_start = 0;

    loop {
        let train_end = train_start + config.train_bars;
        let embargo_end = train_end + config.embargo_bars;
        let test_end = embargo_end + config.test_bars;

        // Stop if test window exceeds data
        if test_end > num_bars {
            break;
        }

        // Only include fold if training window meets minimum
        if config.train_bars >= config.min_train_samples {
            folds.push(Fold {
                fold_index,
                train_range: train_start..train_end,
                embargo_range: train_end..embargo_end,
                test_range: embargo_end..test_end,
            });
            fold_index += 1;
        }

        train_start += config.step_bars;
    }

    folds
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_basic_walk_forward_split() {
        let config = WalkForwardConfig {
            train_bars: 100,
            test_bars: 30,
            embargo_bars: 5,
            step_bars: 30,
            min_train_samples: 50,
        };
        let folds = walk_forward_split(300, &config);
        assert!(!folds.is_empty(), "Should produce at least one fold");

        // Verify first fold
        assert_eq!(folds[0].train_range, 0..100);
        assert_eq!(folds[0].embargo_range, 100..105);
        assert_eq!(folds[0].test_range, 105..135);

        // Verify second fold steps forward
        assert_eq!(folds[1].train_range, 30..130);
        assert_eq!(folds[1].embargo_range, 130..135);
        assert_eq!(folds[1].test_range, 135..165);
    }

    #[test]
    fn test_no_overlap_between_train_and_test() {
        let config = WalkForwardConfig {
            train_bars: 50,
            test_bars: 20,
            embargo_bars: 10,
            step_bars: 20,
            min_train_samples: 10,
        };
        let folds = walk_forward_split(200, &config);
        for fold in &folds {
            // Train must end before embargo starts
            assert_eq!(fold.train_range.end, fold.embargo_range.start);
            // Embargo must end before test starts
            assert_eq!(fold.embargo_range.end, fold.test_range.start);
            // No overlap within a fold
            assert!(fold.train_range.end <= fold.embargo_range.start);
            assert!(fold.embargo_range.end <= fold.test_range.start);
        }
    }

    #[test]
    fn test_temporal_ordering_across_folds() {
        let config = WalkForwardConfig {
            train_bars: 50,
            test_bars: 20,
            embargo_bars: 5,
            step_bars: 25,
            min_train_samples: 10,
        };
        let folds = walk_forward_split(300, &config);
        // Each fold's test start should be >= previous fold's test start
        for window in folds.windows(2) {
            assert!(
                window[1].test_range.start >= window[0].test_range.start,
                "Folds must advance in time"
            );
        }
    }

    #[test]
    fn test_data_too_short_returns_empty() {
        let config = WalkForwardConfig {
            train_bars: 100,
            test_bars: 50,
            embargo_bars: 10,
            step_bars: 50,
            min_train_samples: 100,
        };
        // Need 160 bars minimum, only have 100
        let folds = walk_forward_split(100, &config);
        assert!(folds.is_empty());
    }

    #[test]
    fn test_fold_indices_are_sequential() {
        let config = WalkForwardConfig {
            train_bars: 30,
            test_bars: 10,
            embargo_bars: 2,
            step_bars: 10,
            min_train_samples: 10,
        };
        let folds = walk_forward_split(100, &config);
        for (i, fold) in folds.iter().enumerate() {
            assert_eq!(fold.fold_index, i);
        }
    }

    #[test]
    fn test_test_range_within_bounds() {
        let num_bars = 200;
        let config = WalkForwardConfig {
            train_bars: 50,
            test_bars: 20,
            embargo_bars: 5,
            step_bars: 20,
            min_train_samples: 10,
        };
        let folds = walk_forward_split(num_bars, &config);
        for fold in &folds {
            assert!(fold.test_range.end <= num_bars, "Test range exceeds data bounds");
        }
    }

    #[test]
    fn test_default_config() {
        let config = WalkForwardConfig::default();
        assert_eq!(config.embargo_bars, 20);
        assert_eq!(config.train_bars, 252 * 5);
    }
}

Step 2: Register in mod.rs

Add to ml/src/validation/mod.rs:

pub mod walk_forward;

pub use walk_forward::{walk_forward_split, Fold, WalkForwardConfig};

Step 3: Run tests

Run: SQLX_OFFLINE=true cargo test --manifest-path /home/jgrusewski/Work/foxhunt/ml/Cargo.toml --lib validation::walk_forward -- --nocapture 2>&1 | tail -10 Expected: 7 tests pass

Step 4: Verify full compilation

Run: SQLX_OFFLINE=true cargo check --manifest-path /home/jgrusewski/Work/foxhunt/ml/Cargo.toml

Step 5: Commit

git add ml/src/validation/walk_forward.rs ml/src/validation/mod.rs
git commit -m "feat(validation): implement walk-forward splitter with embargo periods"

Task 4: Statistical math helpers (normal CDF, Sharpe, skewness, kurtosis)

Build the mathematical primitives needed by DSR, PBO, and permutation tests.

Files:

Create: ml/src/validation/statistical.rs
Modify: ml/src/validation/mod.rs

Step 1: Write statistical.rs with helpers and their tests

//! Statistical tests for strategy validation.
//!
//! Implements:
//! - Deflated Sharpe Ratio (Bailey & Lopez de Prado, 2014)
//! - Probability of Backtest Overfitting via CSCV (Bailey et al., 2017)
//! - Monte Carlo permutation tests
//!
//! All statistical math (normal CDF, Sharpe, moments) is self-contained
//! with no external dependencies beyond `rand`.

use rand::seq::SliceRandom;
use rand::SeedableRng;
use rand_chacha::ChaCha8Rng;
use serde::{Deserialize, Serialize};

// ══════════════════════════════════════════════════════
// Math helpers
// ══════════════════════════════════════════════════════

/// Standard normal CDF using Abramowitz & Stegun rational approximation (error < 7.5e-8).
pub fn normal_cdf(x: f64) -> f64 {
    if x < -8.0 {
        return 0.0;
    }
    if x > 8.0 {
        return 1.0;
    }

    let a1 = 0.254829592;
    let a2 = -0.284496736;
    let a3 = 1.421413741;
    let a4 = -1.453152027;
    let a5 = 1.061405429;
    let p = 0.3275911;

    let sign = if x < 0.0 { -1.0 } else { 1.0 };
    let x_abs = x.abs() / std::f64::consts::SQRT_2;

    let t = 1.0 / (1.0 + p * x_abs);
    let t2 = t * t;
    let t3 = t2 * t;
    let t4 = t3 * t;
    let t5 = t4 * t;

    let y = 1.0 - (a1 * t + a2 * t2 + a3 * t3 + a4 * t4 + a5 * t5) * (-x_abs * x_abs).exp();

    0.5 * (1.0 + sign * y)
}

/// Inverse normal CDF (probit function) via rational approximation.
/// Accurate to ~1e-5 for p in (0.0003, 0.9997).
pub fn normal_ppf(p: f64) -> f64 {
    if p <= 0.0 {
        return f64::NEG_INFINITY;
    }
    if p >= 1.0 {
        return f64::INFINITY;
    }
    if (p - 0.5).abs() < 1e-15 {
        return 0.0;
    }

    // Rational approximation (Peter Acklam's algorithm)
    let a = [
        -3.969_683_028_665_376e1,
        2.209_460_984_245_205e2,
        -2.759_285_104_469_687e2,
        1.383_577_518_672_690e2,
        -3.066_479_806_614_716e1,
        2.506_628_277_459_239,
    ];
    let b = [
        -5.447_609_879_822_406e1,
        1.615_858_368_580_409e2,
        -1.556_989_798_598_866e2,
        6.680_131_188_771_972e1,
        -1.328_068_155_288_572e1,
    ];
    let c = [
        -7.784_894_002_430_293e-3,
        -3.223_964_580_411_365e-1,
        -2.400_758_277_161_838,
        -2.549_732_539_343_734,
        4.374_664_141_464_968,
        2.938_163_982_698_783,
    ];
    let d = [
        7.784_695_709_041_462e-3,
        3.224_671_290_700_398e-1,
        2.445_134_137_142_996,
        3.754_408_661_907_416,
    ];

    let p_low = 0.02425;
    let p_high = 1.0 - p_low;

    if p < p_low {
        let q = (-2.0 * p.ln()).sqrt();
        (((((c[0] * q + c[1]) * q + c[2]) * q + c[3]) * q + c[4]) * q + c[5])
            / ((((d[0] * q + d[1]) * q + d[2]) * q + d[3]) * q + 1.0)
    } else if p <= p_high {
        let q = p - 0.5;
        let r = q * q;
        (((((a[0] * r + a[1]) * r + a[2]) * r + a[3]) * r + a[4]) * r + a[5]) * q
            / (((((b[0] * r + b[1]) * r + b[2]) * r + b[3]) * r + b[4]) * r + 1.0)
    } else {
        let q = (-2.0 * (1.0 - p).ln()).sqrt();
        -(((((c[0] * q + c[1]) * q + c[2]) * q + c[3]) * q + c[4]) * q + c[5])
            / ((((d[0] * q + d[1]) * q + d[2]) * q + d[3]) * q + 1.0)
    }
}

/// Compute Sharpe ratio from a series of returns (not annualized).
pub fn sharpe_ratio(returns: &[f64]) -> f64 {
    if returns.len() < 2 {
        return 0.0;
    }
    let n = returns.len() as f64;
    let mean = returns.iter().sum::<f64>() / n;
    let variance = returns.iter().map(|&r| (r - mean).powi(2)).sum::<f64>() / (n - 1.0);
    let std_dev = variance.sqrt();
    if std_dev < 1e-15 {
        return 0.0;
    }
    mean / std_dev
}

/// Compute skewness of a return series.
pub fn skewness(returns: &[f64]) -> f64 {
    if returns.len() < 3 {
        return 0.0;
    }
    let n = returns.len() as f64;
    let mean = returns.iter().sum::<f64>() / n;
    let variance = returns.iter().map(|&r| (r - mean).powi(2)).sum::<f64>() / (n - 1.0);
    let std_dev = variance.sqrt();
    if std_dev < 1e-15 {
        return 0.0;
    }
    let m3 = returns.iter().map(|&r| ((r - mean) / std_dev).powi(3)).sum::<f64>();
    (n / ((n - 1.0) * (n - 2.0))) * m3
}

/// Compute excess kurtosis of a return series.
pub fn excess_kurtosis(returns: &[f64]) -> f64 {
    if returns.len() < 4 {
        return 0.0;
    }
    let n = returns.len() as f64;
    let mean = returns.iter().sum::<f64>() / n;
    let m2 = returns.iter().map(|&r| (r - mean).powi(2)).sum::<f64>() / n;
    if m2 < 1e-30 {
        return 0.0;
    }
    let m4 = returns.iter().map(|&r| (r - mean).powi(4)).sum::<f64>() / n;
    (m4 / (m2 * m2)) - 3.0
}

// ══════════════════════════════════════════════════════
// Deflated Sharpe Ratio
// ══════════════════════════════════════════════════════

/// Result of the Deflated Sharpe Ratio test.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct DsrResult {
    /// The observed (best) Sharpe ratio being tested.
    pub observed_sharpe: f64,
    /// Expected maximum Sharpe under the null hypothesis (no skill).
    pub expected_max_sharpe: f64,
    /// Standard error of the Sharpe ratio estimator.
    pub sharpe_std_error: f64,
    /// The DSR test statistic.
    pub deflated_sharpe: f64,
    /// p-value: probability of observing this Sharpe by chance.
    pub pvalue: f64,
}

/// Compute the Deflated Sharpe Ratio (Bailey & Lopez de Prado, 2014).
///
/// # Arguments
/// * `observed_sharpe` - Sharpe ratio of the best strategy found
/// * `num_trials` - Number of strategies/hyperopt trials tested
/// * `sharpe_variance` - Variance of Sharpe ratios across trials
/// * `skew` - Skewness of the return series
/// * `kurt` - Excess kurtosis of the return series
/// * `num_observations` - Total number of return observations
pub fn deflated_sharpe_ratio(
    observed_sharpe: f64,
    num_trials: usize,
    sharpe_variance: f64,
    skew: f64,
    kurt: f64,
    num_observations: usize,
) -> DsrResult {
    let n = num_observations as f64;
    let num_trials_f = num_trials as f64;

    // Expected maximum Sharpe ratio under null (Euler-Mascheroni approximation)
    let gamma = 0.5772156649015329; // Euler-Mascheroni constant
    let sharpe_std = sharpe_variance.sqrt().max(1e-15);

    let expected_max_sharpe = if num_trials <= 1 {
        0.0
    } else {
        let z = normal_ppf(1.0 - 1.0 / num_trials_f);
        let z_e = normal_ppf(1.0 - 1.0 / (num_trials_f * std::f64::consts::E));
        sharpe_std * ((1.0 - gamma) * z + gamma * z_e)
    };

    // Standard error of the Sharpe ratio (adjusted for non-normality)
    let sr = observed_sharpe;
    let se_sr = if n > 1.0 {
        ((1.0 - skew * sr + ((kurt - 1.0) / 4.0) * sr * sr) / (n - 1.0))
            .max(0.0)
            .sqrt()
    } else {
        1.0
    };

    // DSR statistic
    let deflated_sharpe = if se_sr > 1e-15 {
        (sr - expected_max_sharpe) / se_sr
    } else {
        0.0
    };

    // p-value (one-sided: probability observed SR exceeds expected max by chance)
    let pvalue = 1.0 - normal_cdf(deflated_sharpe);

    DsrResult {
        observed_sharpe,
        expected_max_sharpe,
        sharpe_std_error: se_sr,
        deflated_sharpe,
        pvalue,
    }
}

// ══════════════════════════════════════════════════════
// Monte Carlo Permutation Test
// ══════════════════════════════════════════════════════

/// Result of a Monte Carlo permutation test.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct PermutationResult {
    /// Observed Sharpe ratio of the actual return series.
    pub observed_sharpe: f64,
    /// p-value: fraction of permuted Sharpes >= observed Sharpe.
    pub pvalue: f64,
    /// Number of permutations run.
    pub num_permutations: usize,
    /// Mean of the null distribution.
    pub null_mean: f64,
    /// Std dev of the null distribution.
    pub null_std: f64,
}

/// Run a Monte Carlo permutation test on daily returns.
///
/// Shuffles the return series (destroying temporal structure / any signal),
/// recomputes Sharpe on each permutation, and reports what fraction of
/// permuted Sharpes are >= the observed Sharpe.
///
/// # Arguments
/// * `daily_returns` - The actual return series to test
/// * `num_permutations` - Number of shuffles (default: 10_000)
/// * `seed` - RNG seed for reproducibility
pub fn permutation_test(
    daily_returns: &[f64],
    num_permutations: usize,
    seed: u64,
) -> PermutationResult {
    let observed = sharpe_ratio(daily_returns);

    if daily_returns.len() < 2 || num_permutations == 0 {
        return PermutationResult {
            observed_sharpe: observed,
            pvalue: 1.0,
            num_permutations,
            null_mean: 0.0,
            null_std: 0.0,
        };
    }

    let mut rng = ChaCha8Rng::seed_from_u64(seed);
    let mut shuffled = daily_returns.to_vec();
    let mut count_ge = 0usize;
    let mut null_sharpes = Vec::with_capacity(num_permutations);

    for _ in 0..num_permutations {
        shuffled.shuffle(&mut rng);
        let perm_sharpe = sharpe_ratio(&shuffled);
        null_sharpes.push(perm_sharpe);
        if perm_sharpe >= observed {
            count_ge += 1;
        }
        // Restore original order for next shuffle
        shuffled.copy_from_slice(daily_returns);
    }

    let null_mean = null_sharpes.iter().sum::<f64>() / null_sharpes.len() as f64;
    let null_std = if null_sharpes.len() > 1 {
        let var = null_sharpes
            .iter()
            .map(|&s| (s - null_mean).powi(2))
            .sum::<f64>()
            / (null_sharpes.len() - 1) as f64;
        var.sqrt()
    } else {
        0.0
    };

    PermutationResult {
        observed_sharpe: observed,
        pvalue: count_ge as f64 / num_permutations as f64,
        num_permutations,
        null_mean,
        null_std,
    }
}

// ══════════════════════════════════════════════════════
// Probability of Backtest Overfitting (PBO)
// ══════════════════════════════════════════════════════

/// Result of the PBO analysis.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct PboResult {
    /// Probability of backtest overfitting [0, 1].
    /// PBO > 0.5 = likely overfit, PBO < 0.25 = strong evidence of real signal.
    pub pbo: f64,
    /// Number of CSCV combinations tested.
    pub num_combinations: usize,
    /// Distribution of logit(rank) values for detailed analysis.
    pub logit_distribution: Vec<f64>,
}

/// Compute Probability of Backtest Overfitting via CSCV.
///
/// Uses Combinatorially Symmetric Cross-Validation:
/// 1. Given N fold Sharpes, generate C(N, N/2) combinations
/// 2. For each: split into IS/OOS halves, find IS-best, check OOS rank
/// 3. PBO = fraction where IS-best underperforms OOS median
///
/// # Arguments
/// * `per_fold_sharpes` - Sharpe ratios for each walk-forward fold (N folds).
///   N must be even (>=4) for CSCV to work.
///
/// # Note
/// For large N, this uses sampling instead of exhaustive enumeration
/// to keep computation tractable (C(16,8) = 12870 combinations).
pub fn probability_of_backtest_overfitting(per_fold_sharpes: &[f64]) -> PboResult {
    let n = per_fold_sharpes.len();

    // CSCV requires even N >= 4
    if n < 4 || n % 2 != 0 {
        return PboResult {
            pbo: 0.5, // uninformative prior
            num_combinations: 0,
            logit_distribution: vec![],
        };
    }

    let half = n / 2;
    let combinations = generate_combinations(n, half);

    let mut logits = Vec::with_capacity(combinations.len());
    let mut overfit_count = 0usize;

    for combo in &combinations {
        // IS = indices in combo, OOS = indices not in combo
        let is_sharpes: Vec<f64> = combo.iter().map(|&i| per_fold_sharpes[i]).collect();
        let oos_indices: Vec<usize> = (0..n).filter(|i| !combo.contains(i)).collect();
        let oos_sharpes: Vec<f64> = oos_indices.iter().map(|&i| per_fold_sharpes[i]).collect();

        // Find IS-best fold index (within the combo)
        let is_best_combo_idx = is_sharpes
            .iter()
            .enumerate()
            .max_by(|(_, a), (_, b)| a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal))
            .map(|(i, _)| i)
            .unwrap_or(0);

        // The IS-best fold's original index
        let is_best_original_idx = combo[is_best_combo_idx];

        // What is this fold's Sharpe in OOS context?
        // (it's the same value — we're checking how its rank compares)
        let is_best_sharpe = per_fold_sharpes[is_best_original_idx];

        // Rank among OOS sharpes: how many OOS sharpes beat it?
        let rank = oos_sharpes.iter().filter(|&&s| s > is_best_sharpe).count();
        let relative_rank = rank as f64 / half as f64;

        // Logit: ln(rank / (1 - rank)), clamped to avoid infinity
        let clamped_rank = relative_rank.clamp(0.01, 0.99);
        let logit = (clamped_rank / (1.0 - clamped_rank)).ln();
        logits.push(logit);

        // IS-best underperforms OOS median if rank > N/4 (worse than middle)
        if relative_rank > 0.5 {
            overfit_count += 1;
        }
    }

    let num_combinations = combinations.len();
    let pbo = if num_combinations > 0 {
        overfit_count as f64 / num_combinations as f64
    } else {
        0.5
    };

    PboResult {
        pbo,
        num_combinations,
        logit_distribution: logits,
    }
}

/// Generate all C(n, k) combinations of indices 0..n.
/// For large n, caps at 10_000 random samples to keep computation tractable.
fn generate_combinations(n: usize, k: usize) -> Vec<Vec<usize>> {
    // Estimate C(n, k) — if too large, sample instead
    let max_combinations = 10_000;
    let estimated = binomial_coefficient(n, k);

    if estimated <= max_combinations {
        // Exhaustive enumeration
        let mut result = Vec::new();
        let mut current = Vec::with_capacity(k);
        enumerate_combinations(n, k, 0, &mut current, &mut result);
        result
    } else {
        // Random sampling
        let mut rng = ChaCha8Rng::seed_from_u64(42);
        let indices: Vec<usize> = (0..n).collect();
        let mut result = Vec::with_capacity(max_combinations);
        let mut seen = std::collections::HashSet::new();

        while result.len() < max_combinations {
            let mut sample = indices.clone();
            sample.shuffle(&mut rng);
            let mut combo: Vec<usize> = sample.into_iter().take(k).collect();
            combo.sort_unstable();

            if seen.insert(combo.clone()) {
                result.push(combo);
            }
        }
        result
    }
}

fn enumerate_combinations(
    n: usize,
    k: usize,
    start: usize,
    current: &mut Vec<usize>,
    result: &mut Vec<Vec<usize>>,
) {
    if current.len() == k {
        result.push(current.clone());
        return;
    }
    let remaining = k - current.len();
    for i in start..=(n - remaining) {
        current.push(i);
        enumerate_combinations(n, k, i + 1, current, result);
        current.pop();
    }
}

fn binomial_coefficient(n: usize, k: usize) -> usize {
    if k > n {
        return 0;
    }
    let k = k.min(n - k);
    let mut result: usize = 1;
    for i in 0..k {
        result = result.saturating_mul(n - i) / (i + 1);
    }
    result
}

#[cfg(test)]
mod tests {
    use super::*;

    // ── Math helpers ──

    #[test]
    fn test_normal_cdf_known_values() {
        assert!((normal_cdf(0.0) - 0.5).abs() < 1e-6);
        assert!((normal_cdf(1.96) - 0.975).abs() < 1e-3);
        assert!((normal_cdf(-1.96) - 0.025).abs() < 1e-3);
    }

    #[test]
    fn test_normal_ppf_known_values() {
        assert!(normal_ppf(0.5).abs() < 1e-5);
        assert!((normal_ppf(0.975) - 1.96).abs() < 0.01);
        assert!((normal_ppf(0.025) + 1.96).abs() < 0.01);
    }

    #[test]
    fn test_normal_cdf_ppf_roundtrip() {
        for &p in &[0.01, 0.05, 0.25, 0.5, 0.75, 0.95, 0.99] {
            let x = normal_ppf(p);
            let p_back = normal_cdf(x);
            assert!(
                (p - p_back).abs() < 1e-4,
                "Roundtrip failed for p={}: got {}",
                p,
                p_back
            );
        }
    }

    #[test]
    fn test_sharpe_ratio_positive_trend() {
        // Consistently positive returns → positive Sharpe
        let returns = vec![0.01, 0.02, 0.015, 0.01, 0.025, 0.012, 0.018];
        let sr = sharpe_ratio(&returns);
        assert!(sr > 0.0, "Positive returns should give positive Sharpe: {}", sr);
    }

    #[test]
    fn test_sharpe_ratio_zero_returns() {
        let returns = vec![0.0, 0.0, 0.0, 0.0];
        let sr = sharpe_ratio(&returns);
        assert!((sr - 0.0).abs() < 1e-10);
    }

    #[test]
    fn test_skewness_symmetric() {
        // Roughly symmetric returns → skewness near 0
        let returns = vec![-0.02, -0.01, 0.0, 0.01, 0.02];
        let skew = skewness(&returns);
        assert!(skew.abs() < 0.5, "Symmetric returns should have near-zero skewness: {}", skew);
    }

    #[test]
    fn test_excess_kurtosis_normal() {
        // Normal-ish returns → excess kurtosis near 0
        let returns: Vec<f64> = (0..1000)
            .map(|i| ((i as f64 * 0.01).sin()) * 0.01)
            .collect();
        let kurt = excess_kurtosis(&returns);
        // Sinusoidal returns won't be exactly 0, but should be bounded
        assert!(kurt.abs() < 5.0, "Kurtosis too extreme: {}", kurt);
    }

    // ── DSR ──

    #[test]
    fn test_dsr_single_trial_not_penalized() {
        let result = deflated_sharpe_ratio(2.0, 1, 1.0, 0.0, 3.0, 252);
        // With 1 trial, expected max = 0, so DSR should be favorable
        assert!(result.pvalue < 0.05, "Single trial with SR=2 should be significant");
    }

    #[test]
    fn test_dsr_many_trials_penalized() {
        let result = deflated_sharpe_ratio(1.0, 1000, 1.0, 0.0, 3.0, 252);
        // With 1000 trials, SR=1.0 is likely just the best random draw
        assert!(
            result.pvalue > 0.05,
            "SR=1.0 with 1000 trials should NOT be significant: p={}",
            result.pvalue
        );
    }

    #[test]
    fn test_dsr_higher_sharpe_more_significant() {
        let low = deflated_sharpe_ratio(1.0, 100, 1.0, 0.0, 3.0, 252);
        let high = deflated_sharpe_ratio(3.0, 100, 1.0, 0.0, 3.0, 252);
        assert!(high.pvalue < low.pvalue, "Higher Sharpe should be more significant");
    }

    // ── Permutation test ──

    #[test]
    fn test_permutation_test_random_returns_high_pvalue() {
        // Random returns with no signal → high p-value
        let returns: Vec<f64> = (0..100)
            .map(|i| if i % 2 == 0 { 0.01 } else { -0.01 })
            .collect();
        let result = permutation_test(&returns, 1000, 42);
        assert!(
            result.pvalue > 0.05,
            "Random returns should have high p-value: {}",
            result.pvalue
        );
    }

    #[test]
    fn test_permutation_test_strong_signal_low_pvalue() {
        // Strong upward trend → low p-value
        let returns: Vec<f64> = (0..200).map(|_| 0.02).collect();
        let result = permutation_test(&returns, 1000, 42);
        // All-positive returns: any shuffle preserves the mean, so Sharpe stays the same
        // This tests that the code runs — the p-value for constant returns is ~1.0
        // because every permutation has the same Sharpe
        assert!(result.num_permutations == 1000);
    }

    #[test]
    fn test_permutation_test_reproducible() {
        let returns = vec![0.01, -0.005, 0.02, -0.01, 0.015, -0.002, 0.008];
        let r1 = permutation_test(&returns, 500, 123);
        let r2 = permutation_test(&returns, 500, 123);
        assert!(
            (r1.pvalue - r2.pvalue).abs() < 1e-10,
            "Same seed should give same result"
        );
    }

    // ── PBO ──

    #[test]
    fn test_pbo_random_sharpes_high() {
        // Random per-fold Sharpes with no consistency → PBO should be high
        let sharpes = vec![0.5, -0.3, 1.2, -0.8, 0.1, -0.5, 0.9, -0.2];
        let result = probability_of_backtest_overfitting(&sharpes);
        assert!(result.num_combinations > 0, "Should produce combinations");
        // With random Sharpes, PBO should be around 0.5
        assert!(
            result.pbo > 0.2,
            "Random Sharpes should have elevated PBO: {}",
            result.pbo
        );
    }

    #[test]
    fn test_pbo_consistent_sharpes_low() {
        // All folds have similar positive Sharpe → low PBO
        let sharpes = vec![1.5, 1.6, 1.4, 1.5, 1.7, 1.3, 1.6, 1.4];
        let result = probability_of_backtest_overfitting(&sharpes);
        assert!(
            result.pbo < 0.5,
            "Consistent positive Sharpes should have low PBO: {}",
            result.pbo
        );
    }

    #[test]
    fn test_pbo_too_few_folds() {
        let sharpes = vec![1.0, 2.0]; // Need >= 4 even folds
        let result = probability_of_backtest_overfitting(&sharpes);
        assert_eq!(result.num_combinations, 0);
        assert!((result.pbo - 0.5).abs() < 1e-10); // uninformative prior
    }

    #[test]
    fn test_pbo_odd_folds() {
        let sharpes = vec![1.0, 2.0, 3.0, 4.0, 5.0]; // Odd N
        let result = probability_of_backtest_overfitting(&sharpes);
        assert_eq!(result.num_combinations, 0); // CSCV requires even N
    }

    #[test]
    fn test_binomial_coefficient() {
        assert_eq!(binomial_coefficient(8, 4), 70);
        assert_eq!(binomial_coefficient(6, 3), 20);
        assert_eq!(binomial_coefficient(4, 2), 6);
    }
}

Step 2: Register in mod.rs

Add to ml/src/validation/mod.rs:

pub mod statistical;

pub use statistical::{
    deflated_sharpe_ratio, excess_kurtosis, normal_cdf, normal_ppf, permutation_test,
    probability_of_backtest_overfitting, sharpe_ratio, skewness, DsrResult, PboResult,
    PermutationResult,
};

Step 3: Run tests

Run: SQLX_OFFLINE=true cargo test --manifest-path /home/jgrusewski/Work/foxhunt/ml/Cargo.toml --lib validation::statistical -- --nocapture 2>&1 | tail -15 Expected: 14 tests pass

Step 4: Verify full compilation

Run: SQLX_OFFLINE=true cargo check --manifest-path /home/jgrusewski/Work/foxhunt/ml/Cargo.toml

Step 5: Commit

git add ml/src/validation/statistical.rs ml/src/validation/mod.rs
git commit -m "feat(validation): add DSR, PBO, permutation test, and math helpers"

Task 5: Per-regime analysis

Implements per-regime Sharpe breakdown using the existing RegimeType enum from dqn::regime_conditional.

Files:

Create: ml/src/validation/regime_analysis.rs
Modify: ml/src/validation/mod.rs

Step 1: Write regime_analysis.rs

//! Per-regime performance analysis.
//!
//! Labels each bar with a market regime (Trending, Ranging, Volatile) using
//! the existing regime detection infrastructure, then computes per-regime
//! Sharpe ratio, win rate, and average return.

use std::collections::HashMap;

use serde::{Deserialize, Serialize};

use crate::dqn::RegimeType;

use super::statistical::sharpe_ratio;

/// Performance metrics for a single market regime.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct RegimeMetrics {
    /// Which regime these metrics describe.
    pub regime: RegimeType,
    /// Sharpe ratio for bars in this regime.
    pub sharpe: f64,
    /// Number of bars classified into this regime.
    pub num_bars: usize,
    /// Fraction of bars with positive returns.
    pub win_rate: f64,
    /// Mean return across all bars in this regime.
    pub avg_return: f64,
}

/// Compute per-regime performance breakdown.
///
/// # Arguments
/// * `daily_returns` - Daily returns (length N)
/// * `features` - Feature vectors for each bar (length N, used for regime classification)
///
/// Regime classification uses `RegimeType::classify_from_features()` which looks at
/// ADX and entropy values within the feature vector.
pub fn per_regime_breakdown(
    daily_returns: &[f64],
    features: &[Vec<f32>],
) -> HashMap<RegimeType, RegimeMetrics> {
    let mut regime_returns: HashMap<RegimeType, Vec<f64>> = HashMap::new();

    let n = daily_returns.len().min(features.len());

    for i in 0..n {
        let regime = RegimeType::classify_from_features(&features[i]);
        regime_returns
            .entry(regime)
            .or_default()
            .push(daily_returns[i]);
    }

    let mut result = HashMap::new();
    for (regime, returns) in &regime_returns {
        let num_bars = returns.len();
        let avg_return = if num_bars > 0 {
            returns.iter().sum::<f64>() / num_bars as f64
        } else {
            0.0
        };
        let win_rate = if num_bars > 0 {
            returns.iter().filter(|&&r| r > 0.0).count() as f64 / num_bars as f64
        } else {
            0.0
        };

        result.insert(
            *regime,
            RegimeMetrics {
                regime: *regime,
                sharpe: sharpe_ratio(returns),
                num_bars,
                win_rate,
                avg_return,
            },
        );
    }

    result
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_per_regime_breakdown_groups_correctly() {
        // With insufficient features (< 211), classify_from_features returns Ranging (safe default)
        let returns = vec![0.01, -0.005, 0.02, 0.015, -0.01];
        let features: Vec<Vec<f32>> = (0..5).map(|i| vec![i as f32; 10]).collect();

        let breakdown = per_regime_breakdown(&returns, &features);

        // All should be classified as Ranging (features too short for real classification)
        assert!(breakdown.contains_key(&RegimeType::Ranging));
        let ranging = &breakdown[&RegimeType::Ranging];
        assert_eq!(ranging.num_bars, 5);
    }

    #[test]
    fn test_per_regime_metrics_computation() {
        let returns = vec![0.01, 0.02, 0.03];
        let features: Vec<Vec<f32>> = (0..3).map(|_| vec![0.0; 10]).collect();

        let breakdown = per_regime_breakdown(&returns, &features);
        let metrics = breakdown.values().next().expect("should have one regime");

        assert_eq!(metrics.num_bars, 3);
        assert!((metrics.win_rate - 1.0).abs() < 1e-10); // all positive
        assert!((metrics.avg_return - 0.02).abs() < 1e-10);
    }

    #[test]
    fn test_empty_returns() {
        let breakdown = per_regime_breakdown(&[], &[]);
        assert!(breakdown.is_empty());
    }
}

Step 2: Register in mod.rs

Add to ml/src/validation/mod.rs:

pub mod regime_analysis;

pub use regime_analysis::{per_regime_breakdown, RegimeMetrics};

Step 3: Run tests

Run: SQLX_OFFLINE=true cargo test --manifest-path /home/jgrusewski/Work/foxhunt/ml/Cargo.toml --lib validation::regime_analysis -- --nocapture 2>&1 | tail -10 Expected: 3 tests pass

Step 4: Verify full compilation

Run: SQLX_OFFLINE=true cargo check --manifest-path /home/jgrusewski/Work/foxhunt/ml/Cargo.toml

Step 5: Commit

git add ml/src/validation/regime_analysis.rs ml/src/validation/mod.rs
git commit -m "feat(validation): add per-regime performance analysis"

Task 6: Validation harness orchestrator

Ties everything together: runs a strategy through walk-forward folds, computes DSR, PBO, permutation test, and regime breakdown, then produces a ValidationReport.

Files:

Create: ml/src/validation/harness.rs
Modify: ml/src/validation/mod.rs

Step 1: Write harness.rs

//! Validation harness orchestrator.
//!
//! Runs a `ValidatableStrategy` through the complete validation pipeline:
//! 1. Walk-forward cross-validation with embargo
//! 2. Per-fold Sharpe computation
//! 3. Deflated Sharpe Ratio (adjusts for multiple testing)
//! 4. Probability of Backtest Overfitting (CSCV)
//! 5. Monte Carlo permutation test
//! 6. Per-regime performance breakdown
//! 7. Overall verdict (Pass / Marginal / Fail)

use std::collections::HashMap;

use serde::{Deserialize, Serialize};

use crate::dqn::RegimeType;
use crate::MLError;

use super::regime_analysis::{per_regime_breakdown, RegimeMetrics};
use super::statistical::{
    deflated_sharpe_ratio, excess_kurtosis, permutation_test, probability_of_backtest_overfitting,
    sharpe_ratio, skewness, DsrResult, PboResult, PermutationResult,
};
use super::types::{TimeSeriesData, ValidatableStrategy};
use super::walk_forward::{walk_forward_split, WalkForwardConfig};

/// Overall validation verdict.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
pub enum ValidationVerdict {
    /// All tests pass: DSR p < 0.05, PBO < 0.25, MC p < 0.05
    Pass,
    /// At least one test passes at relaxed threshold
    Marginal,
    /// No statistical evidence of a real signal
    Fail,
}

impl std::fmt::Display for ValidationVerdict {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        match self {
            ValidationVerdict::Pass => write!(f, "PASS"),
            ValidationVerdict::Marginal => write!(f, "MARGINAL"),
            ValidationVerdict::Fail => write!(f, "FAIL"),
        }
    }
}

/// Complete validation report from the harness.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ValidationReport {
    pub strategy_name: String,
    // Walk-forward results
    pub per_fold_sharpes: Vec<f64>,
    pub aggregate_sharpe: f64,
    pub num_folds: usize,
    // DSR
    pub dsr: DsrResult,
    // PBO
    pub pbo: PboResult,
    // Monte Carlo
    pub permutation: PermutationResult,
    // Regime breakdown
    pub per_regime_metrics: HashMap<RegimeType, RegimeMetrics>,
    // Overall
    pub verdict: ValidationVerdict,
}

/// Configuration for the validation harness.
#[derive(Debug, Clone)]
pub struct ValidationHarnessConfig {
    /// Walk-forward configuration.
    pub wf_config: WalkForwardConfig,
    /// Number of Monte Carlo permutations.
    pub num_permutations: usize,
    /// Number of hyperopt trials tested (for DSR).
    pub num_trials: usize,
    /// RNG seed for reproducibility.
    pub seed: u64,
}

impl Default for ValidationHarnessConfig {
    fn default() -> Self {
        Self {
            wf_config: WalkForwardConfig::default(),
            num_permutations: 10_000,
            num_trials: 1,
            seed: 42,
        }
    }
}

/// The validation harness: runs a strategy through the complete pipeline.
pub struct ValidationHarness {
    config: ValidationHarnessConfig,
}

impl ValidationHarness {
    pub fn new(config: ValidationHarnessConfig) -> Self {
        Self { config }
    }

    /// Run full validation on a strategy.
    ///
    /// # Arguments
    /// * `strategy` - A model implementing `ValidatableStrategy`
    /// * `data` - The full time-series dataset to validate against
    ///
    /// # Returns
    /// A `ValidationReport` with all statistical test results and a verdict.
    pub fn validate<S: ValidatableStrategy>(
        &self,
        strategy: &mut S,
        data: &TimeSeriesData,
    ) -> Result<ValidationReport, MLError> {
        // 1. Walk-forward split
        let folds = walk_forward_split(data.len(), &self.config.wf_config);

        if folds.is_empty() {
            return Err(MLError::ValidationError {
                message: format!(
                    "Not enough data for walk-forward validation: {} bars, need at least {}",
                    data.len(),
                    self.config.wf_config.train_bars
                        + self.config.wf_config.embargo_bars
                        + self.config.wf_config.test_bars
                ),
            });
        }

        // 2. Run strategy through each fold
        let mut per_fold_sharpes = Vec::with_capacity(folds.len());
        let mut all_test_returns = Vec::new();
        let mut all_test_features = Vec::new();

        for fold in &folds {
            // Reset strategy for fresh training
            strategy.reset()?;

            // Train on training data
            let train_data = data.slice(fold.train_range.start, fold.train_range.end)?;
            strategy.train(&train_data)?;

            // Evaluate on test data
            let test_data = data.slice(fold.test_range.start, fold.test_range.end)?;
            let fold_returns = strategy.evaluate(&test_data)?;

            // Collect per-fold Sharpe
            let fold_sharpe = sharpe_ratio(&fold_returns);
            per_fold_sharpes.push(fold_sharpe);

            // Accumulate returns and features for aggregate stats
            all_test_returns.extend_from_slice(&fold_returns);
            all_test_features.extend(test_data.features.iter().take(fold_returns.len()).cloned());
        }

        // 3. Aggregate Sharpe
        let aggregate_sharpe = sharpe_ratio(&all_test_returns);

        // 4. DSR
        let sharpe_variance = if per_fold_sharpes.len() > 1 {
            let mean = per_fold_sharpes.iter().sum::<f64>() / per_fold_sharpes.len() as f64;
            per_fold_sharpes
                .iter()
                .map(|&s| (s - mean).powi(2))
                .sum::<f64>()
                / (per_fold_sharpes.len() - 1) as f64
        } else {
            1.0
        };
        let skew = skewness(&all_test_returns);
        let kurt = excess_kurtosis(&all_test_returns);

        let dsr = deflated_sharpe_ratio(
            aggregate_sharpe,
            self.config.num_trials,
            sharpe_variance,
            skew,
            kurt,
            all_test_returns.len(),
        );

        // 5. PBO (requires even number of folds)
        let pbo_sharpes = if per_fold_sharpes.len() % 2 != 0 && per_fold_sharpes.len() > 1 {
            // Drop last fold to make even
            per_fold_sharpes[..per_fold_sharpes.len() - 1].to_vec()
        } else {
            per_fold_sharpes.clone()
        };
        let pbo = probability_of_backtest_overfitting(&pbo_sharpes);

        // 6. Monte Carlo permutation test
        let permutation = permutation_test(
            &all_test_returns,
            self.config.num_permutations,
            self.config.seed,
        );

        // 7. Per-regime breakdown
        let per_regime_metrics = per_regime_breakdown(&all_test_returns, &all_test_features);

        // 8. Verdict
        let verdict = determine_verdict(&dsr, &pbo, &permutation);

        Ok(ValidationReport {
            strategy_name: strategy.name().to_string(),
            per_fold_sharpes,
            aggregate_sharpe,
            num_folds: folds.len(),
            dsr,
            pbo,
            permutation,
            per_regime_metrics,
            verdict,
        })
    }
}

fn determine_verdict(
    dsr: &DsrResult,
    pbo: &PboResult,
    permutation: &PermutationResult,
) -> ValidationVerdict {
    let dsr_pass = dsr.pvalue < 0.05;
    let pbo_pass = pbo.pbo < 0.25;
    let mc_pass = permutation.pvalue < 0.05;

    if dsr_pass && pbo_pass && mc_pass {
        ValidationVerdict::Pass
    } else if dsr.pvalue < 0.10 || pbo.pbo < 0.50 || permutation.pvalue < 0.10 {
        ValidationVerdict::Marginal
    } else {
        ValidationVerdict::Fail
    }
}

#[cfg(test)]
mod tests {
    use super::*;
    use chrono::{Duration, Utc};

    /// A mock strategy that always returns fixed returns (for testing the harness).
    struct MockStrategy {
        name: String,
        returns_per_bar: f64,
    }

    impl MockStrategy {
        fn new(name: &str, returns_per_bar: f64) -> Self {
            Self {
                name: name.to_string(),
                returns_per_bar,
            }
        }
    }

    impl ValidatableStrategy for MockStrategy {
        fn train(&mut self, _data: &TimeSeriesData) -> Result<(), MLError> {
            Ok(())
        }

        fn evaluate(&self, data: &TimeSeriesData) -> Result<Vec<f64>, MLError> {
            Ok(vec![self.returns_per_bar; data.len()])
        }

        fn name(&self) -> &str {
            &self.name
        }

        fn reset(&mut self) -> Result<(), MLError> {
            Ok(())
        }
    }

    fn make_data(n: usize) -> TimeSeriesData {
        let start = Utc::now();
        let timestamps: Vec<_> = (0..n)
            .map(|i| start + Duration::hours(i as i64))
            .collect();
        let features: Vec<Vec<f32>> = (0..n).map(|i| vec![i as f32; 10]).collect();
        let prices: Vec<f64> = (0..n).map(|i| 100.0 + i as f64 * 0.1).collect();
        TimeSeriesData::new(timestamps, features, prices).expect("valid data")
    }

    #[test]
    fn test_harness_runs_end_to_end() {
        let data = make_data(500);
        let config = ValidationHarnessConfig {
            wf_config: WalkForwardConfig {
                train_bars: 100,
                test_bars: 50,
                embargo_bars: 5,
                step_bars: 50,
                min_train_samples: 50,
            },
            num_permutations: 100, // small for test speed
            num_trials: 1,
            seed: 42,
        };

        let harness = ValidationHarness::new(config);
        let mut strategy = MockStrategy::new("test-mock", 0.01);

        let report = harness.validate(&mut strategy, &data);
        assert!(report.is_ok(), "Harness should complete: {:?}", report.err());

        let report = report.expect("valid report");
        assert_eq!(report.strategy_name, "test-mock");
        assert!(!report.per_fold_sharpes.is_empty());
        assert!(report.num_folds > 0);
    }

    #[test]
    fn test_harness_insufficient_data_errors() {
        let data = make_data(20); // way too short
        let config = ValidationHarnessConfig {
            wf_config: WalkForwardConfig {
                train_bars: 100,
                test_bars: 50,
                embargo_bars: 5,
                step_bars: 50,
                min_train_samples: 50,
            },
            ..Default::default()
        };

        let harness = ValidationHarness::new(config);
        let mut strategy = MockStrategy::new("too-short", 0.01);

        let report = harness.validate(&mut strategy, &data);
        assert!(report.is_err());
    }

    #[test]
    fn test_verdict_all_pass() {
        let dsr = DsrResult {
            observed_sharpe: 3.0,
            expected_max_sharpe: 1.0,
            sharpe_std_error: 0.5,
            deflated_sharpe: 4.0,
            pvalue: 0.001,
        };
        let pbo = PboResult {
            pbo: 0.1,
            num_combinations: 70,
            logit_distribution: vec![],
        };
        let perm = PermutationResult {
            observed_sharpe: 3.0,
            pvalue: 0.001,
            num_permutations: 1000,
            null_mean: 0.0,
            null_std: 0.5,
        };
        assert_eq!(determine_verdict(&dsr, &pbo, &perm), ValidationVerdict::Pass);
    }

    #[test]
    fn test_verdict_fail() {
        let dsr = DsrResult {
            observed_sharpe: 0.5,
            expected_max_sharpe: 2.0,
            sharpe_std_error: 0.5,
            deflated_sharpe: -3.0,
            pvalue: 0.99,
        };
        let pbo = PboResult {
            pbo: 0.8,
            num_combinations: 70,
            logit_distribution: vec![],
        };
        let perm = PermutationResult {
            observed_sharpe: 0.5,
            pvalue: 0.6,
            num_permutations: 1000,
            null_mean: 0.0,
            null_std: 0.5,
        };
        assert_eq!(determine_verdict(&dsr, &pbo, &perm), ValidationVerdict::Fail);
    }
}

Step 2: Register in mod.rs

Add to ml/src/validation/mod.rs:

pub mod harness;

pub use harness::{ValidationHarness, ValidationHarnessConfig, ValidationReport, ValidationVerdict};

Step 3: Run tests

Run: SQLX_OFFLINE=true cargo test --manifest-path /home/jgrusewski/Work/foxhunt/ml/Cargo.toml --lib validation::harness -- --nocapture 2>&1 | tail -10 Expected: 4 tests pass

Step 4: Verify full test suite

Run: SQLX_OFFLINE=true cargo test --manifest-path /home/jgrusewski/Work/foxhunt/ml/Cargo.toml --lib validation -- --nocapture 2>&1 | tail -10 Expected: All validation tests pass (5 + 7 + 14 + 3 + 4 = 33 tests)

Step 5: Commit

git add ml/src/validation/harness.rs ml/src/validation/mod.rs
git commit -m "feat(validation): add validation harness orchestrator with verdict system"

Task 7: DQN strategy adapter

Implement ValidatableStrategy for DQN so it can be validated through the harness.

Files:

Create: ml/src/validation/adapters.rs
Modify: ml/src/validation/mod.rs

Step 1: Write adapters.rs

This is a thin wrapper that translates between DQN's native API and the ValidatableStrategy trait.

//! Strategy adapters for the validation harness.
//!
//! Each adapter wraps a model's native API to implement `ValidatableStrategy`,
//! enabling walk-forward validation, DSR, PBO, and permutation tests.

use crate::dqn::{DQNConfig, Experience, DQN};
use crate::MLError;

use super::types::{TimeSeriesData, ValidatableStrategy};

/// Adapter that makes a DQN model validatable through the harness.
pub struct DqnStrategy {
    config: DQNConfig,
    dqn: DQN,
}

impl DqnStrategy {
    /// Create a new DQN strategy adapter.
    pub fn new(config: DQNConfig) -> Result<Self, MLError> {
        let dqn = DQN::new(config.clone())?;
        Ok(Self { config, dqn })
    }
}

impl ValidatableStrategy for DqnStrategy {
    fn train(&mut self, data: &TimeSeriesData) -> Result<(), MLError> {
        // Feed experiences into the replay buffer
        let num_bars = data.len();
        if num_bars < 2 {
            return Ok(());
        }

        for i in 0..num_bars.saturating_sub(1) {
            let state = data.features.get(i).cloned().unwrap_or_default();
            let next_state = data.features.get(i + 1).cloned().unwrap_or_default();

            // Select action using current policy
            let action = self.dqn.select_action(&state)?;

            // Reward = next bar's return (simple reward shaping)
            let reward = data.returns.get(i).copied().unwrap_or(0.0) as f32;
            let done = i == num_bars - 2;

            let exp = Experience::new(
                state,
                action.to_index() as u8,
                reward,
                next_state,
                done,
            );
            self.dqn.store_experience(exp)?;
        }

        // Train for multiple steps if enough data in replay buffer
        let train_steps = (num_bars / self.config.batch_size).min(100);
        for _ in 0..train_steps {
            match self.dqn.train_step(None) {
                Ok(_) => {}
                Err(MLError::TrainingError(msg)) if msg.contains("Not enough") => break,
                Err(e) => return Err(e),
            }
        }

        Ok(())
    }

    fn evaluate(&self, data: &TimeSeriesData) -> Result<Vec<f64>, MLError> {
        let mut returns = Vec::with_capacity(data.len());

        for i in 0..data.len() {
            let state = data.features.get(i).cloned().unwrap_or_default();
            let action = self.dqn.select_action(&state)?;

            // Simple PnL model: Buy=+return, Hold=0, Sell=-return
            let bar_return = data.returns.get(i).copied().unwrap_or(0.0);
            let pnl = match action.to_index() {
                0 => bar_return,      // Buy → long exposure
                2 => -bar_return,     // Sell → short exposure
                _ => 0.0,             // Hold → no exposure
            };
            returns.push(pnl);
        }

        Ok(returns)
    }

    fn name(&self) -> &str {
        "DQN"
    }

    fn reset(&mut self) -> Result<(), MLError> {
        self.dqn = DQN::new(self.config.clone())?;
        Ok(())
    }
}

#[cfg(test)]
mod tests {
    use super::*;
    use chrono::{Duration, Utc};

    fn make_test_config() -> DQNConfig {
        let mut config = DQNConfig::default();
        config.state_dim = 10;
        config.num_actions = 3;
        config.hidden_dims = vec![16, 8];
        config.batch_size = 4;
        config.min_replay_size = 4;
        config.warmup_steps = 0;
        config.use_noisy_nets = false;
        config.use_iqn = false;
        config.use_distributional = false;
        config.use_dueling = false;
        config.use_per = false;
        config.epsilon_start = 0.5;
        config
    }

    fn make_test_data(n: usize, feature_dim: usize) -> TimeSeriesData {
        let start = Utc::now();
        let timestamps: Vec<_> = (0..n)
            .map(|i| start + Duration::hours(i as i64))
            .collect();
        let features: Vec<Vec<f32>> = (0..n)
            .map(|i| vec![0.1 * i as f32; feature_dim])
            .collect();
        let prices: Vec<f64> = (0..n).map(|i| 100.0 + i as f64 * 0.5).collect();
        TimeSeriesData::new(timestamps, features, prices).expect("valid data")
    }

    #[test]
    fn test_dqn_strategy_creation() {
        let config = make_test_config();
        let strategy = DqnStrategy::new(config);
        assert!(strategy.is_ok());
    }

    #[test]
    fn test_dqn_strategy_train_and_evaluate() {
        let config = make_test_config();
        let mut strategy = DqnStrategy::new(config).expect("valid strategy");

        let data = make_test_data(30, 10);
        let train_result = strategy.train(&data);
        assert!(train_result.is_ok(), "Training failed: {:?}", train_result.err());

        let eval_result = strategy.evaluate(&data);
        assert!(eval_result.is_ok(), "Evaluation failed: {:?}", eval_result.err());

        let returns = eval_result.expect("valid returns");
        assert_eq!(returns.len(), 30);
    }

    #[test]
    fn test_dqn_strategy_reset() {
        let config = make_test_config();
        let mut strategy = DqnStrategy::new(config).expect("valid strategy");

        let data = make_test_data(20, 10);
        strategy.train(&data).expect("training ok");
        strategy.reset().expect("reset ok");
        // After reset, should be able to train again cleanly
        strategy.train(&data).expect("retrain ok");
    }

    #[test]
    fn test_dqn_strategy_name() {
        let config = make_test_config();
        let strategy = DqnStrategy::new(config).expect("valid strategy");
        assert_eq!(strategy.name(), "DQN");
    }
}

Step 2: Register in mod.rs

Add to ml/src/validation/mod.rs:

pub mod adapters;

pub use adapters::DqnStrategy;

Step 3: Run tests

Run: SQLX_OFFLINE=true cargo test --manifest-path /home/jgrusewski/Work/foxhunt/ml/Cargo.toml --lib validation::adapters -- --nocapture 2>&1 | tail -10 Expected: 4 tests pass

Step 4: Verify full compilation

Run: SQLX_OFFLINE=true cargo check --manifest-path /home/jgrusewski/Work/foxhunt/ml/Cargo.toml

Step 5: Commit

git add ml/src/validation/adapters.rs ml/src/validation/mod.rs
git commit -m "feat(validation): add DQN strategy adapter for ValidatableStrategy trait"

Task 8: Integration test — DQN through full validation pipeline

End-to-end test that runs a DQN model through the entire validation harness.

Files:

Create: ml/tests/validation_harness_integration_test.rs

Step 1: Write the integration test

//! Integration test: Full validation pipeline with DQN model.
//!
//! Runs DQN through walk-forward validation, DSR, PBO, permutation test,
//! and per-regime analysis to verify the complete pipeline works end-to-end.

use chrono::{Duration, Utc};
use ml::dqn::DQNConfig;
use ml::validation::{
    DqnStrategy, TimeSeriesData, ValidationHarness, ValidationHarnessConfig,
    WalkForwardConfig, ValidatableStrategy,
};

fn make_synthetic_data(num_bars: usize, feature_dim: usize) -> TimeSeriesData {
    let start = Utc::now();
    let timestamps: Vec<_> = (0..num_bars)
        .map(|i| start + Duration::hours(i as i64))
        .collect();
    let features: Vec<Vec<f32>> = (0..num_bars)
        .map(|i| {
            let base = (i as f32) * 0.01;
            vec![base; feature_dim]
        })
        .collect();
    let prices: Vec<f64> = (0..num_bars)
        .map(|i| 100.0 + (i as f64 * 0.1).sin() * 5.0 + i as f64 * 0.01)
        .collect();
    TimeSeriesData::new(timestamps, features, prices).expect("valid synthetic data")
}

#[test]
fn test_full_validation_pipeline_with_dqn() {
    let mut config = DQNConfig::default();
    config.state_dim = 8;
    config.num_actions = 3;
    config.hidden_dims = vec![16, 8];
    config.batch_size = 4;
    config.min_replay_size = 4;
    config.warmup_steps = 0;
    config.use_noisy_nets = false;
    config.use_iqn = false;
    config.use_distributional = false;
    config.use_dueling = false;
    config.use_per = false;
    config.epsilon_start = 0.3;

    let mut strategy = DqnStrategy::new(config).expect("DQN creation");
    let data = make_synthetic_data(300, 8);

    let harness_config = ValidationHarnessConfig {
        wf_config: WalkForwardConfig {
            train_bars: 50,
            test_bars: 30,
            embargo_bars: 5,
            step_bars: 30,
            min_train_samples: 20,
        },
        num_permutations: 100, // small for test speed
        num_trials: 1,
        seed: 42,
    };

    let harness = ValidationHarness::new(harness_config);
    let report = harness.validate(&mut strategy, &data);

    assert!(report.is_ok(), "Validation pipeline failed: {:?}", report.err());

    let report = report.expect("valid report");
    assert_eq!(report.strategy_name, "DQN");
    assert!(report.num_folds >= 2, "Should have multiple folds: {}", report.num_folds);
    assert!(report.aggregate_sharpe.is_finite(), "Sharpe must be finite");
    assert!(report.dsr.pvalue >= 0.0 && report.dsr.pvalue <= 1.0, "DSR p-value out of range");
    assert!(report.pbo.pbo >= 0.0 && report.pbo.pbo <= 1.0, "PBO out of range");
    assert!(report.permutation.pvalue >= 0.0 && report.permutation.pvalue <= 1.0, "MC p-value out of range");
    // Regime breakdown should have at least one regime
    assert!(!report.per_regime_metrics.is_empty(), "Should have at least one regime");

    println!("=== Validation Report ===");
    println!("Strategy: {}", report.strategy_name);
    println!("Folds: {}", report.num_folds);
    println!("Aggregate Sharpe: {:.4}", report.aggregate_sharpe);
    println!("DSR p-value: {:.4}", report.dsr.pvalue);
    println!("PBO: {:.4}", report.pbo.pbo);
    println!("MC p-value: {:.4}", report.permutation.pvalue);
    println!("Verdict: {}", report.verdict);
    for (regime, metrics) in &report.per_regime_metrics {
        println!("  {:?}: Sharpe={:.3}, WinRate={:.1}%, Bars={}",
            regime, metrics.sharpe, metrics.win_rate * 100.0, metrics.num_bars);
    }
}

#[test]
fn test_walk_forward_split_standalone() {
    use ml::validation::{walk_forward_split, WalkForwardConfig};

    let config = WalkForwardConfig {
        train_bars: 50,
        test_bars: 20,
        embargo_bars: 5,
        step_bars: 20,
        min_train_samples: 20,
    };
    let folds = walk_forward_split(200, &config);
    assert!(!folds.is_empty());

    // Verify no test range exceeds data bounds
    for fold in &folds {
        assert!(fold.test_range.end <= 200);
        assert!(fold.train_range.end <= fold.embargo_range.start);
        assert!(fold.embargo_range.end <= fold.test_range.start);
    }
}

#[test]
fn test_dsr_and_pbo_standalone() {
    use ml::validation::{deflated_sharpe_ratio, probability_of_backtest_overfitting};

    // DSR: high Sharpe with few trials should be significant
    let dsr = deflated_sharpe_ratio(2.5, 5, 0.5, 0.1, 3.5, 500);
    assert!(dsr.pvalue < 0.1, "Strong Sharpe with few trials should be significant");

    // PBO: consistent fold Sharpes should show low overfitting probability
    let sharpes = vec![1.0, 1.2, 0.8, 1.1, 0.9, 1.3, 1.0, 0.95];
    let pbo = probability_of_backtest_overfitting(&sharpes);
    assert!(pbo.num_combinations > 0);
}

Step 2: Run the integration test

Run: SQLX_OFFLINE=true cargo test --manifest-path /home/jgrusewski/Work/foxhunt/ml/Cargo.toml --test validation_harness_integration_test -- --nocapture 2>&1 | tail -20 Expected: 3 tests pass

Step 3: Verify all existing tests still pass

Run: SQLX_OFFLINE=true cargo test --manifest-path /home/jgrusewski/Work/foxhunt/ml/Cargo.toml --lib 2>&1 | tail -5 Expected: 1823+ tests pass (same as before plus ~37 new validation tests)

Step 4: Commit

git add ml/tests/validation_harness_integration_test.rs
git commit -m "test(validation): add end-to-end integration test for full validation pipeline"

Task 9: Final verification

Full test suite run and compilation check to ensure nothing is broken.

Step 1: Full workspace compilation

Run: SQLX_OFFLINE=true cargo check --manifest-path /home/jgrusewski/Work/foxhunt/ml/Cargo.toml Expected: Compiles clean

Step 2: All lib tests

Run: SQLX_OFFLINE=true cargo test --manifest-path /home/jgrusewski/Work/foxhunt/ml/Cargo.toml --lib 2>&1 | tail -5 Expected: All tests pass (1823 existing + ~37 new validation = ~1860)

Step 3: All integration tests

Run: SQLX_OFFLINE=true cargo test --manifest-path /home/jgrusewski/Work/foxhunt/ml/Cargo.toml --tests 2>&1 | tail -10 Expected: All integration tests pass

Step 4: Summary of new files

ml/src/validation/
├── mod.rs              # Re-exports + module declarations
├── financial.rs        # Existing financial validation (moved from validation.rs)
├── types.rs            # TimeSeriesData + ValidatableStrategy trait
├── walk_forward.rs     # Walk-forward splitter with embargo
├── statistical.rs      # DSR + PBO + Monte Carlo + math helpers
├── regime_analysis.rs  # Per-regime Sharpe breakdown
├── harness.rs          # Orchestrator + ValidationReport + Verdict
└── adapters.rs         # DqnStrategy (ValidatableStrategy for DQN)

ml/tests/
└── validation_harness_integration_test.rs  # End-to-end pipeline test

Step 5: Verify git status is clean

Run: git status Expected: All changes committed, working tree clean.

72 KiB Raw Blame History

Statistical Validation Stack Implementation Plan

Task 1: Convert validation.rs to directory module

Task 2: TimeSeriesData struct + ValidatableStrategy trait

Task 3: Walk-forward splitter

Task 4: Statistical math helpers (normal CDF, Sharpe, skewness, kurtosis)

Task 5: Per-regime analysis

Task 6: Validation harness orchestrator

Task 7: DQN strategy adapter

Task 8: Integration test — DQN through full validation pipeline

Task 9: Final verification

72 KiB

Raw Blame History