d313486dc2
Extract 9 new sub-crates from the ml monolith to enable parallel compilation across the workspace: New crates (this commit): - ml-features (282 tests): feature engineering, 21 modules - ml-labeling (45 tests): triple barrier, meta-labeling, fractional diff - ml-ensemble (116 tests): ensemble coordination, voting, confidence - ml-hyperopt (47 tests): core PSO/TPE optimizer, parameter space - ml-checkpoint (41 tests): checkpoint persistence, compression, signing - ml-regime (68 tests): CUSUM, Bayesian changepoint, regime classification - ml-data-validation (67 tests): FDR correction, CPCV, data quality - ml-risk (33 tests): neural VaR, Kelly criterion, circuit breakers - ml-validation (43 tests): statistical validation, walk-forward, DSR Extended existing crates: - ml-dqn: added evaluation/ (backtesting engine, metrics, reports) and checkpoint implementation - ml-supervised: added checkpoint implementations - ml-core: added shared types needed by new sub-crates Pattern: each module in ml/ becomes a thin facade (pub use subcrate::*) with bridge modules staying in ml for cross-model adapter code. Dead code deleted (~7K lines): - 13 undeclared files in microstructure/ (never compiled) - 7 undeclared files + tests/ in risk/ (never compiled) - parquet_io, cache_service, cache_storage, minio_integration (unused) - extraction_wave_d_impl.rs (bare fn outside impl block) All 2,746 sub-crate tests + 951 ml tests pass. Full workspace builds clean. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
71 lines
2.5 KiB
Rust
71 lines
2.5 KiB
Rust
#![deny(clippy::unwrap_used, clippy::expect_used)]
|
|
#![cfg_attr(test, allow(clippy::unwrap_used, clippy::expect_used))]
|
|
//! # ML Data Validation
|
|
//!
|
|
//! Data quality validation, cross-validation, and statistical correction
|
|
//! for Foxhunt ML training data.
|
|
//!
|
|
//! ## Modules
|
|
//!
|
|
//! - [`corrector`] -- Automatic correction of data quality issues
|
|
//! - [`cpcv`] -- Combinatorial Purged Cross-Validation (Lopez de Prado)
|
|
//! - [`fdr`] -- False Discovery Rate correction for multiple hypothesis testing
|
|
//! - [`rules`] -- Composable validation rules for OHLCV data
|
|
//! - [`validator`] -- Orchestrator that runs rules and generates reports
|
|
|
|
pub mod corrector;
|
|
pub mod cpcv;
|
|
pub mod fdr;
|
|
pub mod rules;
|
|
pub mod validator;
|
|
|
|
// Re-export core types used by this crate
|
|
pub use ml_core::types::OHLCVBar;
|
|
pub use ml_core::MLError;
|
|
|
|
/// Technical indicators (10 essential ones).
|
|
///
|
|
/// All indicators are calculated with standard parameters for 1-minute OHLCV data:
|
|
/// - RSI(14): Relative Strength Index
|
|
/// - MACD(12,26,9): Moving Average Convergence Divergence
|
|
/// - Bollinger Bands(20, 2.0): Price envelope
|
|
/// - ATR(14): Average True Range
|
|
/// - EMA(12, 26): Exponential moving averages
|
|
/// - Volume MA(20): Volume moving average
|
|
///
|
|
/// This type mirrors `ml::data_loader::Indicators` so that the validation crate
|
|
/// can operate independently of the full `ml` crate. Conversion between the two
|
|
/// is trivial (both are plain `pub` field structs with identical layout).
|
|
#[derive(Debug, Clone)]
|
|
pub struct Indicators {
|
|
/// RSI(14) - values 0-100
|
|
pub rsi: Vec<f32>,
|
|
/// MACD line (12,26)
|
|
pub macd: Vec<f32>,
|
|
/// MACD signal line (9)
|
|
pub macd_signal: Vec<f32>,
|
|
/// Bollinger upper band (20, 2.0)
|
|
pub bb_upper: Vec<f32>,
|
|
/// Bollinger middle band (SMA 20)
|
|
pub bb_middle: Vec<f32>,
|
|
/// Bollinger lower band (20, 2.0)
|
|
pub bb_lower: Vec<f32>,
|
|
/// ATR(14) - volatility measure
|
|
pub atr: Vec<f32>,
|
|
/// EMA(12) - fast exponential moving average
|
|
pub ema_fast: Vec<f32>,
|
|
/// EMA(26) - slow exponential moving average
|
|
pub ema_slow: Vec<f32>,
|
|
/// Volume MA(20) - volume moving average
|
|
pub volume_ma: Vec<f32>,
|
|
}
|
|
|
|
// Re-export main types
|
|
pub use corrector::DataCorrector;
|
|
pub use cpcv::{CPCVConfig, CPCVResult, CPCVSplit, CPCVValidator};
|
|
pub use fdr::{FDRConfig, FDRCorrector, FDRMethod, FDRResult};
|
|
pub use rules::{
|
|
CompletenessRule, ContinuityRule, IndicatorRule, IntegrityRule, TimestampRule, ValidationRule,
|
|
};
|
|
pub use validator::{DataValidator, ValidationReport, ValidationResult};
|