refactor(ml): split monolith into 9 sub-crates + delete dead code

Extract 9 new sub-crates from the ml monolith to enable parallel
compilation across the workspace:

New crates (this commit):
- ml-features (282 tests): feature engineering, 21 modules
- ml-labeling (45 tests): triple barrier, meta-labeling, fractional diff
- ml-ensemble (116 tests): ensemble coordination, voting, confidence
- ml-hyperopt (47 tests): core PSO/TPE optimizer, parameter space
- ml-checkpoint (41 tests): checkpoint persistence, compression, signing
- ml-regime (68 tests): CUSUM, Bayesian changepoint, regime classification
- ml-data-validation (67 tests): FDR correction, CPCV, data quality
- ml-risk (33 tests): neural VaR, Kelly criterion, circuit breakers
- ml-validation (43 tests): statistical validation, walk-forward, DSR

Extended existing crates:
- ml-dqn: added evaluation/ (backtesting engine, metrics, reports)
  and checkpoint implementation
- ml-supervised: added checkpoint implementations
- ml-core: added shared types needed by new sub-crates

Pattern: each module in ml/ becomes a thin facade (pub use subcrate::*)
with bridge modules staying in ml for cross-model adapter code.

Dead code deleted (~7K lines):
- 13 undeclared files in microstructure/ (never compiled)
- 7 undeclared files + tests/ in risk/ (never compiled)
- parquet_io, cache_service, cache_storage, minio_integration (unused)
- extraction_wave_d_impl.rs (bare fn outside impl block)

All 2,746 sub-crate tests + 951 ml tests pass.
Full workspace builds clean.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-03-08 13:09:14 +01:00
parent 3db7f4828b
commit d313486dc2
170 changed files with 3340 additions and 10225 deletions

View File

@@ -28,7 +28,7 @@ simd = [] # SIMD without heavy dependencies
# Storage and memory management features
gc = [] # Garbage collection features
s3-storage = ["aws-config", "aws-sdk-s3", "aws-types", "aws-credential-types", "urlencoding"] # S3 storage backend with AWS SDK
s3-storage = ["ml-checkpoint/s3-storage", "aws-config", "aws-sdk-s3", "aws-types", "aws-credential-types", "urlencoding"] # S3 storage backend with AWS SDK
cuda = ["candle-core/cuda", "candle-core/cudnn", "candle-nn/cuda", "candle-nn/cudnn"] # CUDA support (includes LSTM sigmoid ops) - OPTIONAL for CI/Docker
nccl = ["cuda"] # NCCL multi-GPU data parallelism (requires NCCL library + cudarc nccl feature)
@@ -69,9 +69,18 @@ colored = "2.1" # Terminal color output for evaluation reports
# Internal workspace crates
ml-core.workspace = true
ml-ensemble.workspace = true
ml-dqn.workspace = true
ml-ppo.workspace = true
ml-supervised.workspace = true
ml-hyperopt.workspace = true
ml-features.workspace = true
ml-labeling.workspace = true
ml-regime.workspace = true
ml-checkpoint.workspace = true
ml-data-validation.workspace = true
ml-validation.workspace = true
ml-risk.workspace = true
config.workspace = true
common = { workspace = true, features = ["questdb"] }
risk = { path = "../risk" }