Commit Graph

5 Commits

Author SHA1 Message Date
jgrusewski
d313486dc2 refactor(ml): split monolith into 9 sub-crates + delete dead code
Extract 9 new sub-crates from the ml monolith to enable parallel
compilation across the workspace:

New crates (this commit):
- ml-features (282 tests): feature engineering, 21 modules
- ml-labeling (45 tests): triple barrier, meta-labeling, fractional diff
- ml-ensemble (116 tests): ensemble coordination, voting, confidence
- ml-hyperopt (47 tests): core PSO/TPE optimizer, parameter space
- ml-checkpoint (41 tests): checkpoint persistence, compression, signing
- ml-regime (68 tests): CUSUM, Bayesian changepoint, regime classification
- ml-data-validation (67 tests): FDR correction, CPCV, data quality
- ml-risk (33 tests): neural VaR, Kelly criterion, circuit breakers
- ml-validation (43 tests): statistical validation, walk-forward, DSR

Extended existing crates:
- ml-dqn: added evaluation/ (backtesting engine, metrics, reports)
  and checkpoint implementation
- ml-supervised: added checkpoint implementations
- ml-core: added shared types needed by new sub-crates

Pattern: each module in ml/ becomes a thin facade (pub use subcrate::*)
with bridge modules staying in ml for cross-model adapter code.

Dead code deleted (~7K lines):
- 13 undeclared files in microstructure/ (never compiled)
- 7 undeclared files + tests/ in risk/ (never compiled)
- parquet_io, cache_service, cache_storage, minio_integration (unused)
- extraction_wave_d_impl.rs (bare fn outside impl block)

All 2,746 sub-crate tests + 951 ml tests pass.
Full workspace builds clean.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 15:17:22 +01:00
jgrusewski
7d1b0f232f refactor(ml): move core types to ml-core + dedup MLError (5a)
Move all inline type definitions from ml/src/lib.rs to ml-core:
MLError, MLResult, Trade, MarketRegime, HealthStatus, Features,
MLModel trait, ModelRegistry, ParallelExecutor, LatencyOptimizer,
TrainingMetrics, ValidationMetrics, InferenceResult, ModelMetadata.

Dedup: consolidate ConfigError{reason}/ConfigurationError(msg) into
single ConfigError(String) tuple variant (was 2 variants, 174 refs).

Cleanup: convert create_hft_* free functions to associated methods
(HFTPerformanceProfile::ultra_low_latency(), ParallelExecutor::hft()).

ml facade re-exports via `pub use ml_core::*`.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 15:13:41 +01:00
jgrusewski
03062f2401 fix(ml): use exposure index (0-4) in DQN experience storage and tracking
FactoredAction::to_index() returns 0-44 (exposure×9 + order×3 + urgency),
but DQN Q-network has 5 outputs. Storing the factored index in Experience
caused train_step() to reject valid actions (e.g. Flat→index 19 > num_actions=5).

Fix: use action.exposure as usize (0-4) for:
- Experience storage (training loop + validation adapter)
- Count bonus tracking (diversity metrics)
- Safety action counts (diversity monitor)
- Debug logging (action indices)
- Test assertions (< 45 → < 5)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 16:27:17 +01:00
jgrusewski
512535076f feat(ml): wire PPO to 45 factored actions via FactoredAction
sample_action(), act(), act_with_log_prob(), greedy_action() now return
FactoredAction instead of TradingAction. Fixes the architectural disconnect
where num_actions=45 output neurons were sampled through a 3-action bottleneck.

TrajectoryStep.action and TrajectoryBatch.actions now use FactoredAction.
Added FactoredAction::from_legacy() for backward compatibility in tests.

Updated all PPO consumers: trainers/ppo.rs, hyperopt/adapters/ppo.rs,
validation/ppo_adapter.rs, benchmark/ppo_benchmark.rs.

2487 tests pass, 0 clippy warnings.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 15:41:23 +01:00
jgrusewski
9c3d741a08 refactor: restructure repo — crates/, bin/, testing/ layout
Move 17 library crates into crates/, CLI binary into bin/fxt,
consolidate 10 test crates into testing/, split config crate
from deployment config files.

Root directory reduced from 38+ to ~17 directories.
All Cargo.toml paths and build.rs proto refs updated.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 11:56:00 +01:00