Three fixes:
- Streaming MBP-10 parser (parse_mbp10_streaming): computes OFI inline
during decode — eliminates Vec<Mbp10Snapshot> allocation (41.9M clones)
- Parallel file processing: rayon par_iter across 9 MBP-10 files
(778s sequential → ~90s expected on H100 24-core)
- Fix hardcoded state_dim=54 in walk-forward backtest tensor creation
that caused panic "range end index 55296 out of range for slice of
length 52224" — now uses dynamic state_dim (43 or 51 with OFI)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wire 8 OFI features (OFI L1/L5, depth imbalance, VPIN, Kyle's lambda,
bid/ask slopes, trade imbalance) through the DQN training pipeline:
- Add mbp10_data_dir config field to DQNHyperparameters
- Dynamic state_dim: 43 (no OFI) or 51 (with OFI) based on config
- Compute OFI per bar during data loading, store on trainer
- Pass OFI features through regime_features slot in TradingState
- Configurable MBP-10 path with recursive .dbn/.dbn.zst discovery
- Add zstd auto-detection to DbnParser::parse_mbp10_file()
- Add --mbp10-data-dir CLI flag to train_baseline_rl
- Fix hardcoded [f64; 51] → FeatureVector51 ([f64; 40]) across
examples, walk_forward, GPU memory profile, and test fixtures
- Fix stale state_dim=51 in dqn_config_2025() and DQN tests
2747 tests pass, 0 failures.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Re-export HardwareTimestamp through data crate instead of ml/risk
depending directly on trading_engine. Reduces coupling between
the ML pipeline and the trading engine.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove stale test reports, quick-start guides, benchmark analyses,
profiling reports, and tool artifacts from across the workspace.
Keeps only root README.md per crate/service.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three duplicate ErrorSeverity enums (data/error.rs, data/validation.rs,
database/error.rs) with variants Low/Medium/High/Critical now use the
canonical definition in common::error::ErrorSeverity.
Extended common's ErrorSeverity with Low, Medium, High variants (alongside
existing Debug, Info, Warn, Error, Critical) and added PartialOrd/Ord derives
so both severity models coexist in a single type.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace all `.to_string().parse::<f32/f64>()` patterns with
`num_traits::ToPrimitive` methods (`.to_f32()`, `.to_f64()`).
Each string roundtrip heap-allocated per conversion — fatal in
DQN hot loop (300K+ bars × epochs). Decimal stays as canonical
financial type; conversions happen at GPU/float boundaries only.
Also fixes blocking_read() in async context (risk_integration.rs).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Delete 22 dead/placeholder/broken example files (-3,489 lines code)
- Delete 4 tracked CSV files (-1.1M lines, were accidentally committed)
- Move baseline training data default from data/cache/ to test_data/
- Update 5 unified binary defaults, gitignore, k8s upload comment, docs
- Consolidate all training data under test_data/futures-baseline/
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move 17 library crates into crates/, CLI binary into bin/fxt,
consolidate 10 test crates into testing/, split config crate
from deployment config files.
Root directory reduced from 38+ to ~17 directories.
All Cargo.toml paths and build.rs proto refs updated.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
