gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
aa848bb9be342c8a5205b4b4d43d1986257b8d98
foxhunt/ml/Cargo.toml
jgrusewski 6bc40d9412 🎉 Wave 12: Fixed 766 test compilation errors (92% reduction) 
Wave 12 Achievement - 12 Parallel Agents Deployed:
- Starting errors: 832 test compilation errors
- Ending errors: 66 errors
- Fixed: 766 errors (92.1% error reduction)

Package Results:
✅ Storage: 3 → 0 errors (100% complete)
✅ Trading Engine: 36 → 0 errors (100% complete)
✅ Risk: 29 → 0 errors (100% complete)
✅ ML: ~584 → ~0 errors (core infrastructure fixed)
✅ Data: 127 → 62 errors (51% reduction, pipeline tests fixed)
⚠️ Adaptive-Strategy: 60 → 18 errors (70% reduction, Wave 13 needed)

Agent Accomplishments:

Agent 1 - ML Core Infrastructure:
- Fixed blocking config crate compilation (num_cpus import)
- Created test_common module for reusable test utilities
- Fixed SignalStatistics export visibility
- Added comprehensive documentation and automation scripts

Agent 2 - ML Tracing & Logging:
- Added tracing-subscriber to dev-dependencies
- Fixed data_to_ml_pipeline_test.rs imports
- Added Clone derives for mock services
- Created proper test module structure

Agent 3 - MAMBA-2 & TLOB Models:
- Fixed mamba_test.rs config structure (18 fields updated)
- Fixed tlob_transformer_test.rs missing types
- Created helper functions for test configs
- Updated to use actual struct implementations

Agent 4 - DQN & PPO RL:
- Fixed 9 DQN test files
- Updated WorkingDQNConfig to use emergency_safe_defaults()
- Fixed Price/Decimal type conversions
- Fixed multi-step learning and Rainbow network tests
- PPO tests already working (no fixes needed)

Agent 5 - Liquid Networks & TFT:
- Fixed 4 Liquid Networks test files (20 tests)
- Added PRECISION, SolverType, ActivationType imports
- Fixed Result return types on all test functions
- TFT tests already correct (no changes needed)

Agent 6 - ML Labeling & Features:
- Fixed 7 labeling module test files
- Added BarrierResult imports
- Fixed fractional_diff import paths
- Updated 15+ test functions with proper Result returns
- Fixed meta-labeling, triple barrier, sample weights tests

Agent 7 - Training Pipeline:
- Added comprehensive config re-exports to training_pipeline.rs
- Created DataProcessingConfig struct
- Extended enum variants (MissingDataHandling, OutlierDetectionMethod)
- Fixed training pipeline tests: 94 errors → 0
- Fixed training_pipeline_demo example

Agent 8 - Parquet Persistence:
- Enabled parquet_persistence module
- Fixed ParquetMarketDataEvent schema (8 fields, not 12)
- Updated imports to trading_engine::types::metrics
- Fixed storage_test.rs config import conflicts
- Removed non-existent bid/ask price/size fields

Agent 9 - Trading Engine:
- Fixed 9 files with 36 errors → 0
- Updated event_types.rs decimal macros
- Fixed SIMD intrinsic imports
- Fixed account_manager and order_manager test imports
- Fixed CommonError variant usage
- Fixed event_processing_demo example

Agent 10 - Risk Management:
- Fixed 8 files with 29 errors → 0
- Added num_cpus dependency to config
- Fixed AssetClass import (config::asset_classification)
- Fixed MarketCapTier import paths
- Updated position tracker method names (update_position_sync)
- Fixed EnhancedRiskPosition field access patterns
- Fixed type conversions (Price::from_f64, Quantity::from_f64)

Agent 11 - Adaptive Strategy:
- Fixed 2 example files
- Fixed 42 errors (60 → 18)
- Added tracing-subscriber dependency
- Fixed MarketRegime variants
- Fixed async/await patterns
- Fixed RiskConfig, RegimeConfig field mismatches
- 18 errors remain for Wave 13

Agent 12 - Storage & Verification:
- Fixed 3 storage errors → 0
- Updated S3Config schema in tests
- Verified workspace compilation: 66 errors remaining
- Generated comprehensive reports
- 24/26 storage tests passing (92.3%)

Key Technical Fixes:
1. Configuration types: Proper imports from config::data_config
2. Type safety: Price/Decimal conversions with from_f64()
3. Async patterns: Proper .await usage
4. Import organization: Canonical paths from common crate
5. Test infrastructure: Reusable test_common module
6. Error handling: Result return types on test functions

Remaining Work (66 errors):
- Adaptive-strategy: 58 errors (88% of remaining)
- Trading engine: 6 errors (hidden behind adaptive-strategy)
- Config examples: 2 errors (non-critical)

Next: Wave 13 to fix remaining 66 errors

Reports Generated:
- /tmp/wave12_test_fixes_summary.md
- /tmp/wave12_quick_summary.txt
- /tmp/test_compilation_wave12_final.log
2025-09-30 14:46:43 +02:00

152 lines
4.3 KiB
TOML
Raw Blame History

[package]
name = "ml"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
authors.workspace = true
license.workspace = true
repository.workspace = true
homepage.workspace = true
documentation.workspace = true
publish.workspace = true
keywords.workspace = true
categories.workspace = true
[features]
# MINIMAL features for HFT inference only - ALL HEAVY ML REMOVED
default = ["minimal-inference"]
# PRODUCTION FEATURES - LIGHTWEIGHT ONLY
minimal-inference = [] # Minimal inference with no optional deps
financial = [] # Basic financial calculations
high-precision = ["rust_decimal/serde-float"]
# PERFORMANCE FEATURES - NO HEAVY ML
simd = [] # SIMD without heavy dependencies
# Storage and memory management features
gc = [] # Garbage collection features
s3-storage = [] # S3 storage backend
cuda = [] # CUDA support (moved to training service)
# ALL HEAVY ML FEATURES REMOVED:
# gpu, pytorch, linfa-ml - MOVED TO ml_training_service
# optimization, graph-models, reinforcement-learning - MOVED TO ml_training_service
# transformers-advanced - MOVED TO ml_training_service
[dependencies]
# Core async and utilities
tokio.workspace = true
futures.workspace = true
async-trait.workspace = true
# Serialization and error handling
serde.workspace = true
serde_json.workspace = true
uuid.workspace = true
thiserror.workspace = true
anyhow.workspace = true
# System and I/O
memmap2.workspace = true
tempfile.workspace = true
tracing.workspace = true
prometheus.workspace = true
reqwest.workspace = true
# Internal workspace crates
trading_engine.workspace = true
config.workspace = true
common.workspace = true
risk = { path = "../risk" }
# Model loading functionality is in storage crate
storage = { path = "../storage" }
# Essential ML frameworks for HFT inference - CUDA REQUIRED FOR PERFORMANCE
candle-core = { version = "0.9", features = ["cuda", "cudnn"] } # CUDA mandatory for HFT latency
candle-nn = { version = "0.9" }
candle-optimisers = { version = "0.9" }
# HEAVY ML FRAMEWORKS REMOVED - MOVED TO ml_training_service
# ort (ONNX Runtime) - REMOVED (1000+ dependencies alone!)
# tch, torch-sys (PyTorch bindings) - REMOVED (500+ dependencies!)
# Mathematical libraries
ndarray = { version = "0.15", features = ["rayon", "blas", "serde"] }
nalgebra = { version = "0.33", features = ["serde-serialize"] }
arrayfire = { version = "3.8", optional = true }
# MINIMAL statistics only - ALL HEAVY ML ALGORITHMS REMOVED
# linfa ecosystem (linfa, linfa-clustering, linfa-linear, linfa-reduction) - REMOVED (200+ deps)
# smartcore - REMOVED (100+ dependencies)
# Basic statistics - always included (not optional)
statrs.workspace = true # Required for statistical computations
rust_decimal.workspace = true
# gymnasium, rerun - REMOVED (RL frameworks moved to ml_training_service)
# cudarc, wgpu - REMOVED (GPU frameworks moved to ml_training_service)
rayon.workspace = true
crossbeam = { version = "0.8", features = ["std"] }
petgraph = { version = "0.6", features = ["serde"] } # Required for TGNN graphs
semver = "1.0"
# chronoutil, ta, polars - REMOVED or moved to workspace dependencies
# argmin, nlopt, ipopt - REMOVED (optimization frameworks moved to ml_training_service)
half = { version = "2.6.0", features = ["serde"] }
rand = { version = "0.8.5", features = ["small_rng", "getrandom"] }
rand_distr = { version = "0.4.3" }
chrono = { version = "0.4.38", features = ["serde", "clock"] }
parking_lot = { version = "0.12", features = ["hardware-lock-elision"] }
dashmap = { version = "6.1", features = ["serde"] }
once_cell = "1.19"
lazy_static.workspace = true
flate2 = "1.0"
sha2 = "0.10"
bincode = "1.3"
fastrand = "2.1"
# wide - REMOVED (SIMD moved to trading_engine)
num-traits = "0.2"
num = "0.4"
libc = "0.2"
fs2 = "0.4"
num_cpus = "1.16"
approx.workspace = true
[dev-dependencies]
tokio-test = "0.4"
proptest = "1.5"
tempfile = "3.12"
futures-test = "0.3"
mockall = "0.13"
test-case = "3.0"
rstest = "0.22"
criterion = { version = "0.5", features = ["html_reports", "async_tokio"] }
tokio = { workspace = true, features = ["test-util", "macros"] }
insta = "1.34" # Snapshot testing for ML outputs
serial_test = "3.0" # Sequential testing for GPU resources
tracing-subscriber = { version = "0.3", features = ["env-filter", "fmt"] }
[[example]]
name = "cuda_test"
path = "examples/cuda_test.rs"
[lints]
workspace = true
Reference in New Issue View Git Blame Copy Permalink