The three TODO comments claiming \"TrainingPipelineConfig needs a new
struct\" were stale — the config already exposes `storage.base_directory`
(test_process_features_full_workflow_success already uses it). Wire
up the two previously-neutered tests so they actually exercise their
intended behaviour:
- `test_pipeline_creation_storage_dir_is_file_fails` now sets
base_directory to a regular file and asserts pipeline creation
returns Err (create_dir_all on a file path fails with ENOTDIR).
- `test_process_features_dataset_not_found` now points storage at a
fresh tempdir so the NotFound assertion runs against an isolated
state rather than the default on-disk location.
- The inline \"fixed from TODO comment\" note in the third test is
replaced with a plain description of why the tempdir override is
needed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- common/sqlx_test.rs: the entire module was a single /* ... */ block
behind a TODO because the identifier types (`OrderId`, `TradeId`,
`Symbol`, `AccountId`, `OrderSide`) do not derive `sqlx::Type`. The
file has been a dead placeholder behind `#[cfg(all(test,
feature=\"database\"))]` for a long time; delete it and drop the
`mod sqlx_test;` declaration.
- data/brokers/mod.rs: re-enable the `pub type BrokerAdapter = Box<dyn
BrokerClient>;` alias — the trait is already implemented by
`InteractiveBrokersAdapter` and re-exported from the module. Delete
the commented-out `BrokerFactory::create_client` block: it
referenced `ICMarketsClient`, which does not exist in this crate.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Removes commented-out ProviderMetrics tests (struct was replaced by
ConnectionStatus long ago) and reword the Parquet-reader integration
tests so they stop claiming the reader is a "placeholder" — the Parquet
reader is fully implemented and surfaces `File::open` errors via
anyhow. Also tightens test_12_invalid_file_handling to assert the real
Err behaviour rather than the stale "Ok(vec![])" expectation.
No production code change; tests still compile and run under the same
ignore gates.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three fixes:
- Streaming MBP-10 parser (parse_mbp10_streaming): computes OFI inline
during decode — eliminates Vec<Mbp10Snapshot> allocation (41.9M clones)
- Parallel file processing: rayon par_iter across 9 MBP-10 files
(778s sequential → ~90s expected on H100 24-core)
- Fix hardcoded state_dim=54 in walk-forward backtest tensor creation
that caused panic "range end index 55296 out of range for slice of
length 52224" — now uses dynamic state_dim (43 or 51 with OFI)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wire 8 OFI features (OFI L1/L5, depth imbalance, VPIN, Kyle's lambda,
bid/ask slopes, trade imbalance) through the DQN training pipeline:
- Add mbp10_data_dir config field to DQNHyperparameters
- Dynamic state_dim: 43 (no OFI) or 51 (with OFI) based on config
- Compute OFI per bar during data loading, store on trainer
- Pass OFI features through regime_features slot in TradingState
- Configurable MBP-10 path with recursive .dbn/.dbn.zst discovery
- Add zstd auto-detection to DbnParser::parse_mbp10_file()
- Add --mbp10-data-dir CLI flag to train_baseline_rl
- Fix hardcoded [f64; 51] → FeatureVector51 ([f64; 40]) across
examples, walk_forward, GPU memory profile, and test fixtures
- Fix stale state_dim=51 in dqn_config_2025() and DQN tests
2747 tests pass, 0 failures.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Re-export HardwareTimestamp through data crate instead of ml/risk
depending directly on trading_engine. Reduces coupling between
the ML pipeline and the trading engine.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove stale test reports, quick-start guides, benchmark analyses,
profiling reports, and tool artifacts from across the workspace.
Keeps only root README.md per crate/service.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three duplicate ErrorSeverity enums (data/error.rs, data/validation.rs,
database/error.rs) with variants Low/Medium/High/Critical now use the
canonical definition in common::error::ErrorSeverity.
Extended common's ErrorSeverity with Low, Medium, High variants (alongside
existing Debug, Info, Warn, Error, Critical) and added PartialOrd/Ord derives
so both severity models coexist in a single type.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace all `.to_string().parse::<f32/f64>()` patterns with
`num_traits::ToPrimitive` methods (`.to_f32()`, `.to_f64()`).
Each string roundtrip heap-allocated per conversion — fatal in
DQN hot loop (300K+ bars × epochs). Decimal stays as canonical
financial type; conversions happen at GPU/float boundaries only.
Also fixes blocking_read() in async context (risk_integration.rs).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Delete 22 dead/placeholder/broken example files (-3,489 lines code)
- Delete 4 tracked CSV files (-1.1M lines, were accidentally committed)
- Move baseline training data default from data/cache/ to test_data/
- Update 5 unified binary defaults, gitignore, k8s upload comment, docs
- Consolidate all training data under test_data/futures-baseline/
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move 17 library crates into crates/, CLI binary into bin/fxt,
consolidate 10 test crates into testing/, split config crate
from deployment config files.
Root directory reduced from 38+ to ~17 directories.
All Cargo.toml paths and build.rs proto refs updated.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
