The three TODO comments claiming \"TrainingPipelineConfig needs a new struct\" were stale — the config already exposes `storage.base_directory` (test_process_features_full_workflow_success already uses it). Wire up the two previously-neutered tests so they actually exercise their intended behaviour: - `test_pipeline_creation_storage_dir_is_file_fails` now sets base_directory to a regular file and asserts pipeline creation returns Err (create_dir_all on a file path fails with ENOTDIR). - `test_process_features_dataset_not_found` now points storage at a fresh tempdir so the NotFound assertion runs against an isolated state rather than the default on-disk location. - The inline \"fixed from TODO comment\" note in the third test is replaced with a plain description of why the tempdir override is needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
data
Market data ingestion, broker integration, and feature extraction.
Providers
- Databento — historical and real-time market data via DBN format
- Benzinga — news and fundamentals feed
Broker Integrations
- IB TWS — Interactive Brokers TWS/Gateway socket connection
- ICMarkets — FIX 4.4 protocol integration
Key Modules
brokers— broker adapters (IB TWS, ICMarkets FIX)providers— data provider clients (Databento, Benzinga)parquet_persistence— Parquet read/write for tick and bar datareplay— market data replay for backtestingtraining_pipeline— data preparation for ML model trainingfeatures— technical indicator and feature computationvalidation— data quality checks and schema validation
Cargo Features
| Feature | Default | Description |
|---|---|---|
databento |
yes | Databento provider support |
benzinga |
yes | Benzinga provider support |
icmarkets |
yes | ICMarkets FIX 4.4 integration |
redis-cache |
no | Redis caching layer |
ib |
no | Interactive Brokers adapter |
mock |
no | Mock providers for testing |