Files
foxhunt/crates/data
jgrusewski e91a0c9a6a cleanup: wire storage.base_directory in training_pipeline tests
The three TODO comments claiming \"TrainingPipelineConfig needs a new
struct\" were stale — the config already exposes `storage.base_directory`
(test_process_features_full_workflow_success already uses it). Wire
up the two previously-neutered tests so they actually exercise their
intended behaviour:

- `test_pipeline_creation_storage_dir_is_file_fails` now sets
  base_directory to a regular file and asserts pipeline creation
  returns Err (create_dir_all on a file path fails with ENOTDIR).
- `test_process_features_dataset_not_found` now points storage at a
  fresh tempdir so the NotFound assertion runs against an isolated
  state rather than the default on-disk location.
- The inline \"fixed from TODO comment\" note in the third test is
  replaced with a plain description of why the tempdir override is
  needed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 08:38:44 +02:00
..

data

Market data ingestion, broker integration, and feature extraction.

Providers

  • Databento — historical and real-time market data via DBN format
  • Benzinga — news and fundamentals feed

Broker Integrations

  • IB TWS — Interactive Brokers TWS/Gateway socket connection
  • ICMarkets — FIX 4.4 protocol integration

Key Modules

  • brokers — broker adapters (IB TWS, ICMarkets FIX)
  • providers — data provider clients (Databento, Benzinga)
  • parquet_persistence — Parquet read/write for tick and bar data
  • replay — market data replay for backtesting
  • training_pipeline — data preparation for ML model training
  • features — technical indicator and feature computation
  • validation — data quality checks and schema validation

Cargo Features

Feature Default Description
databento yes Databento provider support
benzinga yes Benzinga provider support
icmarkets yes ICMarkets FIX 4.4 integration
redis-cache no Redis caching layer
ib no Interactive Brokers adapter
mock no Mock providers for testing