# data

Market data ingestion, broker integration, and feature extraction.

## Providers

- **Databento** — historical and real-time market data via DBN format
- **Benzinga** — news and fundamentals feed

## Broker Integrations

- **IB TWS** — Interactive Brokers TWS/Gateway socket connection
- **ICMarkets** — FIX 4.4 protocol integration

## Key Modules

- `brokers` — broker adapters (IB TWS, ICMarkets FIX)
- `providers` — data provider clients (Databento, Benzinga)
- `parquet_persistence` — Parquet read/write for tick and bar data
- `replay` — market data replay for backtesting
- `training_pipeline` — data preparation for ML model training
- `features` — technical indicator and feature computation
- `validation` — data quality checks and schema validation

## Cargo Features

| Feature       | Default | Description                   |
|---------------|---------|-------------------------------|
| `databento`   | yes     | Databento provider support    |
| `benzinga`    | yes     | Benzinga provider support     |
| `icmarkets`   | yes     | ICMarkets FIX 4.4 integration |
| `redis-cache` | no      | Redis caching layer           |
| `ib`          | no      | Interactive Brokers adapter   |
| `mock`        | no      | Mock providers for testing    |