9c3d741a08
Move 17 library crates into crates/, CLI binary into bin/fxt, consolidate 10 test crates into testing/, split config crate from deployment config files. Root directory reduced from 38+ to ~17 directories. All Cargo.toml paths and build.rs proto refs updated. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
43 lines
1.5 KiB
Markdown
43 lines
1.5 KiB
Markdown
# data
|
|
|
|
Market data ingestion, broker integration, and feature extraction.
|
|
|
|
## Providers
|
|
|
|
- **Databento** -- Historical and real-time market data via DBN format
|
|
- **Benzinga** -- News and fundamentals feed
|
|
|
|
## Broker Integrations
|
|
|
|
- **IB TWS** -- Interactive Brokers TWS/Gateway socket connection
|
|
- **ICMarkets** -- FIX 4.4 protocol integration
|
|
|
|
## Key Modules
|
|
|
|
- **brokers** -- Broker adapters (IB TWS, ICMarkets FIX)
|
|
- **providers** -- Data provider clients (Databento, Benzinga)
|
|
- **dbn_uploader** -- DBN file upload and processing
|
|
- **parquet_persistence** -- Parquet read/write for tick and bar data
|
|
- **replay** -- Market data replay for backtesting
|
|
- **training_pipeline** -- Data preparation for ML model training
|
|
- **features** -- Technical indicator and feature computation
|
|
- **unified_feature_extractor** -- Normalized feature vectors across providers
|
|
- **validation** -- Data quality checks and schema validation
|
|
|
|
## Cargo Features
|
|
|
|
| Feature | Default | Description |
|
|
|---------------|---------|------------------------------------|
|
|
| `databento` | yes | Databento provider support |
|
|
| `benzinga` | yes | Benzinga provider support |
|
|
| `icmarkets` | yes | ICMarkets FIX 4.4 integration |
|
|
| `redis-cache` | no | Redis caching layer |
|
|
| `ib` | no | Interactive Brokers TWS adapter |
|
|
| `mock` | no | Mock providers for testing |
|
|
|
|
## Testing
|
|
|
|
```bash
|
|
SQLX_OFFLINE=true cargo test -p data --lib
|
|
```
|