8b81138262
Rewrite 7 crate READMEs to reflect current architecture: correct model types (DQN/PPO/TFT/Mamba2), AtomicKillSwitch, real EnsembleConfig source from ml, actual data crate purpose, web-dashboard project details, ml_training_service ports. Fix 5 api_gateway/TLI docs: strip swarm agent framing, update service endpoints to api_gateway:50050, remove deleted dashboard references and hardcoded paths. Add missing web-gateway/README.md documenting 24 REST endpoints, WebSocket support, JWT auth, and 3-tier rate limiting. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
43 lines
1.5 KiB
Markdown
43 lines
1.5 KiB
Markdown
# data
|
|
|
|
Market data ingestion, broker integration, and feature extraction.
|
|
|
|
## Providers
|
|
|
|
- **Databento** -- Historical and real-time market data via DBN format
|
|
- **Benzinga** -- News and fundamentals feed
|
|
|
|
## Broker Integrations
|
|
|
|
- **IB TWS** -- Interactive Brokers TWS/Gateway socket connection
|
|
- **ICMarkets** -- FIX 4.4 protocol integration
|
|
|
|
## Key Modules
|
|
|
|
- **brokers** -- Broker adapters (IB TWS, ICMarkets FIX)
|
|
- **providers** -- Data provider clients (Databento, Benzinga)
|
|
- **dbn_uploader** -- DBN file upload and processing
|
|
- **parquet_persistence** -- Parquet read/write for tick and bar data
|
|
- **replay** -- Market data replay for backtesting
|
|
- **training_pipeline** -- Data preparation for ML model training
|
|
- **features** -- Technical indicator and feature computation
|
|
- **unified_feature_extractor** -- Normalized feature vectors across providers
|
|
- **validation** -- Data quality checks and schema validation
|
|
|
|
## Cargo Features
|
|
|
|
| Feature | Default | Description |
|
|
|---------------|---------|------------------------------------|
|
|
| `databento` | yes | Databento provider support |
|
|
| `benzinga` | yes | Benzinga provider support |
|
|
| `icmarkets` | yes | ICMarkets FIX 4.4 integration |
|
|
| `redis-cache` | no | Redis caching layer |
|
|
| `ib` | no | Interactive Brokers TWS adapter |
|
|
| `mock` | no | Mock providers for testing |
|
|
|
|
## Testing
|
|
|
|
```bash
|
|
SQLX_OFFLINE=true cargo test -p data --lib
|
|
```
|