8b81138262
Rewrite 7 crate READMEs to reflect current architecture: correct model types (DQN/PPO/TFT/Mamba2), AtomicKillSwitch, real EnsembleConfig source from ml, actual data crate purpose, web-dashboard project details, ml_training_service ports. Fix 5 api_gateway/TLI docs: strip swarm agent framing, update service endpoints to api_gateway:50050, remove deleted dashboard references and hardcoded paths. Add missing web-gateway/README.md documenting 24 REST endpoints, WebSocket support, JWT auth, and 3-tier rate limiting. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1.5 KiB
1.5 KiB
data
Market data ingestion, broker integration, and feature extraction.
Providers
- Databento -- Historical and real-time market data via DBN format
- Benzinga -- News and fundamentals feed
Broker Integrations
- IB TWS -- Interactive Brokers TWS/Gateway socket connection
- ICMarkets -- FIX 4.4 protocol integration
Key Modules
- brokers -- Broker adapters (IB TWS, ICMarkets FIX)
- providers -- Data provider clients (Databento, Benzinga)
- dbn_uploader -- DBN file upload and processing
- parquet_persistence -- Parquet read/write for tick and bar data
- replay -- Market data replay for backtesting
- training_pipeline -- Data preparation for ML model training
- features -- Technical indicator and feature computation
- unified_feature_extractor -- Normalized feature vectors across providers
- validation -- Data quality checks and schema validation
Cargo Features
| Feature | Default | Description |
|---|---|---|
databento |
yes | Databento provider support |
benzinga |
yes | Benzinga provider support |
icmarkets |
yes | ICMarkets FIX 4.4 integration |
redis-cache |
no | Redis caching layer |
ib |
no | Interactive Brokers TWS adapter |
mock |
no | Mock providers for testing |
Testing
SQLX_OFFLINE=true cargo test -p data --lib