78a9e08358
Replaces Option<u32> instrument_id_filter with InstrumentFilter enum {All,
Id(u32), FrontMonth}. FrontMonth runs a two-pass detect over the DBN
stream: pass 1 counts instrument_ids and collects SymbolMapping records,
picks the dominant id, validates it resolves to an ES contract via regex
ES[FGHJKMNQUVXZ]\d{1,2}; pass 2 streams the filtered records.
Motivated by alpha-perception-k54wd: a single-id filter on parent-symbol
ES.FUT data caught Q1 2024 (kept=73M) but kept=0 for Q2-Q9 because ES
front-month rolls quarterly (ESH4 -> ESM4 -> ESU4 -> ESZ4 ...). FrontMonth
self-tunes across the rolls without needing a per-file id table.
Sidecar keys distinguish modes: mbp10 / mbp10_instr<id> / mbp10_front_month.
CLI flag renamed --instrument-id -> --instrument-mode {all,id=N,front-month}
with matching parameter rename in argo-alpha-perception.sh + template.
data
Market data ingestion, broker integration, and feature extraction.
Providers
- Databento — historical and real-time market data via DBN format
- Benzinga — news and fundamentals feed
Broker Integrations
- IB TWS — Interactive Brokers TWS/Gateway socket connection
- ICMarkets — FIX 4.4 protocol integration
Key Modules
brokers— broker adapters (IB TWS, ICMarkets FIX)providers— data provider clients (Databento, Benzinga)parquet_persistence— Parquet read/write for tick and bar datareplay— market data replay for backtestingtraining_pipeline— data preparation for ML model trainingfeatures— technical indicator and feature computationvalidation— data quality checks and schema validation
Cargo Features
| Feature | Default | Description |
|---|---|---|
databento |
yes | Databento provider support |
benzinga |
yes | Benzinga provider support |
icmarkets |
yes | ICMarkets FIX 4.4 integration |
redis-cache |
no | Redis caching layer |
ib |
no | Interactive Brokers adapter |
mock |
no | Mock providers for testing |