Files

jgrusewski 78db6f5389 fix(ml): streaming parallel OFI + fix hardcoded state_dim=54 in backtest

Three fixes:
- Streaming MBP-10 parser (parse_mbp10_streaming): computes OFI inline
  during decode — eliminates Vec<Mbp10Snapshot> allocation (41.9M clones)
- Parallel file processing: rayon par_iter across 9 MBP-10 files
  (778s sequential → ~90s expected on H100 24-core)
- Fix hardcoded state_dim=54 in walk-forward backtest tensor creation
  that caused panic "range end index 55296 out of range for slice of
  length 52224" — now uses dynamic state_dim (43 or 51 with OFI)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

2026-03-07 23:28:18 +01:00

benches

perf: eliminate heap-alloc Decimal→float casts across 19 files (36 instances)

2026-02-28 02:29:04 +01:00

data

refactor: restructure repo — crates/, bin/, testing/ layout

2026-02-25 11:56:00 +01:00

src

fix(ml): streaming parallel OFI + fix hardcoded state_dim=54 in backtest

2026-03-07 23:28:18 +01:00

tests

cleanup: delete 92 scattered .md files and .serena artifacts

2026-03-01 22:42:36 +01:00

Cargo.toml

fix(deps): unify workspace dependency versions and clean up CI

2026-02-27 20:52:06 +01:00

README.md

docs: create/update README.md for all 17 crates

2026-03-01 22:47:39 +01:00

README.md

data

Market data ingestion, broker integration, and feature extraction.

Providers

Databento — historical and real-time market data via DBN format
Benzinga — news and fundamentals feed

Broker Integrations

IB TWS — Interactive Brokers TWS/Gateway socket connection
ICMarkets — FIX 4.4 protocol integration

Key Modules

brokers — broker adapters (IB TWS, ICMarkets FIX)
providers — data provider clients (Databento, Benzinga)
parquet_persistence — Parquet read/write for tick and bar data
replay — market data replay for backtesting
training_pipeline — data preparation for ML model training
features — technical indicator and feature computation
validation — data quality checks and schema validation

Cargo Features

Feature	Default	Description
`databento`	yes	Databento provider support
`benzinga`	yes	Benzinga provider support
`icmarkets`	yes	ICMarkets FIX 4.4 integration
`redis-cache`	no	Redis caching layer
`ib`	no	Interactive Brokers adapter
`mock`	no	Mock providers for testing