Files

jgrusewski 5235b4515b fix(data,ml-backtesting): DBN nanoprice scaling + skip-on-corrupt-top-of-book

Two root-cause fixes surfaced by cluster smoke v74v4 (sweep_smoke-a2dfc6d99):

Bug D — DBN parser price scaling:
- BidAskPair::price_to_f64/price_from_f64 used /1e12 / *1e12 from test-data
  calibration. DBN production uses 1e-9 nanoprice (the DBN standard). ES at
  5500 raw 5_500_000_000_000 → 5.5 instead of 5500. Smoke trade records
  showed entry_px=5.24 instead of expected ~5240 ES index points (1000×
  too small). Fix: 1e12 → 1e9 in both functions. Round-trip symmetric;
  tests updated.

Bug C-b — corrupt top-of-book sentinel:
- Per-level sanitization (Task 15) zeros each unhealthy MBP-10 level
  individually. At session-boundary events with all 10 levels invalid,
  the book becomes uniformly zero. apply_fill_to_pos then reads
  bid_px[0]=0 / ask_px[0]=0 → vwap_entry=0 → trade record entry_px=0
  (zero sentinel in v74v4 CSV, 162/1024 trades in n59t4).
- Fix: pre-validate top-of-book in apply_snapshot_kernel. If
  bid_px[0]/ask_px[0]/bid_sz[0]/ask_sz[0] are non-finite or
  bid_px[0]<=0/ask_px[0]<=0/bid_sz[0]<=0/ask_sz[0]<=0, atomically skip
  the entire snapshot (book/prev_mid/atr_mid_ema unchanged). Add
  per-backtest snapshots_skipped_d counter for observability.

Test corrupt_top_of_book_skips_snapshot_and_increments_counter validates
NaN top-price + zero top-size cases both increment the counter without
mutating state, and that a subsequent valid snapshot updates normally.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-20 09:37:07 +02:00

benches

fix(clippy): resolve all clippy warnings across entire workspace (--all-targets)

2026-03-13 10:18:35 +01:00

data

refactor: restructure repo — crates/, bin/, testing/ layout

2026-02-25 11:56:00 +01:00

src

fix(data,ml-backtesting): DBN nanoprice scaling + skip-on-corrupt-top-of-book

2026-05-20 09:37:07 +02:00

tests

fix(data,ml-backtesting): DBN nanoprice scaling + skip-on-corrupt-top-of-book

2026-05-20 09:37:07 +02:00

Cargo.toml

build: per-package CGU + dep dedup — workspace builds ~30% faster

2026-05-01 01:00:52 +02:00

README.md

docs: create/update README.md for all 17 crates

2026-03-01 22:47:39 +01:00

README.md

data

Market data ingestion, broker integration, and feature extraction.

Providers

Databento — historical and real-time market data via DBN format
Benzinga — news and fundamentals feed

Broker Integrations

IB TWS — Interactive Brokers TWS/Gateway socket connection
ICMarkets — FIX 4.4 protocol integration

Key Modules

brokers — broker adapters (IB TWS, ICMarkets FIX)
providers — data provider clients (Databento, Benzinga)
parquet_persistence — Parquet read/write for tick and bar data
replay — market data replay for backtesting
training_pipeline — data preparation for ML model training
features — technical indicator and feature computation
validation — data quality checks and schema validation

Cargo Features

Feature	Default	Description
`databento`	yes	Databento provider support
`benzinga`	yes	Benzinga provider support
`icmarkets`	yes	ICMarkets FIX 4.4 integration
`redis-cache`	no	Redis caching layer
`ib`	no	Interactive Brokers adapter
`mock`	no	Mock providers for testing