docs: ADR 0003 — research/trader split over SSOT feature-warehouse (architecture + port roadmap + conservation principle)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

docs/architecture/0003-research-trader-feature-warehouse.md

@@ -0,0 +1,86 @@
+# ADR 0003 — Research/Trader split over an SSOT Feature Warehouse
+
+Status: accepted (vision); building foundation-first.
+Extends ADR 0002 (the factory becomes the research-validation + trading engine; analyst agents + the
+feature warehouse are the new top layer).
+
+## Context
+
+The first agentic factory was too simplistic: an LLM choosing parameters over toy features (rolling
+mean/std). A real quant shop is organized as a **research floor** (domain-specialist analysts who *study*
+markets through rich, professional features) and a **trading desk** (sizing, risk, execution). We split
+the engine the same way, over a **Single Source of Truth feature warehouse** built from the advanced
+features the foxhunt Rust `ml-features` crate already computes (microprice, OFI, VPIN, PIN, Hawkes,
+Bouchaud, frac-diff, trend-scanning, LOB-PCA, alternative bars — research-grade, causal, deterministic).
+
+## First principles
+
+1. **Conservation of money ("follow the money").** Money never vanishes — every cent is accounted for.
+   This is BOTH an invariant (the warehouse + trading engine must reconcile to the cent: cash + position
+   value + realized = constant modulo external flows; an imbalance is a bug, not noise — the foxhunt
+   mislabeled-PnL and broker-inconsistency failures were undetected conservation violations) AND an
+   analytical lens (order flow conserves; OFI/VPIN/book-deltas/Kyle-λ literally trace where capital
+   moves). It elevates the flow features to first priority.
+2. **SSOT, layered, each layer a deterministic transform of the prior.** raw (bronze) → normalized
+   (silver) → features (gold) → analyst views. The same SSOT input always yields the same gold.
+3. **Point-in-time / no lookahead.** Features are stored as-known-at-time; an `as_of(ts)` read can never
+   leak the future. A single lookahead bug poisons every conclusion above it — so the foundation comes
+   first.
+4. **The narrative buys nothing.** LLM analysts are narrative engines; a plausible market story is a
+   *search heuristic* (where to look), never evidence. Every hypothesis still faces the deflated-Sharpe
+   gauntlet, regime-conditioning, and forward-validation. The analyst's value is smarter priors over a
+   rich feature space (fewer, better hypotheses → lower multiple-testing burden) + reading signals a grid
+   cannot. The smarter the research, the MORE load-bearing the discipline.
+
+## Architecture
+```
+ Databento ─(ingest)─►┌──── FEATURE WAREHOUSE (point-in-time, DuckDB/parquet) ────┐
+ Rust ml-features ────►│ raw OHLCV/MBP-10/trades  +  gold features (TDD-ported)   │
+ (oracle, not runtime) └──────────────────────────┬──────────────────────────────┘
+ RESEARCH ENGINE ─► [microstructure] [volatility] [macro/flow] [crypto] analyst agents
+   (domain specialists)  query warehouse → analyze → grounded, TESTABLE hypotheses
+                                          │
+                      GAUNTLET + REGIME + FORWARD-VALIDATION (the discipline)
+                                          │ validated regime-specialists
+ ═══════════════════════════════════════════════════════════════════════════════
+ TRADING ENGINE ─► regime-conditional book → sizing/risk → IBKR   (conservation-checked)
+```
+
+## Feature-port methodology (Python, Rust as oracle)
+
+Port each Rust feature to a pure/streaming Python calculator, TDD against the Rust `#[cfg(test)]` values
+(reuse them as golden assertions). Match bit-for-bit where possible; strict tolerance otherwise. Gotchas:
+1e-9 DBN price scaling; NaN/Inf→0 sanitation (frac-diff legitimately NaN during warmup); `f64::signum(0)
+= +1.0` (use `math.copysign`); exact loop order in Sanger/Hawkes/EM; sequential single-pass is the
+bit-exact reference (skip the rayon chunk-warmup). Streaming features = stateful classes mirroring the
+Rust online state; batch features = pure transforms. numpy/numba/polars for hot paths, with the tolerance
+test as the safety net.
+
+## Feature-port roadmap (from the ml-features inventory — alpha vector is 134-dim)
+
+- **Tier 1 (Low, pure, rich tests):** ✅ microprice (Block N+V — DONE) · frac-diff (M) · return-moments
+  (AA+EE) · trend-scanning (X) · OFI stateless per-level fns (F/G/H/CC) · bouchaud (BB) · order-events (W).
+- **Tier 2 (Medium, streaming, exact loop order):** spread-decomposition (O) · multi-level Kyle-λ + VPIN
+  (T/U) · LOB-PCA Sanger (Y) · price/volume/statistical (A/B/C) · ADX/regime-ADX Wilder (D/D') · time +
+  seasonality (E/DD).
+- **Tier 3 (High, optimization):** Hawkes-MLE grid search (R) · PIN EM-MLE (Z).
+- **Tier 4 (assembly):** imbalance/volume bar samplers + tick-rule · 134-dim alpha + 81-dim snapshot
+  pipelines.
+- **Defer/skip:** legacy 65-dim pipeline, config.rs Wave A-D, position_features (RL state), DBN readers.
+- **Prioritized by conservation:** the order-flow blocks (OFI/VPIN/book-deltas/Kyle-λ) are the
+  follow-the-money signals → bring them forward within their tiers.
+
+## Open optimization (verified)
+
+The Rust "Stoikov microprice" is the *weighted mid-price*. Stoikov's true micro-price (Quant Finance
+2018) is a recursive Markov-chain estimator and empirically a better future-price predictor. ADD it as a
+separate `true_microprice` feature (not a silent oracle divergence) so the gauntlet A/Bs them empirically.
+
+## Build order
+
+1. ✅ Feature-warehouse foundation (silver marketdata + gold FeatureStore + pipeline + microprice TDD).
+2. Port Tier-1 features (flow first), feeding the warehouse from the real MBP-10/trades data we hold.
+3. Research/Trader split formalized; analyst-agent query tools over the warehouse.
+4. Domain-specialist analyst agents → grounded hypotheses → gauntlet.
+5. Conservation/reconciliation layer in the trading engine (every cent accounted for).
+6. Falsification analysts; true micro-price + advanced features.