# ADR 0003 — Research/Trader split over an SSOT Feature Warehouse

Status: accepted (vision); building foundation-first.
Extends ADR 0002 (the factory becomes the research-validation + trading engine; analyst agents + the
feature warehouse are the new top layer).

## Context

The first agentic factory was too simplistic: an LLM choosing parameters over toy features (rolling
mean/std). A real quant shop is organized as a **research floor** (domain-specialist analysts who *study*
markets through rich, professional features) and a **trading desk** (sizing, risk, execution). We split
the engine the same way, over a **Single Source of Truth feature warehouse** built from the advanced
features the foxhunt Rust `ml-features` crate already computes (microprice, OFI, VPIN, PIN, Hawkes,
Bouchaud, frac-diff, trend-scanning, LOB-PCA, alternative bars — research-grade, causal, deterministic).

## First principles

1. **Conservation of money ("follow the money").** Money never vanishes — every cent is accounted for.
   This is BOTH an invariant (the warehouse + trading engine must reconcile to the cent: cash + position
   value + realized = constant modulo external flows; an imbalance is a bug, not noise — the foxhunt
   mislabeled-PnL and broker-inconsistency failures were undetected conservation violations) AND an
   analytical lens (order flow conserves; OFI/VPIN/book-deltas/Kyle-λ literally trace where capital
   moves). It elevates the flow features to first priority.
2. **SSOT, layered, each layer a deterministic transform of the prior.** raw (bronze) → normalized
   (silver) → features (gold) → analyst views. The same SSOT input always yields the same gold.
3. **Point-in-time / no lookahead.** Features are stored as-known-at-time; an `as_of(ts)` read can never
   leak the future. A single lookahead bug poisons every conclusion above it — so the foundation comes
   first.
4. **The narrative buys nothing.** LLM analysts are narrative engines; a plausible market story is a
   *search heuristic* (where to look), never evidence. Every hypothesis still faces the deflated-Sharpe
   gauntlet, regime-conditioning, and forward-validation. The analyst's value is smarter priors over a
   rich feature space (fewer, better hypotheses → lower multiple-testing burden) + reading signals a grid
   cannot. The smarter the research, the MORE load-bearing the discipline.

## Architecture
```
 Databento ─(ingest)─►┌──── FEATURE WAREHOUSE (point-in-time, DuckDB/parquet) ────┐
 Rust ml-features ────►│ raw OHLCV/MBP-10/trades  +  gold features (TDD-ported)   │
 (oracle, not runtime) └──────────────────────────┬──────────────────────────────┘
 RESEARCH ENGINE ─► [microstructure] [volatility] [macro/flow] [crypto] analyst agents
   (domain specialists)  query warehouse → analyze → grounded, TESTABLE hypotheses
                                          │
                      GAUNTLET + REGIME + FORWARD-VALIDATION (the discipline)
                                          │ validated regime-specialists
 ═══════════════════════════════════════════════════════════════════════════════
 TRADING ENGINE ─► regime-conditional book → sizing/risk → IBKR   (conservation-checked)
```

## Feature-port methodology (Python, Rust as oracle)

Port each Rust feature to a pure/streaming Python calculator, TDD against the Rust `#[cfg(test)]` values
(reuse them as golden assertions). Match bit-for-bit where possible; strict tolerance otherwise. Gotchas:
1e-9 DBN price scaling; NaN/Inf→0 sanitation (frac-diff legitimately NaN during warmup); `f64::signum(0)
= +1.0` (use `math.copysign`); exact loop order in Sanger/Hawkes/EM; sequential single-pass is the
bit-exact reference (skip the rayon chunk-warmup). Streaming features = stateful classes mirroring the
Rust online state; batch features = pure transforms. numpy/numba/polars for hot paths, with the tolerance
test as the safety net.

## Feature-port roadmap (from the ml-features inventory — alpha vector is 134-dim)

- **Tier 1 (Low, pure, rich tests):** ✅ microprice (Block N+V — DONE) · frac-diff (M) · return-moments
  (AA+EE) · trend-scanning (X) · OFI stateless per-level fns (F/G/H/CC) · bouchaud (BB) · order-events (W).
- **Tier 2 (Medium, streaming, exact loop order):** spread-decomposition (O) · multi-level Kyle-λ + VPIN
  (T/U) · LOB-PCA Sanger (Y) · price/volume/statistical (A/B/C) · ADX/regime-ADX Wilder (D/D') · time +
  seasonality (E/DD).
- **Tier 3 (High, optimization):** Hawkes-MLE grid search (R) · PIN EM-MLE (Z).
- **Tier 4 (assembly):** imbalance/volume bar samplers + tick-rule · 134-dim alpha + 81-dim snapshot
  pipelines.
- **Defer/skip:** legacy 65-dim pipeline, config.rs Wave A-D, position_features (RL state), DBN readers.
- **Prioritized by conservation:** the order-flow blocks (OFI/VPIN/book-deltas/Kyle-λ) are the
  follow-the-money signals → bring them forward within their tiers.

## Open optimization (verified)

The Rust "Stoikov microprice" is the *weighted mid-price*. Stoikov's true micro-price (Quant Finance
2018) is a recursive Markov-chain estimator and empirically a better future-price predictor. ADD it as a
separate `true_microprice` feature (not a silent oracle divergence) so the gauntlet A/Bs them empirically.

## Build order

1. ✅ Feature-warehouse foundation (silver marketdata + gold FeatureStore + pipeline + microprice TDD).
2. Port Tier-1 features (flow first), feeding the warehouse from the real MBP-10/trades data we hold.
3. Research/Trader split formalized; analyst-agent query tools over the warehouse.
4. Domain-specialist analyst agents → grounded hypotheses → gauntlet.
5. Conservation/reconciliation layer in the trading engine (every cent accounted for).
6. Falsification analysts; true micro-price + advanced features.
