# Real-LOB Integration Design Spec

**Status:** DESIGN — approved through brainstorming, awaiting user spec-review before plan handoff
**Date:** 2026-05-18
**Branch:** `ml-alpha-phase-a`
**Owner:** ml-alpha team

## 0. Context

Phase 1+2+3 architecture (VSN → Mamba2-l1 → LN_a → Mamba2-l2 → LN_b → attention-pool → CfC + GRN heads) at commit `73925b15d` hit 3-fold CV `mean_auc = 0.7749 ± 0.024` and h6000 AUC `0.7591 ± 0.018` on ES MBP-10. The Phase B K-loop block-per-batch refactor (commit chain landed earlier today) gave ~17 s/epoch on the L40S — ~28× wall speedup over the per-K serial launch pattern.

Per `pearl_phase1d4_backtest_cost_edge_frontier` (2026-05-15): frictionless annualised Sharpe ~4.4 (78% hit rate); break-even cost ~quarter-tick (0.0625 price units); half-tick (0.125) kills it; retail 1-tick deeply unprofitable. **Signal is real; deployment hinges on execution venue.**

This spec turns the AUC predictor into a P&L producer by building a realistic limit-order-book simulator + decision policy + execution layer that consumes `p_h(t)` and emits trades with proper accounting of fees, spread crossing, slippage, queue position, and latency.

Out of scope (separate specs):
- Per-horizon attention-pool variants (Axis 1 follow-on, model-side spec).
- Sweep tool / executable (Axis 2, deferred per user prioritization).
- Live IBKR FIX/REST adapter (production deployment, separate spec).
- Multi-instrument hedging / basket strategies.
- Market-impact modelling of our own order flow (acceptable for 1–10 lot ES sizes; revisit if size grows).

## 1. Crate layout + trainer parity

**No new library crate.** The LOB sim + policy + harness live inside the existing `crates/ml-backtesting/` crate alongside the barrier-backtest code — both are backtest tools, they belong together. **The backtest harness is a thin orchestrator on top of the trainer's existing data pipeline + inference graph** — it does NOT reimplement loading, feature assembly, or model forward.

| Crate | New modules added | Existing contents (unchanged) |
|---|---|---|
| `crates/ml-backtesting/` | `lob/` (CUDA kernels — pre-compiled cubins per `feedback_no_nvrtc.md`), `policy/` (`Strategy` tree, `StopRules`, `LatencyConfig`, `SizingPolicyId`), `harness.rs` (`BacktestHarness` driving loader → trunk → sim chain), `sim.rs` (`LobSimCuda` device + mapped-pinned buffer owner), `order.rs` (`OrderIntent`, `OrderEvent`, `SlotTag`), `artifacts.rs`, `aggregate.rs`. New deps: `cudarc`, `ml-alpha` (for loader + trunk reuse). | `barrier_backtest.rs`, `action_loader.rs`, `report.rs` |

| Binary | Purpose |
|---|---|
| `bin/fxt-backtest` | New CLI with two subcommands: `fxt-backtest run --model <ckpt> --data <mbp10.dbn> --n-parallel <N> --policy-grid <config.yaml> --decision-stride <S> --out <results/>` and `fxt-backtest aggregate <sweep_dir>`. One binary, fewer crate-count, single entry point. |

**Reused from `crates/ml-alpha/`** (no duplication — zero train-vs-deploy skew possible):

| ml-alpha component | Used as |
|---|---|
| `data::loader::MultiHorizonLoader` | MBP-10 file discovery + chronological streaming. Add an `inference_only: bool` flag that bypasses label packing. Decision-stride field already exists (Task #182 landed Phase 2A.1). |
| `cfc::snap_features::Mbp10RawInput` | Single struct drives both **LOB book update** (uses `bid_px/sz`, `ask_px/sz`, `ts_ns`) and **model inference** (passed to `update_input_buffers`). One source of truth. |
| `cfc::snap_features::snap_feature_assemble` CUDA kernel | Same cubin invoked inside the captured graph. No separate inference feature path. |
| `cfc::trunk::CfcTrunk::capture_graph_a` | Captures the full perception forward (VSN → Mamba2-l1 → LN_a → Mamba2-l2 → LN_b → attn_pool → CfC → heads). Captured once at backtest start. |
| `cfc::trunk::CfcTrunk::update_input_buffers` + `perception_forward_captured` | Per-decision: mapped-pinned input write, graph replay, returns `[f32; N_HORIZONS]` + `[f32; PROJ_DIM]`. Stateful — Mamba2 SSM state persists across calls. |

The **only new code** is the LOB sim itself and the orchestration that drives the loader → captured-graph → sim chain. There is no new data loader, no new feature pipeline, no new model surface.

No CPU variant. Single CUDA code path runs on RTX 3050 (4 GB, dev, N ≤ 16 backtests, ≤ 1 hr data) and L40S (48 GB, production, N = 256+ backtests). Per `feedback_no_cpu_test_fallbacks.md`, validation uses GPU-oracle tests against fixture sequences. Custom kernels (LOB sim is an event-driven state machine — cuBLAS is not the right hammer).

## 2. CUDA data layout

Per-block shared memory state (one block = one parallel backtest):

```cuda
__shared__ struct {
    float bid_px[10];   // sorted high → low (MBP-10 invariant)
    float bid_sz[10];
    float ask_px[10];   // sorted low → high
    float ask_sz[10];
} book;                  // 160 B

__shared__ struct {
    struct {
        uint8_t  active;          // 0=free, 1=resting, 2=in-flight (latency)
        uint8_t  side;            // 0=buy, 1=sell
        uint8_t  type;            // LIMIT|IOC|FOK
        uint8_t  oco_pair;        // 0xFF = no pair, else paired-slot index
        float    price;
        float    size_remaining;
        float    queue_position;  // contracts ahead of us at this level
    } limits[32];                  // 768 B
    struct {
        uint8_t  active;
        uint8_t  side;
        uint8_t  type;            // STOP_MARKET|STOP_LIMIT
        uint8_t  oco_pair;
        float    trigger_price;
        float    limit_price;     // STOP_LIMIT only; ignored for STOP_MARKET
        float    size;
    } stops[16];                   // 256 B
} orders;

__shared__ struct {
    int   position_lots;          // signed
    float vwap_entry;             // running entry-price VWAP for current position
    float realized_pnl;
    float peak_equity;
    uint32_t submission_overflow; // diagnostic counter
    uint32_t open_horizon_mask;   // bit h set if current position was authorized by horizon h
} pos;                             // ~24 B

#define N_HORIZONS 5  // matches crates/ml-alpha/src/heads.rs N_HORIZONS

__shared__ struct {
    float pnl_ema_win;            // Wiener-α blended
    float pnl_ema_loss;
    float win_rate_ema;
    uint32_t n_trades_seen;
    float realised_return_var;    // Welford running variance
    float recent_sharpe;           // composite for WeightedByRealizedSharpe aggregator
} isv_kelly[N_HORIZONS];           // 24 B × 5 = 120 B (per-horizon state, see Section 5)
```

Total per-block shared mem: ~1.4 KB. Default 48 KB / block budget is plenty; no `cudaFuncSetAttribute` opt-in needed. Both RTX 3050 and L40S handle this trivially.

Grid layout: `grid_dim = (N_backtests, 1, 1)`, `block_dim = (32, 1, 1)` — one warp per backtest. The 32-thread choice is profile-driven (per Open Q3 resolution); promotion to 64 threads (two warps) is a measurement-driven decision at implementation time.

## 3. Order types — flat `OrderIntent` representation

```rust
// Host side: ergonomic enum for config + audit log.
pub enum OrderIntent {
    SubmitMarket { side: Side, size: u16 },
    SubmitLimit  { side: Side, size: u16, px: i32, kind: LimitKind },  // LIMIT | IOC | FOK
    SubmitStop   { side: Side, size: u16, trigger_px: i32, limit_px: Option<i32> },
    SubmitOcoLimits { leg_a: LimitParams, leg_b: LimitParams },         // both legs are limits
    Cancel       { slot_tag: SlotTag },
    Modify       { slot_tag: SlotTag, new_size: Option<u16>, new_px: Option<i32> },
}

pub struct LimitParams { pub side: Side, pub size: u16, pub px: i32, pub kind: LimitKind }
pub enum LimitKind { Limit, Ioc, Fok }
pub enum Side { Buy, Sell }
```

Price is `i32` ticks (ES tick = 0.25 price units = $12.50 per contract). Size is `u16` contracts. `SlotTag` is `{slot_kind: u8, slot_index: u8, gen_counter: u16}` packed into a `u32` so cancel/modify is robust against intervening fills + slot reuse.

**Device-side**: the policy and matching kernels do NOT consume `OrderIntent`. Decisions stay on-device — the policy kernel writes directly into the per-block shared-mem `orders.limits[]` / `orders.stops[]` state and sets the `oco_pair` link byte for OCO. No host bounce in the decision path. The host-side `OrderIntent` enum is the **audit representation**: when the policy kernel commits an order, it also writes a flat `OrderEvent { ts_ns, slot_tag, kind: u8, side: u8, size: u16, px: i32 }` record (24 B) into a per-block **mapped-pinned audit ring**. The host reads this ring on `flush_trade_log` (Section 7) and reconstructs `OrderIntent` for the trade log + audit.parquet.

This means OCO doesn't need a recursive on-device representation: it's two separate slots (limits[] or stops[]) linked by `oco_pair`, with the audit ring emitting two `OrderEvent` records sharing the same `gen_counter`.

## 4. Latency model

```rust
pub enum LatencyConfig {
    Constant   { total_ns: u32 },                // e.g. 100_000_000 = 100 ms IBKR + Scaleway baseline
    Empirical  { samples_d: *const f32, n_samples: u32 },  // sampled per-decision
    Decomposed { inference_ns: u32, decision_ns: u32, wire_ns: u32, match_ns: u32 },
}
```

Default: `Constant { total_ns: 100_000_000 }` reflecting IBKR via Scaleway hosting + Databento data feed roundtrip. Empirical mode draws from a samples array (host pre-populates from production telemetry once available). Decomposed mode lets us attribute latency budget across pipeline stages.

Mechanics: every `OrderIntent` carries an arrival-event timestamp = `submit_event_ts + latency_ns`. The matching kernel ignores in-flight orders (`active == 2`) until the event timestamp reaches arrival; then transitions to `active == 1` (resting) and applies marketability check at *that* event's book state. This is what makes `latency_in_flight_miss` (Section 8 fixture) work: prices can move during the in-flight window and the order arrives stale.

**Horizon viability** — model emits all 5, policy chooses which to trade on.

The trained Phase 1+2+3 model already emits all 5 horizons per forward pass (`N_HORIZONS = 5; HORIZONS = [30, 100, 300, 1000, 6000]` in `crates/ml-alpha/src/heads.rs`). No retraining is needed to enable any of them in the policy. Snapshot-rate anchor from `pearl_phase1d4_backtest_cost_edge_frontier`: K=6000 ≈ 2.4 hr, so ≈ 0.7 snapshots/sec (one snapshot per ~1.4 sec). Converting:

| Horizon | Holding | Validation status | Default |
|---|---|---|---|
| h30 | ~43 sec | Not yet backtested. Could compound short alpha into many small trades; could also be friction-killed. Stateful Mamba2 expected to help per `pearl_state_amplifies_short_horizon_into_long_horizon`. | OFF |
| h100 | ~2.4 min | Same as h30. | OFF |
| h300 | ~7.2 min | Same. | OFF |
| h1000 | ~24 min | Same. | OFF |
| h6000 | ~2.4 hr | **Validated** at Phase 1d.4: frictionless Sharpe 4.4, quarter-tick Sharpe 0.7, half-tick negative. | **ON** |
| h9000 | ~3.6 hr | Out of scope for v1 (`N_HORIZONS 5→6` requires retrain). | N/A |

**Why short horizons are "untested" not "infeasible":** Latency (100 ms) is not binding — even h30 has ~400× more holding time than the latency budget. The binding constraint is **per-trade alpha vs per-trade friction**. Shorter horizons have smaller alpha-per-trade but more trades per unit time, so the net economics are a sweep question, not a derivable bound. Per `pearl_horizon_decay_falsifies_long_horizon_stateless`, stateless features fail past K≈20-50; per `pearl_state_amplifies_short_horizon_into_long_horizon`, Mamba2 SSM state lifts AUC 0.50→0.66 at K=6000 (12× the stateless horizon). Whether the same amplification helps the short horizons enough to clear friction is exactly what a per-horizon backtest sweep would answer.

**Implication:** Static horizon-masking is the wrong abstraction. Instead, each horizon is a leaf strategy and the composition aggregator `WeightedByRealizedSharpe` auto-shifts capital toward whichever horizons are empirically earning their per-trade friction cost (Section 6). A per-horizon cost-frontier diagnostic sweep — running with one horizon enabled at a time across the cost grid — still has value as an offline analysis tool and is a natural downstream consumer of the deferred sweep workstream (`#201` / `#202`); it just isn't needed for production policy enablement.

## 5. Per-horizon ISV-Kelly sizing + adaptive horizon weighting

Per `pearl_kelly_cap_signal_driven_floors` and `feedback_isv_for_adaptive_bounds`: all sizing bounds derive from in-sample-variance signals. **Per-horizon state** — each horizon is its own ISV-Kelly tracker, and the policy auto-shifts capital toward whichever horizons are empirically paying off (answers user's "isn't the policy shifting by itself?" — yes, by design).

```rust
// Per-horizon, per-backtest state. Lives in GPU shared memory (Section 5b).
pub struct IsvKellyState {
    pub pnl_ema_win:         f32,   // EMA of winning-trade returns (Wiener-α blended)
    pub pnl_ema_loss:        f32,   // EMA of losing-trade returns (positive magnitude)
    pub win_rate_ema:        f32,
    pub n_trades_seen:       u32,
    pub realised_return_var: f32,   // Welford running variance over realized per-trade returns
    pub recent_sharpe:       f32,   // pnl_ema_win × win_rate − pnl_ema_loss × (1−win_rate), normalised by sqrt(var)
}
```

**Per-decision sizing per horizon h:**

```
sig_mag[h]    = |p_h − 0.5| × 2                                           // ∈ [0, 1]
kelly_frac[h] = (win_rate[h] × pnl_ema_win[h] − (1 − win_rate[h]) × pnl_ema_loss[h]) / pnl_ema_win[h]
              = clamp(kelly_frac[h], 0, 1)
size_units[h] = sig_mag[h] × kelly_frac[h] × cap_units[h]
cap_units[h]  = target_annual_vol_units / sqrt(realised_return_var[h] × annualisation_factor[h])
```

No floor. The cap collapses to zero when realized variance is unknown/large; sizing collapses to zero when alpha magnitude is small; nothing prevents zero trading in a bad regime. This is correct behaviour — the previous "kelly_floor = max(1, cap/4)" was the exact hardcoded-engagement-floor anti-pattern called out in `pearl_blend_formulas_must_have_permanent_floor` (the floor formula should derive from signals, not from a constant).

**Adaptive horizon weighting** (composition mode `Ensemble { aggregator: WeightedByRealizedSharpe }`):

```
weight[h] = max(0, recent_sharpe[h]) / sum_h(max(0, recent_sharpe[h]))    // softmax-with-floor-zero
final_size = sum_h( weight[h] × signed_size_units[h] )                     // signed: + long, − short
```

If `recent_sharpe[h]` is negative across all horizons, total weight is zero → no trading. If only h6000 has positive Sharpe, it gets all the weight. If h100 + h6000 both have positive Sharpe, they compose proportionally. Short horizons auto-promote if they prove profitable; they auto-suppress if they don't.

**Welford updates**: `realised_return_var[h]` rolls forward per-horizon on segment_complete events from the LOB sim, indexed by the horizon that authorized the trade. EMA wins/losses use Wiener-optimal α with the `pearl_wiener_alpha_floor_for_nonstationary` 0.4 floor (per-horizon P&L is non-stationary).

### Honest constants list

| Constant | Value | Justification |
|---|---|---|
| `target_annual_vol_units` | User config | Risk-preference input, NOT a signal. User specifies their tolerance; the spec ships with no default — explicit in policy YAML. |
| Wiener-α floor | 0.4 | Documented in `pearl_wiener_alpha_floor_for_nonstationary` — analytical lower bound for non-stationary control loops. Not arbitrary. |
| `annualisation_factor[h]` | Derived | Computed from trade rate per horizon at runtime, not hardcoded. |
| Bootstrap threshold | None | First observation replaces directly (per `pearl_first_observation_bootstrap`). `n_trades_seen` is used as a divisor-safety check (avoid div-by-zero on first trade), not as a "wait N trades before trading" gate. |
| Min sizing-units | 0 | Explicit zero. No "always trade at least 1 lot" floor. |

Everything adaptive is genuinely adaptive; everything fixed is honestly fixed and documented in a pearl.

## 5b. State ownership and lifetime

| State | Where it lives | Lifetime | Mutation pattern |
|---|---|---|---|
| Mamba2 SSM hidden state | Device, inside captured graph A buffers (`CfcTrunk` instance) | One per process (single model) | Updated by `perception_forward_captured()` graph replay |
| LOB book (10 levels bid/ask) | Per-block shared memory (`book` struct, ~160 B) | Per-backtest, lifetime of run | Mutated by `book_update.cu` on every event |
| Resting orders + stops | Per-block shared memory (`orders` struct, ~1 KB) | Per-backtest, lifetime of run | Mutated by decision-policy kernel (submit) + matching kernel (fill/cancel) |
| Position + P&L accounting | Per-block shared memory (`pos` struct, ~20 B) | Per-backtest, lifetime of run | Mutated by matching kernel on fills |
| **Per-horizon ISV-Kelly state** | **Per-block shared memory, array of `N_HORIZONS=5` `IsvKellyState`** (~120 B per backtest) | Per-backtest, lifetime of run | Mutated on segment_complete events; read by decision-policy kernel |
| Strategy composition program | Device-global memory, indexed by `blockIdx.x` (NOT `__constant__`) | Per-backtest, set once at construction | Read-only after construction |
| Audit ring (OrderEvent records) | Per-block mapped-pinned host buffer | Per-backtest, lifetime of run | Kernel writes; host reads via `flush_trade_log` |
| Trade log (closed trades) | Per-block mapped-pinned host buffer | Per-backtest, lifetime of run | Kernel writes; host reads at chunk boundaries |

**No host-side state for ISV-Kelly or Mamba2.** The `ml_backtesting::policy` submodule is config-only types. The `BacktestHarness` host-side struct owns: the loader, the trunk (single instance, owns Mamba2 state), the `LobSimCuda` handle (owns per-backtest device + mapped-pinned buffers), and artifact writers. Nothing else.

## 6. Multi-strategy composition

```rust
// Recursive composition tree — supports nesting (RegimeSwitch → Portfolio → Ensemble of leaves).
pub enum Strategy {
    Leaf(StrategyConfig),
    Ensemble     { children: Vec<Strategy>, aggregator: EnsembleAggregator },
    Portfolio    { children: Vec<Strategy>, capital_allocation: Vec<f32>, conflict_policy: ConflictPolicy },
    RegimeSwitch { regime_to_child: HashMap<Regime, Strategy> },
}

pub struct StrategyConfig {
    pub horizon_idx:         u8,                    // exactly one horizon per leaf (0..N_HORIZONS)
    pub sizing_policy:       SizingPolicyId,        // IsvKelly (default) | FixedLots
    pub sl_tp_rules:         StopRules,
    pub max_concurrent_lots: u16,
}

pub enum EnsembleAggregator {
    Mean,                       // equal-weight average of signed sizes
    MaxConfidence,              // pick whichever child has largest |sig_mag × kelly_frac|
    WeightedByValAuc,           // static weights from training-time per-horizon val AUC
    WeightedByRealizedSharpe,   // ADAPTIVE — weights = max(0, recent_sharpe[h]) / Σ; auto-shifts in-run
}

pub enum ConflictPolicy { NetOut, FirstSubmitWins, BlockOpposing }
pub enum Regime         { SpreadQ1, SpreadQ2, SpreadQ3, SpreadQ4, /* extensible */ }
```

**Default `Strategy` constructor** — one leaf per horizon, composed with the adaptive aggregator. No static horizon mask. The policy figures out which horizons earn their place from in-run evidence:

```rust
pub fn default_strategy(max_lots: u16) -> Strategy {
    let children = (0..N_HORIZONS as u8).map(|h| Strategy::Leaf(StrategyConfig {
        horizon_idx:         h,
        sizing_policy:       SizingPolicyId::IsvKelly,
        sl_tp_rules:         StopRules::default(),
        max_concurrent_lots: max_lots,
    })).collect();
    Strategy::Ensemble { children, aggregator: EnsembleAggregator::WeightedByRealizedSharpe }
}
```

At t=0 all per-horizon `recent_sharpe` are at sentinel (Wiener-α first-observation bootstrap). As trades close, the per-horizon ISV-Kelly state of Section 5 accumulates evidence; horizons with positive realized Sharpe rise in weight, negative-Sharpe horizons collapse to zero. The validated h6000 will ramp up first because it has the strongest expected per-trade alpha; untested horizons get a fair shot and either prove themselves or self-suppress. **This replaces "ship with h6000 ON, others OFF" — there is no horizon mask to set.**

Per-horizon warm-start (optional, behind `--warm-start <path>`): load saved `IsvKellyState` arrays from a prior backtest. Lets v1 ship with h6000 state pre-seeded from Phase 1d.4 if desired; absent the flag, everything cold-starts.

The recursive `Strategy` tree lets composition modes nest — e.g. `RegimeSwitch { Q4 -> Ensemble{horizons}, Q1 -> Leaf{h6000_only} }` — for users who want regime-conditional behaviour on top of horizon adaptivity.

**Host-side flattening**: before kernel launch, the orchestrator walks the `Strategy` tree per backtest and emits a flat **decision program** (a small bytecode-ish array of ops: `EVAL_REGIME`, `EMIT_PER_HORIZON_SIZE`, `AGG_WEIGHTED_SHARPE`, `APPLY_CONFLICT`, `WRITE_ORDER`, etc.). The flat program is written to **device-global memory** at slot `program_table[blockIdx.x]`.

**Why not `__constant__`**: `__constant__` is 64 KB **per device**, broadcast-identical to all threads. With N=256 backtests each with own composition we'd blow the budget and defeat the purpose. Device-global memory is N-indexed by `blockIdx.x` with coalesced reads (each warp reads the same offset = perfect locality).

Composition resolution happens host-side at backtest construction; the policy kernel just interprets the bytecode against current alpha probs + per-horizon ISV-Kelly state to mutate the shared-mem `orders` state.

## 7. Backtest harness + output artifacts

### Rust orchestrator (`bin/fxt-backtest`)

Built directly on the trainer's existing surface — no re-implemented loaders, no re-implemented feature pipeline.

**Reused trainer surface:**
- `MultiHorizonLoader::new(cfg)` with `inference_only: true` flag (new — add to `MultiHorizonLoaderConfig`) yields chronological `Mbp10RawInput` items without label packing.
- `CfcTrunk::capture_graph_a(&Mbp10RawInput)` — one-time capture of the full perception forward (VSN → Mamba2-l1 → LN_a → Mamba2-l2 → LN_b → attn_pool → CfC → heads).
- `CfcTrunk::update_input_buffers(&Mbp10RawInput)` — mapped-pinned write of the next input into the capture's bound buffers.
- `CfcTrunk::perception_forward_captured() -> ([f32; N_HORIZONS], [f32; PROJ_DIM])` — graph replay; Mamba2 SSM state persists across calls (stateful inference is the whole point per `pearl_state_amplifies_short_horizon_into_long_horizon`).

```rust
fn run_backtest(args: &Args) -> Result<()> {
    let loader_cfg  = MultiHorizonLoaderConfig {
        data_root:        args.data.clone(),
        inference_only:   true,          // NEW — skip label packing for backtest
        decision_stride:  args.decision_stride.unwrap_or(4),
        seq_len:          1,             // streaming one snapshot at a time
        // ... rest from defaults matching training config
    };
    let mut loader  = MultiHorizonLoader::new(&loader_cfg)?;
    let mut trunk   = CfcTrunk::load_checkpoint(&args.model)?;
    let policy_grid = PolicyGrid::from_yaml(&args.policy_grid)?;
    let mut sim     = LobSimCuda::new(args.n_parallel, &policy_grid)?;
    if let Some(p) = &args.warm_start { sim.load_isv_kelly_state(p)?; }

    // One-time graph captures using the first input as a template.
    let first = loader.peek_first()?;        // NEW lightweight peek; uses same Mbp10RawInput
    trunk.capture_graph_a(&first)?;          // perception forward graph
    sim.capture_step_event_graph(&first)?;   // per-event book-update + match graph (captured once,
                                             //   replayed per event; host branches only between captures)

    let mut event_idx = 0u64;
    while let Some(raw) = loader.next_inference_input()? {
        if event_idx % loader_cfg.decision_stride as u64 == 0 {
            trunk.update_input_buffers(&raw)?;
            let (probs, _proj) = trunk.perception_forward_captured()?;
            sim.broadcast_alpha(&probs)?;            // device-global write
            sim.replay_decision_graph()?;            // policy kernel decides per-block from broadcast
        }
        sim.replay_step_event_graph(&raw)?;          // book update + match (uses raw.bid_px/sz directly)
        event_idx += 1;
        if event_idx % args.flush_every == 0 {
            sim.flush_trade_log(&args.out)?;         // mapped-pinned read; no drain thread
        }
    }
    sim.flush_trade_log(&args.out)?;
    sim.write_summary(&args.out)?;
    Ok(())
}
```

### CUDA Graph capture granularity (resolves I9)

**Three separate captured graphs**, each replayed many times. Host-side branches between graph replays are fine; no host branch inside any capture (per `pearl_no_host_branches_in_captured_graph`).

| Graph | Captures | Replay rate |
|---|---|---|
| Trunk perception (existing) | snap_feature_assemble → VSN → Mamba2-l1 → LN_a → Mamba2-l2 → LN_b → attn_pool → CfC → heads | Once per decision point (rate = events / decision_stride) |
| LOB step-event | book_update → match_resting_orders → check_stop_triggers → update_pos_pnl | Once per event |
| LOB decision | apply_program → per_horizon_size → aggregate_weighted_sharpe → conflict_resolve → emit_orders → emit_audit | Once per decision point (only when broadcast_alpha just wrote new probs) |

The orchestrator's `if event_idx % decision_stride == 0` is host-side and lives BETWEEN graph replays — not inside any captured region. Per `pearl_cudarc_disable_event_tracking_for_graph_capture`, all three captures bracket with event-tracking disable/enable.

### Inference scope (resolves I7)

**v1 supports policy sweeps only** (single model checkpoint, multiple per-backtest sizing/composition configs). One Mamba2 forward per decision point, broadcast to all N backtests. Inference cost is O(N_events / decision_stride), independent of N_backtests.

**Model-checkpoint sweeps (v2)** would require N forwards per decision point (one per checkpoint) and divergent SSM state per checkpoint. Out of scope here; a separate spec must address inference batching across checkpoints.

### Trainer parity contract

Because the backtest reuses `MultiHorizonLoader`, `Mbp10RawInput`, `snap_feature_assemble`, and the captured trunk graph **verbatim** (no copies, no re-implementations), the inference output stream during backtest is bit-equivalent to what training-time inference on the same data would produce. **Train-vs-deploy skew is structurally impossible.** This is enforced by a Ring 1b validation test (Section 8) that asserts bit-equality of `[probs; N_HORIZONS]` between a training-time forward and a backtest-time forward on the same first input.

### Output artifacts (per backtest cell)

| File | Format | Contents |
|---|---|---|
| `summary.json` | JSON | Top-level stats: total_pnl, sharpe_ann, sortino_ann, max_drawdown, calmar, n_trades, win_rate, avg_win, avg_loss, profit_factor, total_fees, exposure_pct, kelly_cap_history (downsampled) |
| `trades.csv` | CSV | Per-trade audit: `entry_ts, exit_ts, side, size_lots, entry_px_ticks, exit_px_ticks, fees_usd, realized_pnl_usd, strategy_id, horizon, alpha_logit_at_decision` |
| `pnl_curve.bin` | f32 array | Cumulative P&L sampled at every event timestamp (binary, mmap-friendly) |
| `audit.parquet` | Parquet | Optional behind `--audit` flag: per-decision-point full state dump (book snapshot, all alpha logits, kelly state, all active orders). Large; only enable for debugging. |

### Sweep aggregation

`fxt-backtest aggregate <sweep_dir>` walks all cells, parses each `summary.json`, emits a single `aggregate.parquet` with one row per cell + a top-level `pareto_frontier.json` highlighting Pareto-optimal (sharpe, drawdown, fees) cells.

## 8. Validation strategy

Three concentric rings.

### Ring 1: Unit-level fixture tests (N=1, bit-exact)

Single-backtest (`grid_dim = (1, 1, 1)`) synthetic MBP-10 event sequences with known expected outcomes. Asserted bit-exactly on GPU output — at N=1 there's no cross-block reduction order to worry about and each block writes only to its own slot, so floating-point output is deterministic across runs. One fixture per mechanism, loaded from JSON by `tests/lob_sim_fixtures.rs`.

| Fixture | Tests | Asserted output |
|---|---|---|
| `book_update_add` | Add at level k → `bid_sz[k] += size`, other levels unchanged | Book state byte-for-byte |
| `book_update_cancel_partial` | Cancel reduces level size; if size→0, level removed, deeper levels shift up | Level-shift mechanics |
| `book_update_trade_top` | Trade at best bid → trade-side liquidity consumed; level shift if emptied | Top-of-book mutation |
| `market_order_consumes_top` | Market buy size 5; ask[0]=3, ask[1]=10 → fills 3@ask[0] + 2@ask[1] | Walking-levels math |
| `limit_rest_queue_init` | Submit limit buy at P; bid_sz[level]=10 prior → our `queue_position=10` | Queue init |
| `limit_fill_via_trade` | Resting buy at 5500.00, qpos=10; trade consumes 12 size @ 5500.00 → 10 evicted then 2 fills us | Queue-decay math |
| `stop_trigger` | Long stop trigger=5510.00; best ask 5509.75 → 5510.25 → stop triggers, market buy emitted | Trigger semantics |
| `oco_one_cancels_other` | OCO {limit buy 5500, limit sell 5510}; trade at 5510 fills sell → buy auto-cancelled | OCO link |
| `latency_in_flight_miss` | Submit limit @ 5500 at event 1000, latency 100 ms; events 1001-1500 = price drops to 5495 → on arrival, limit is marketable, crosses at 5495 | Latency-aware fills |
| `pnl_accounting` | Sequence of fills with known prices → realized P&L matches hand-computed value to FP tolerance | Cost-model math |
| `submission_overflow` | Submit 33 limits when slots=32 → 32 accepted, 33rd increments `submission_overflow` | Slot accounting |

~15–20 fixtures total. Run on RTX 3050 locally + L40S in CI.

### Ring 1b: Trainer-parity bit-equality check

A single test that captures the trunk graph, runs `perception_forward_captured` on a fixed seed `Mbp10RawInput`, and asserts the returned `[probs; N_HORIZONS]` is bit-equal between:
- Backtest harness path: `MultiHorizonLoader` with `inference_only=true` → trunk inference.
- Training path: `MultiHorizonLoader` with normal training config → trunk inference at the same loader position.

If they ever diverge, train-vs-deploy skew has snuck in (a reimplementation of feature assembly somewhere, or a config divergence in the loader). The test guards against any future refactor breaking the structural parity claim of Section 7.

### Ring 2: Property-based fuzz invariants (N>1, invariants only)

Multi-backtest (`grid_dim = (N, 1, 1)` with N ∈ {1, 16, 256}) random event sequences + random orders. **Not** bit-exact across runs because nondeterminism in block scheduling order is allowed; we assert only the invariants that must hold regardless of input or scheduling:

- **Book monotonicity:** `bid_px[0] ≥ bid_px[1] ≥ ...`; `ask_px[0] ≤ ask_px[1] ≤ ...`
- **No-crossed-book:** `ask_px[0] > bid_px[0]` always.
- **Position accounting:** `Σ entry_size − Σ exit_size = position_lots`.
- **P&L conservation:** `realized_pnl = Σ (exit_px − entry_px) × side × lots × $50 − Σ commissions`.
- **Order-slot stewardship:** `limits[i].active == 1 ⟺ i ∈ active_set`; no leaked slots after cancel/fill.
- **OCO mutual exclusion:** at most one leg of any OCO pair ever has active fills.
- **`submission_overflow` monotone non-decreasing.**

100 random sequences × 1000 random orders each = 100K fuzz cases. Failures get the seed for reproduction.

### Ring 3: Production-data cross-validation

After Rings 1 + 2 pass:

1. **Quiet day replay** (e.g. ES 2025-Q1 calm session): trivial policy ("buy 1 lot at open, hold to close, sell"). Sim P&L vs hand-computed `(close_mid − open_mid) × $50 − 2 × commission` must agree within ±2 ticks.
2. **Fast-market day replay** (e.g. FOMC announcement): same trivial policy; assert wider slippage is consistent with hand-recomputed walked-book P&L.
3. **(Future)** Paper-trading IBKR fill replay: feed the policy that produced actual paper-traded orders; compare sim fill prices to IBKR-reported fills within ~1 tick. This is the gold-standard validation but requires a paper-trading data collection step.

### Acknowledged validation limitations

- **Own-order market impact** not modelled. Exogenous-event assumption. OK for 1–10 lot ES sizes (book depth ~100s at touch); revisit if size grows.
- **Off-screen/iceberg liquidity** invisible to MBP-10; sim assumes book-depth-as-shown is the only liquidity.
- **Cross-instrument hedging** out of scope (single-instrument ES).

## 9. Risks + resolved open questions

### Risks

| Risk | Severity | Mitigation |
|---|---|---|
| Queue-decay model assumes pessimistic baseline (all cancels ahead) → understates fills | Medium | Documented; function-pointer hook for swap-in; empirical calibration follow-up against IBKR paper-trading data |
| Shared-mem overflow if > 32 limits / > 16 stops per backtest | Low | Current limits cover ~3× normal usage. If exceeded, opt into 96 KB via `cudaFuncSetAttribute` (both target GPUs support). |
| RTX 3050 4 GB budget restricts local dev to N≤16 + ~1 hr data | Low | Documented dev workflow; production sweeps unaffected on L40S 48 GB. |
| No CPU oracle for validation | Low | Resolved: fixture-based bit-exact GPU tests; no CPU reference per `feedback_no_cpu_test_fallbacks.md` |
| CUDA Graph capture incompatibility with `cudaMallocAsync` | Medium | Pre-allocate ALL scratch at construction (same pattern as ml-alpha trainer). No `cudaMallocAsync` in captured region. Per `pearl_cudarc_disable_event_tracking_for_graph_capture`: bracket capture with event-tracking disable/enable. |
| Decision policy single-thread-per-block bottleneck if `max_strategies > ~8` | Low | At ~50 ops × 8 strategies = 400 ops/decision, dwarfed by event-processing cost. Parallelize across strategies within block if it becomes hot. |
| Market-impact not modelled | Acknowledged | Fit-for-purpose for 1–10 lot ES sizes. Larger size = follow-up spec. |
| IBKR live integration out of scope | Separate concern | This spec stops at "simulated P&L matches realistic execution at IBKR + Scaleway latency profile". Live adapter is a separate spec. |
| ES contract roll dates not handled | Medium | ES futures roll quarterly; the continuous-contract assumption breaks at roll boundaries. Host-side preprocessing concern — the orchestrator must either skip the rolled-out contract's last hours + new contract's first hours, or use a back-adjusted continuous series. **Acknowledge in policy YAML schema**: each backtest cell declares `roll_handling: SkipBoundary { hours: u8 } | BackAdjusted | NoneAssumeContinuous`. v1 default = `SkipBoundary { hours: 4 }`. |
| Session boundaries (RTH vs ETH) | Medium | Per-snapshot regime features include session indicators; the policy can gate on them. Sim itself is session-agnostic. Document the assumption that the input MBP-10 stream respects whatever session-filter the user wants (orchestrator passes through). |
| MBP-10 schema version drift | Low | `Mbp10RawInput` is owned by `crates/ml-alpha/cfc/snap_features.rs`; any schema change propagates automatically through the trainer-parity reuse. Sim consumes only the bid/ask level fields, which are stable Databento contract. |
| Adaptive-horizon cold-start eats early backtest performance | Medium | At t=0 all per-horizon `recent_sharpe` is at sentinel; the aggregator emits zero size until per-horizon trades close and ISV-Kelly state stabilizes. For short backtests this is meaningful drag. Mitigation: `--warm-start <isv_kelly_state.bin>` flag pre-loads per-horizon state from a prior backtest. v1 ships with an h6000 warm-start file derived from Phase 1d.4 results. |

### Resolved open questions (user-confirmed)

| Q | Resolution |
|---|---|
| 1. Event chunk size for streaming | **Streaming with chunked mapped-pinned upload.** Start with 100K events/chunk; profile and adjust. |
| 2. Decision-stride alignment | **Match training (stride=4 default).** Flag to override. |
| 3. Per-block thread count | **32 threads (one warp) default.** Profile-driven adjustment in implementation. |
| 4. Trade-log persistence | **Mapped-pinned host buffer (hard requirement).** Kernel writes per-trade records directly to mapped-pinned memory via zero-copy device pointer. No ring buffer, no host reader thread, no async copies. Consistent with `feedback_no_htod_htoh_only_mapped_pinned.md`. |

## 10. Compliance with HARD memory rules

| Rule | How this spec complies |
|---|---|
| `feedback_no_atomicadd.md` | All per-block reductions use block tree-reduce. Cross-block aggregation (Ring 2 fuzz invariant counters, sweep stats) happens host-side after `cudaDeviceSynchronize`. |
| `feedback_no_htod_htoh_only_mapped_pinned.md` | All CPU↔GPU buffers (MBP-10 input chunks via loader, alpha probs broadcast, audit ring, trade log) are mapped-pinned. No `cudaMemcpyAsync` HtoD/DtoH paths. Open Q4 resolution: per-trade records written directly into mapped-pinned via zero-copy device pointer. |
| `feedback_no_nvrtc.md` | All kernels pre-compiled to cubins via `build.rs`. Pair every `std::env::var` with `cargo:rerun-if-env-changed` per `pearl_build_rs_rerun_if_env_changed`. |
| `feedback_no_cpu_test_fallbacks.md` | Validation Ring 1 fixtures are GPU-oracle (kernel output asserted against hand-computed expected values from the fixture JSON); Ring 1b asserts bit-equality between two GPU paths (trainer vs backtest); Ring 2 asserts GPU-emitted invariants. No CPU reference implementation. |
| `feedback_cpu_is_read_only.md` | Host is orchestrator + telemetry consumer only. The `BacktestHarness` host struct owns no compute state — loader produces inputs, trunk runs inference, sim runs everything else on-device. |
| `feedback_isv_for_adaptive_bounds.md` | Kelly cap is ISV-derived from `realised_return_var`; per-horizon Sharpe-weighted aggregation is fully adaptive; no `horizon_mask` constant. Honest constants are listed explicitly in Section 5 (target_annual_vol = user risk preference; Wiener-α floor 0.4 = documented per `pearl_wiener_alpha_floor_for_nonstationary`). No `max(1, ...)` engagement floor. |
| `feedback_no_stubs.md` / `feedback_no_todo_fixme.md` | This is a design spec; implementation plan (next step) commits to full wiring with no return-zero stubs or TODO markers. |
| `feedback_no_partial_refactor.md` | All new modules extend the existing `ml-backtesting` crate; single binary `fxt-backtest` lands with it. `ml-alpha` change is contract-compatible: add `inference_only: bool` to `MultiHorizonLoaderConfig` + `peek_first()` + `next_inference_input()` methods on `MultiHorizonLoader`. All trainer call-sites pass `inference_only: false` and are unaffected; no migration needed. |
| `pearl_no_host_branches_in_captured_graph.md` | THREE captured graphs (trunk perception, LOB step-event, LOB decision) — each captures pure device work. Host-side branching (`if event_idx % decision_stride == 0`) happens BETWEEN captures, not inside. Per `pearl_cudarc_disable_event_tracking_for_graph_capture`, all captures bracket with event-tracking disable/enable. |
| `pearl_first_observation_bootstrap.md` | Per-horizon ISV-Kelly state uses sentinel-zero init; first observation replaces directly. `n_trades_seen` is a divisor-safety guard, not a "wait N trades before trading" gate. |
| `pearl_kelly_cap_signal_driven_floors.md` | Per-horizon cap derived from realised_return_var × annualisation_factor; floor explicitly absent (the prior "max(1, cap/4)" was the anti-pattern this pearl warns against). |
| Trainer parity (new) | Backtest reuses `MultiHorizonLoader`, `Mbp10RawInput`, `snap_feature_assemble` cubin, and `CfcTrunk` captured graph verbatim. Train-vs-deploy skew is structurally impossible; Ring 1b test enforces bit-equality. |

---

## Appendix: file inventory (for plan handoff)

New files (all inside the existing `crates/ml-backtesting/` crate):

```
crates/ml-backtesting/build.rs                              # NEW — compiles cubins, rerun-if-env-changed pairs
crates/ml-backtesting/src/harness.rs                        # NEW — BacktestHarness orchestrator (Section 7)
crates/ml-backtesting/src/sim.rs                            # NEW — LobSimCuda; owns device + mapped-pinned bufs
crates/ml-backtesting/src/order.rs                          # NEW — OrderIntent + OrderEvent (audit) + SlotTag
crates/ml-backtesting/src/artifacts.rs                      # NEW — summary.json / trades.csv / pnl_curve.bin writers
crates/ml-backtesting/src/aggregate.rs                      # NEW — sweep aggregation
crates/ml-backtesting/src/policy/mod.rs                     # NEW — Strategy tree + flattening to bytecode program
crates/ml-backtesting/src/policy/composition.rs             # NEW — Ensemble / Portfolio / RegimeSwitch + aggregators
crates/ml-backtesting/src/policy/latency.rs                 # NEW — LatencyConfig sampling helpers
crates/ml-backtesting/src/policy/sizing.rs                  # NEW — SizingPolicyId variants
crates/ml-backtesting/src/policy/stop_rules.rs              # NEW — StopRules
crates/ml-backtesting/src/lob/mod.rs                        # NEW — Rust bindings to cubins
crates/ml-backtesting/cuda/book_update.cu                   # NEW — MBP-10 event → shared-mem book mutation
crates/ml-backtesting/cuda/order_match.cu                   # NEW — resting-order matching + queue decay
crates/ml-backtesting/cuda/decision_policy.cu               # NEW — program bytecode interpreter + per-horizon sizing
crates/ml-backtesting/cuda/pnl_track.cu                     # NEW — realized/unrealized P&L + segment_complete events
crates/ml-backtesting/cuda/risk_reduce.cu                   # NEW — per-block stats reductions for summary
crates/ml-backtesting/tests/lob_sim_fixtures.rs             # NEW — Ring 1 fixture tests (N=1, bit-exact)
crates/ml-backtesting/tests/fixtures/                       # NEW — JSON fixture sequences (×15-20)
crates/ml-backtesting/tests/trainer_parity.rs               # NEW — Ring 1b bit-equality test
crates/ml-backtesting/tests/lob_sim_fuzz.rs                 # NEW — Ring 2 invariant fuzz tests (N∈{1,16,256})

bin/fxt-backtest/Cargo.toml                                 # NEW — single binary crate
bin/fxt-backtest/src/main.rs                                # NEW — CLI with `run` + `aggregate` subcommands
```

Modified files:

```
Cargo.toml                                                   # add bin/fxt-backtest to workspace members
crates/ml-backtesting/Cargo.toml                             # add deps: cudarc, ml-alpha, plus build-dependencies for cubin compile
crates/ml-backtesting/src/lib.rs                             # re-export new modules alongside existing barrier_backtest exports
crates/ml-alpha/src/data/loader.rs                           # add: MultiHorizonLoaderConfig.inference_only: bool,
                                                             #      MultiHorizonLoader::peek_first(),
                                                             #      MultiHorizonLoader::next_inference_input()
                                                             # All existing trainer call-sites pass inference_only=false (unchanged behaviour).
```

Notably **NOT modified**: `crates/ml-alpha/src/cfc/`, `crates/ml-alpha/src/heads.rs`, `crates/ml-alpha/src/mamba2_block.rs`, `crates/ml-alpha/src/trainer/`, and existing `crates/ml-backtesting/src/{barrier_backtest, action_loader, report}.rs`. The trunk's existing public API (`capture_graph_a`, `update_input_buffers`, `perception_forward_captured`, `snapshot_hidden`) is sufficient.

No existing-file deletions.
