# ml-alpha — Greenfield CfC + PPO Trading System Design

**Date**: 2026-05-16
**Status**: Approved with **2026-05-16 stacked-architecture decision** (see below)
**Supersedes**: `2026-05-16-alpha-ppo-trainer.md` (the earlier DQN-port plan, abandoned per user direction to rebuild from first principles)

## 2026-05-16 Decision: Mamba2 → CfC stacked perception is the default

The original spec framed CfC as a *replacement* for Mamba2 at the perception trunk, with a competitive gate requiring CfC AUC ≥ Mamba2 AUC at every horizon. Mid-execution of Plan 1, the architecture was revised to a **stacked design** where Mamba2 acts as the sequence encoder feeding into CfC. The CfC layer learns per-cell time-constants over the SSM output.

**Why**: Phase 1d.3 already validated `Mamba2 → MLP stacker`. Replacing the MLP stacker with CfC strictly extends the validated path. The stacked model is now THE default production architecture — there's no competing-baseline gate (CfC vs Mamba2-alone was the old framing); validation is just normal training metrics (per-horizon val AUC, train loss curve, sanity-check baseline floor of >0.5 AUC).

**Affected sections**:
- Section 2 — `Perception trunk` reads: Mamba2 SSM (input → 128-dim h_mamba) → CfC cell (128-dim h_mamba → 128-dim h_cfc) → multi-horizon heads + projection
- Section 4 — Validation: per-horizon val AUC monitored each epoch; no competitive comparison required
- Section 7 — Param count: ~50K trainable (15K Mamba2 + 35K CfC stack)

The CfC kernels (`cfc_step.cu`), heads, projection, BCE, AdamW, and Graph A are unchanged. Only the trunk's forward path adds the Mamba2 prefix.

## TL;DR

A clean greenfield rebuild of `ml-alpha` as a full closed-loop alpha trading system for ES.FUT on IBKR with $35k starting capital. Three components: (1) a Rust model library that owns a CfC perception trunk + 5 multi-horizon alpha heads + a CfC actor-critic policy, (2) an offline training binary, (3) an `AlphaPpoStrategy` integration in the existing `trading_agent_service` for live runtime. Everything GPU-resident on the hot path. Online learning with EWC anchoring + replay buffer (80% offline / 20% live) + kill-switch on σ-band metric deviations. Hyperparameters that vary during training are ISV-driven (signal-driven controllers, no magic numbers). Reuse the existing IBKR adapter, data ingestion, execution engine, and risk manager.

## Why this design exists

The session that led here started with the DQN smoke baseline (`alpha_baseline.rs`) running at per-bar trade frequency, then discovered via stride sweep that **the framing was the bug**: decision cadence at per-bar level was coin-flipping on noise. Switching to `decision_stride=200` with scaled training flipped 3-fold mean Sharpe at quarter-tick cost from **-4.29 (per-bar) to +1.78 (stride=200 + scaled)**, with variance collapsing from ±8.8 to ±1.15.

That validated the alpha signal at IBKR-realistic costs but exposed that `alpha_baseline.rs` was a smoke trainer — not the right vehicle for production. The first attempt to port the production DQN trainer (`gpu_dqn_trainer.rs`, 1.9 MB) was abandoned because (a) DQN's per-bar Bellman target is structurally wrong for minute-horizon alpha, (b) we have no validation that the existing DQN architecture fits this problem, (c) porting DQN means inheriting DQN-shaped assumptions. User direction: rebuild from first principles, no backward-compat, no transitional layers — "consistent in development without endless gates or backward compats."

## Architecture summary (one paragraph)

A snapshot-level CfC perception trunk (Closed-form Continuous-time recurrent network, ~128 cells, each with its own learnable time constant) reads Databento MBP-10 snapshots and emits 5 multi-horizon direction probabilities (P(up at h) for h ∈ {30, 100, 300, 1000, 6000} snapshots forward). A small CfC actor-critic policy reads a 20-dimensional state (5 alpha probs + 8-dim perception projection + position state + spread / vol / time features) and outputs a target position categorical over {-10, ..., 0, ..., +10}. The policy fires every `decision_stride=50` snapshots (~1 minute). Trades happen when `target ≠ previous_target`. Reward per decision step is dense per-segment P&L minus IBKR commission minus optional vol penalty. The model is fully GPU-resident on the hot path. Offline training is two-phase: (A) supervised multi-horizon BCE on the perception, with a hard validation gate requiring CfC AUC ≥ existing Mamba2 baseline before continuing, and (B) PPO on the frozen perception with chronological walk-forward CV over 6 folds spanning 9 quarters of historical data. Live runtime reuses `data_acquisition_service` (Databento streaming already built), `trading_agent_service` (where a new `AlphaPpoStrategy` plugs in), `trading_service::execution_engine` + `risk_manager`, and `crates/trading_engine/src/brokers/interactive_brokers.rs` (the IBKR adapter, 1177 LOC, already exists). Online updates run every 4 hours or 10K live decisions, applying PPO with an EWC term anchored to the offline-trained baseline, drawing 80% from a frozen offline replay buffer + 20% from live trajectories, with a kill-switch that rolls back weights if any of {loss, mean reward, hit-rate, KL divergence, entropy} deviates outside σ-band thresholds.

---

## Section 1 — System scope and integration

### Component boundaries

| | Where it lives | Role | Approx new LOC |
|---|---|---|---|
| `ml-alpha` Rust library | `crates/ml-alpha/` (gut existing files, rebuild clean) | CfC perception + heads + CfC policy + EWC anchor + Adam state + replay buffer types + training kernels | ~3000 |
| `alpha_train` binary | `crates/ml-alpha/examples/alpha_train.rs` | Offline training entry: Phase A (perception pretrain), then Phase B (PPO) per fold | ~300 |
| `AlphaPpoStrategy` | `services/trading_agent_service/src/strategies/alpha_ppo.rs` (NEW) | Live runtime adapter: owns `AlphaModel` + `ReplayBuffer`, called per snapshot, emits target signals | ~250 |

### What gets reused (no new work)

| Component | Existing path |
|---|---|
| Databento streaming | `crates/data/src/providers/databento_streaming.rs` |
| Live MBP-10 ingestion | `services/data_acquisition_service/` |
| Order execution + routing | `services/trading_service/src/core/execution_engine.rs`, `broker_routing.rs`, `order_manager.rs`, `position_manager.rs` |
| Risk manager (limits, kill-switch hook) | `services/trading_service/src/core/risk_manager.rs` |
| Broker gateway | `services/broker_gateway_service/` |
| IBKR adapter (1177 LOC, `ibapi` v10.19+) | `crates/trading_engine/src/brokers/interactive_brokers.rs` |

### Public API of the model library

```rust
// crates/ml-alpha/src/lib.rs (public surface)

pub struct AlphaModel {
    // CfC weights + heads + policy + EWC anchor + Adam state — all GPU-resident.
    // Position state, PnL state, last-segment-open-mid — also GPU-resident, owned here.
    // Pre-allocated mapped-pinned ingress slots for snapshot bytes and fill events.
}

impl AlphaModel {
    pub fn load(weights_path: &Path) -> Result<Self>;
    pub fn save(&self, weights_path: &Path) -> Result<()>;

    // Hot path — called per snapshot during live. Target latency p99 ≤ 500μs.
    // The snapshot has already been written into AlphaModel's mapped-pinned ingress
    // slot by the caller (zero HtoD). This call launches the captured CUDA Graph
    // for: snap_feature_assemble → CfC step → multi-horizon heads → projection.
    // No return value: outputs are on-device in ISV slots 0..13. The caller does
    // NOT read those slots — only the policy_action and update paths consume them.
    pub fn perception_forward(&mut self);

    // Decision point — called every decision_stride snapshots, also via captured
    // CUDA Graph. Internally:
    //   1. build_state kernel reads ISV slots 0..13 + position/PnL/time state +
    //      market-context features → writes 20-dim AlphaState into next replay
    //      buffer slot (no temporary tensor)
    //   2. policy_forward kernel: state → CfC actor → 21 logits + log_softmax,
    //      CfC critic → value; logit-mask outside ±max_live; sample via cuRAND;
    //      write (action_idx, log_prob, value) into the same replay buffer slot
    //   3. cudaMemcpyAsync the single action_idx byte to a mapped-pinned readback
    //      slot; cudaStreamSynchronize once at the end so the strategy can read
    //      the action enum on CPU. Total round-trip ≈ 50-100μs.
    // Returns ONLY the action (an i8 enum). log_prob and value live in the buffer
    // on-device — never seen by CPU.
    pub fn policy_action(&mut self) -> Action;

    // Called from on_fill — the fill event has been written into the model's
    // mapped-pinned fill-ingress slot. Updates position state, PnL state, then
    // BACK-FILLS the reward into the prior replay buffer slot's reward field
    // (kernel reads pinned fill, updates state, computes ΔPnL, writes reward).
    // No CPU compute, no plain HtoD.
    pub fn ingest_fill(&mut self);

    // Slow path — periodic online update; ~30s wall on L40S.
    pub fn update(&mut self, batch: &MiniBatch) -> UpdateStatus;

    // Mapped-pinned ingress accessors — strategy writes via volatile stores.
    pub fn snap_pinned_slot_mut(&mut self) -> &mut MappedPinnedSnapshotSlot;
    pub fn fill_pinned_slot_mut(&mut self) -> &mut MappedPinnedFillSlot;

    // Audit / observability — slow path, slow snapshot of ISV slots to a small
    // mapped-pinned readback buffer. Called from metrics task on a timer, never
    // from the hot path. The 1-shot DtoH copy is wrapped in mapped-pinned per
    // feedback_no_htod_htoh_only_mapped_pinned.md (slow path is not exempt).
    pub fn read_isv_snapshot(&self) -> IsvSnapshot;
}

pub struct ReplayBuffer { /* offline + live trajectories, all GPU-resident */ }

impl ReplayBuffer {
    // No record_step / complete_last_segment_reward in the public API —
    // those happen INSIDE AlphaModel kernels (policy_action writes the
    // forward fields; ingest_fill writes the reward). The replay buffer
    // is opaque from CPU.
    pub fn sample_mixed(&self, n: usize, offline_frac: f32) -> MiniBatch;
    pub fn load_offline_from(&mut self, path: &Path) -> Result<()>;
}

pub enum UpdateStatus {
    Applied   { metrics: TrainingMetrics },
    RolledBack { reason: String, deviating_metric: String, sigma: f32 },
}

#[repr(i8)]
pub enum Action { /* target_position ∈ {-10, ..., +10} */ }
```

#### GPU/CPU contract (NON-NEGOTIABLE)

| Quantity | Lives on | Movement |
|---|---|---|
| Model weights, anchor, Fisher, Adam state, hidden states | **GPU** | DtoD only; serialized to disk only at save/load (slow path) |
| Snapshot bytes from Databento socket | **CPU** → **mapped-pinned** → **GPU reads directly** | No plain HtoD anywhere |
| Fill events from broker | **CPU** → **mapped-pinned** → **GPU reads directly** | No plain HtoD anywhere |
| Position state, PnL state, time-since-entry, segment-open-mid | **GPU** | Updated by GPU kernels triggered by snapshot or fill ingress |
| AlphaState (20-dim) | **GPU** | Assembled by `build_state` kernel directly into replay slot |
| log_prob, value, reward in replay buffer | **GPU** | Written by `policy_action` and `ingest_fill` kernels; never crosses CPU |
| Chosen `Action` enum (i8) | **GPU → mapped-pinned readback → CPU** | One byte per decision, the only DtoH on the hot path |
| `UpdateStatus`, `TrainingMetrics`, `IsvSnapshot` | **GPU → mapped-pinned readback → CPU** | Slow path only, ≤1 KB |

Foxhunt rules enforced:
- `feedback_cpu_is_read_only.md` — no CPU arithmetic on model tensors. Counter increments and config branches in the strategy are control flow, not compute.
- `feedback_no_htod_htoh_only_mapped_pinned.md` — every CPU→GPU transfer (snapshot, fill, sampled-action seed) goes via mapped-pinned. No plain `cuMemcpyHtoD`.
- `feedback_no_atomicadd.md` — block tree-reduce for all reductions (BCE loss, PPO loss, EWC term, advantage normalisation, GAE return, mean reward EMA). Never `atomicAdd` in our kernels.
- `feedback_no_nvrtc.md` — every CUDA kernel pre-compiled to a cubin via `build.rs`. No runtime nvrtc. CfC step kernel, build_state, policy_forward, BCE loss, PPO loss + EWC term, AdamW, sample_mixed, GAE return — all cubins.
- `feedback_cudarc_f64_f32_abi.md` — Rust callers cast f64 → f32 explicitly at the FFI boundary before invoking float kernels.
- All GPU buffers pre-allocated at `AlphaModel::load`. No allocations on the hot path. Replay buffer ring is fixed size at startup.

---

## Section 2 — Model architecture

### Snapshot feature vector (input to CfC perception, ~32 dims)

Per MBP-10 snapshot, build a fixed-shape feature vector for the CfC trunk. Built on GPU once when the snapshot lands (via mapped-pinned), reused across all downstream calls.

| Index | Field | Source | Notes |
|---|---|---|---|
| 0 | mid log-return since prev snapshot | (mid_t − mid_{t-1}) / mid_{t-1}, then log | Direction signal |
| 1 | L1 spread in ticks | (best_ask − best_bid) / tick_size | Liquidity signal |
| 2–6 | Bid-side per-level size (L1..L5) | Normalized log-size | Top-of-book depth |
| 7–11 | Ask-side per-level size (L1..L5) | Normalized log-size | Top-of-book depth |
| 12–16 | Per-level OFI (L1..L5, signed) | Standard OFI from `ml_features::ofi_calculator` | Order flow imbalance |
| 17 | Trade arrival count since prev snapshot | Integer count, log-scaled | Trade-flow proxy |
| 18 | Trade signed volume since prev snapshot | Sum(buy_size) − Sum(sell_size), normalized | Trade-flow direction |
| 19 | Snapshot inter-arrival time (Δt in ms) | From MBP-10 timestamp | Already used by CfC for time-constant integration; included here for the input network too |
| 20–31 | Reserved for v2 (Hawkes intensity, microprice deviation, multi-level imbalances, etc.) | Initially zero | Architectural headroom |

L5 depth chosen (not full L10) to keep the input dim modest. L6-L10 are rarely informative for ES.FUT given liquidity is concentrated in L1-L3. Reserved slots 20-31 allow extension without retraining if v2 features prove necessary.

### Perception trunk (CfC, snapshot-level, stateful)

Closed-form Continuous-time recurrent network (Hasani et al. 2022). The practical trainable variant of LTC — replaces ODE solve with a closed-form recurrence, ~50× faster than naive LTC, preserves the "each neuron has its own time-constant" property.

- 128 CfC cells in the perception trunk
- Each cell has its own learnable time constant `τ_i`. Time-constant diversity gives multi-horizon natively: some cells specialize to fast dynamics, others to slow.
- Forward equation per snapshot:
  ```
  h_new_i = h_old_i · exp(-Δt / τ_i)
          + (1 − exp(-Δt / τ_i)) · tanh(W_in_i · snap_features + W_rec_i · h_old + b_i)
  ```
  where `Δt` is the actual nanoseconds-since-last-snapshot from MBP-10 timestamps. Irregular timestamps are native — no aggregation or padding to fixed bar intervals.
- Hidden state `h ∈ ℝ^128` lives on device, persists across snapshots. During training: one state per parallel env. During live: single state, persists across calls (re-initialized only at startup or after explicit reset).

### Multi-horizon alpha heads (5 thin linear projections)

For each `h ∈ {30, 100, 300, 1000, 6000}` snapshots forward: a `128 → 1` linear layer + sigmoid produces P(up at h). Trained jointly with the trunk via multi-horizon BCE loss.

### Projection layer (perception → policy state)

A learned `W_proj ∈ ℝ^(8×128)` linear layer projects the perception hidden state to an 8-dim summary that flows into the policy state. Trained jointly with perception (so the projection learns what the policy needs beyond just the explicit alpha probabilities).

### Policy network (CfC actor-critic)

| Component | Architecture |
|---|---|
| Actor | 20-dim state → CfC(64) → linear → 21 logits (target position ∈ {-10, ..., +10}) |
| Critic | 20-dim state → CfC(64) → linear → 1 scalar (V(s)) |

CfC time-constants in the policy let the policy "remember" recent context across decision steps. Useful because the policy fires every `decision_stride=50` snapshots, and the gap between calls is meaningful state.

### EWC anchor

- Stores baseline weights `θ_anchor` (copy of model weights after offline training)
- Stores diagonal Fisher information `F_i = E_val[(∂L_PPO / ∂θ_i)²]` (one scalar per parameter)
- Online updates add a penalty term: `λ_ewc · Σ_i F_i · (θ_current_i − θ_anchor_i)²`
- `θ_anchor` is frozen — only updated by explicit operator action

### Parameter count

| Component | Params |
|---|---|
| CfC perception trunk (128 cells, ~20 input dims) | ~20K |
| Multi-horizon heads (5 × 128 → 1) | ~700 |
| Projection (128 → 8) | ~1K |
| CfC actor (20 → 64 → 21) | ~6K |
| CfC critic (20 → 64 → 1) | ~5K |
| EWC anchor + Fisher diagonal | 2× weight count |
| AdamW optimizer state | 2× weight count |
| **Total trainable weights** | **~35K** |
| **Total device memory (weights + state + optimizer)** | **~1 MB** |

Inference is microseconds per snapshot. Training is minutes per fold even on consumer GPU.

---

## Section 3 — Action space, reward, state

### Action space

Single categorical head, target_position ∈ {-10, -9, ..., 0, ..., +9, +10} (21 outputs).

- Architecture is fixed at `max_train = 10` (21 outputs always).
- Live inference takes runtime parameter `max_live ≤ max_train`. Logits outside `[-max_live, +max_live]` are set to `-inf` on-device before softmax; sample or argmax operates on the remaining live range. Zero retraining cost to dial down at deployment.
- Trade-event semantics are derived: `trade_happens = (target_t ≠ target_{t-1})`.

### Reward (dense per-segment, all units in PRICE UNITS, not dollars)

```
r_t = ΔPnL_segment_t                              ← P&L over the decision_stride window (50 snapshots)
    − C_trade · |target_t − target_{t-1}|         ← IBKR commission per contract changed
    − C_vol · max(0, |segment_return_t| − r_threshold)   ← vol penalty (default coef 0)
```

All quantities in **price units of the underlying (ES.FUT points)**, not dollars. Conversion: $ amount = price units × $50/point × contracts. Keeping the reward in price units makes it position-size invariant: a 1-point move is the same reward regardless of how many contracts are held; the policy already accounts for `target` magnitude. Dollar P&L is computed only for risk monitoring (daily loss limit, margin) — never in the reward function.

`ΔPnL_segment_t` formal definition:
```
ΔPnL_segment_t = target_t · (mid_close_of_segment − mid_open_of_segment)
```
where `mid_open_of_segment` is the mid-price at the snapshot at which the segment started (the previous decision), and `mid_close_of_segment` is the mid at the current decision snapshot.

Constants:
- `C_trade = 0.034` price units per contract (IBKR ES: $0.85/side × 2 = $1.70 RT ÷ $50/point = 0.034)
- `C_vol = 0.0` initially (vol penalty disabled; ISV-driven controller can ramp up if vol becomes a problem — see Section 7)
- `r_threshold = 0.25` (1 tick — first tick treated as expected noise)

### State vector (20 floats, GPU-resident)

| Index | Field | Source | Normalization |
|---|---|---|---|
| 0–4 | 5 multi-horizon alpha probs (h=30/100/300/1000/6000) | Perception heads (sigmoid) | [0, 1] |
| 5–12 | 8-dim projection of perception hidden | Learned `W_proj: 128 → 8` | ~unit-variance from layer-norm |
| 13 | Current target position | Strategy state | `current / max_train` ∈ [-1, +1] |
| 14 | Unrealized P&L (in ticks since entry) | Strategy state | clipped to [-20, +20], scaled by /20 |
| 15 | Time-since-entry (decision steps) | Strategy state | clipped to [0, 100], scaled by /100 |
| 16, 17 | Minute-of-day sin, cos | Wall clock | [-1, +1] |
| 18 | Spread quantile (rolling 500-bar) | Live computation | [0, 1] |
| 19 | Recent realized vol (rolling 100-bar log-return std) | Live computation | scaled by historical 95th percentile |

---

## Section 3.5 — Capital envelope and IBKR broker constraints

### Capital math

Initial capital: **$35,000**. ES.FUT contract value ≈ $50/point × ~5500 points ≈ $275k notional per contract.

IBKR margin requirements (subject to broker discretion):
- Day-trading initial margin: ~$500-1,500/contract
- Overnight initial margin: ~$13,200-15,000/contract (exchange minimum ~$11,500 plus IBKR cushion)
- Maintenance margin (intraday): ~$1,000-1,500/contract

Conservative `max_live` ramp plan:

| Phase | `max_live` | Notional exposure | Trigger |
|---|---|---|---|
| Initial deployment | **1 contract** | ~7.8× leverage | Day-1 |
| After 100+ trades with positive realized Sharpe | 2 | ~16× leverage | Manual config bump |
| After 500+ trades, ≥3-month positive track record | 3 | ~23× leverage | Manual config bump |
| Beyond 3 | — | — | Requires more capital or better margin tier |

The model is always trained with `max_train = 10` (network output dim). The live cap grows by manual config bump only — never automatically.

### IBKR quirks (the "tricks" to design around)

| # | Quirk | Required design response |
|---|---|---|
| 1 | Pattern Day Trader rule: ≥4 day-trades in 5 biz days requires ≥$25k equity | Monitor day-trade count; halt strategy if equity drops near $25k threshold to avoid disqualification |
| 2 | Margin can be raised mid-session by IBKR risk dept | Pre-trade `whatif` margin check via IBKR API before submit; reject if margin insufficient |
| 3 | Liquidation if margin breach: IBKR force-closes at market | Maintain ≥40% margin cushion; if margin utilization > 60%, force flat *before* IBKR does |
| 4 | Order rejection without warning | Robust order-status state machine; treat rejection as halt-and-alert; manual ack to resume |
| 5 | Contract rollover: ES quarterly expiry (Mar/Jun/Sep/Dec) | Auto-detect rollover via front-month volume; flat positions in expiring contract 5 days before expiry; open in next month |
| 6 | Market halts (LULDs, circuit breakers) | Subscribe to TWS halt status; force-cancel pending orders; log + alert if positioned |
| 7 | API rate limits (>50 messages/sec) | Bound order submission rate; coalesce target changes within decision_stride |
| 8 | Commission tier (Pro vs Lite, Smart vs direct routing) | Use Smart Routing for ES; pin `$0.85/side` as assumed commission |
| 9 | Symbol drift on rollover (ESM6 → ESU6) | Symbol-lookup layer; never hardcode contract month |
| 10 | Daily loss circuit by IBKR | Match internal threshold; trigger flat earlier than IBKR would |

### Pre-deployment IBKR audit tasks

The existing `crates/trading_engine/src/brokers/interactive_brokers.rs` (1177 LOC, `ibapi` crate) needs an audit before go-live:
1. Verify it handles ES futures specifically (the adapter is built but may be stock/option focused)
2. Confirm `whatif` pre-trade margin check is exposed
3. Confirm rollover detection and symbol-update logic exists
4. Confirm halt-status subscription
5. Audit rejection-code catalog coverage
6. Check PDT counter integration with `risk_manager`

---

## Section 4 — Training pipeline

### Phase A: Perception pretrain (supervised, multi-horizon BCE)

**Data path:**
- Source: predecoded MBP-10 sidecars from the PVC (already built earlier this session). 61M snapshots / 9Q.
- Loader: zero-copy mmap of sidecars → mapped-pinned device staging buffer → CfC reads device memory directly. No CPU shadow.
- Sample shape: sequences of `seq_len = 64` consecutive snapshots, anchored at random valid positions.
- Labels: multi-horizon labels at `{30, 100, 300, 1000, 6000}` snapshots forward, computed once during data pipeline init.

**Training step (per minibatch):**
1. Sample `batch_size = 256` sequences. Upload bar indices to device via mapped-pinned.
2. CfC perception forward over `seq_len = 64`, producing 128-dim hidden + 8-dim projection at every position.
3. Multi-horizon heads emit 5 logits per position → sigmoid.
4. Loss = `(1/5) · Σ_h BCE(σ(logit_h), label_h)` averaged over valid positions.
5. Backward → AdamW step.

**Defaults:**
- Optimizer: AdamW, lr=3e-3, weight_decay=1e-2
- Epochs: 5 over the training window
- Time-constant init: log-uniform over [10ms, 1000s]

### Validation: per-horizon val AUC

After perception pretrain on the training window:
- Measure val AUC at each of the 5 horizons on the held-out window
- Sanity floor: every horizon AUC > 0.50 (above chance). The model is broken if any falls below.
- Reference: Phase 1d.3 stacker AUC was 0.66-0.71 at K=6000 on similar data; useful as a rough orientation but not a hard gate.

Per the 2026-05-16 stacked-architecture decision, perception is the default production path — there's no competing-baseline gate. Training continues to Phase B whenever the perception trainer converges cleanly (loss curve descending, val AUC > 0.50 everywhere). If perception fails to converge, the spec amendment doesn't have a planned alternative path — investigate the trainer.

### Phase B: PPO policy training on frozen perception

**Data path:**
- Perception is frozen; its forward outputs (5 alpha probs + 8-dim projection) are computed once per snapshot and cached in a device tensor.
- Iterated in temporal order — we're running an env, not sampling.

**Rollout collection (on-device):**
- `n_par = 64` parallel envs. Each env maintains (current_position, current_pnl, time_since_entry, last_trade_pos) on device.
- Step loop: every snapshot, perception forward (frozen). Every `decision_stride = 50` snapshots, build AlphaState, policy forward, sample action, compute reward over the 50-snapshot segment, store transition.
- After `rollout_length = 2048` decisions per env: stop collecting, run PPO updates.

**PPO update:**
1. Compute GAE: γ=0.99, λ=0.95 (both fixed)
2. K=4 PPO epochs, minibatches of 256 decisions
3. Per minibatch: PPO clip loss + value MSE + entropy bonus + EWC term; AdamW step
4. KL early-stop within K epochs if `KL(current || rollout) > 0.02`

### Walk-forward CV (chronological, 6 folds, expanding window)

| Fold | Train | Val (select within-fold hyperparams) | Test (held-out) |
|---|---|---|---|
| 1 | Q1-Q4 2024 | Q1 2025 | Q2 2025 |
| 2 | Q1 2024 - Q1 2025 | Q2 2025 | Q3 2025 |
| 3 | Q1 2024 - Q2 2025 | Q3 2025 | Q4 2025 |
| 4 | Q1 2024 - Q3 2025 | Q4 2025 | Q1 2026 |
| 5 | Q1 2024 - Q4 2025 | Q1 2026 | (no test — future) |
| 6 | All 9Q | (final calibration) | (live) |

Fold 4's test (Q1 2026) is true forward OOS. Reported test Sharpe is the mean of folds 1-4 test segments, with std as the uncertainty band.

---

## Section 5 — Live runtime integration

### Data flow at runtime

```
data_acquisition_service                trading_agent_service                       trading_service
   Databento stream    ──→  snapshot event  ──→   AlphaPpoStrategy:               ──→  emit target signal
                                                    1. mapped-pinned snapshot           ──→  risk_manager (limits, kill-switch, PDT, margin)
                                                    2. model.perception_forward (GPU)   ──→  execution_engine
                                                    3. every 50 snaps: model.policy     ──→  broker_routing → IBKR adapter
                                                    4. build Action                                ↓
                                                    5. buffer.record_step                     broker_gateway_service
                                                                                                       ↓
                                                                                              IBKR TWS / Gateway
```

### AlphaPpoStrategy lifecycle (sketch — revised, GPU-resident state)

```rust
pub struct AlphaPpoStrategy {
    model: AlphaModel,                  // owns ALL state on GPU: weights, anchor, hidden,
                                        //                       position, PnL, replay ring
    decision_counter: usize,            // CPU control flow only
    last_target: i8,                    // CPU mirror of GPU position state (read-only sentinel
                                        //   for emit-signal-only-on-change branch)
    config: AlphaConfig,                // CPU config (max_live, decision_stride, ...)
    metrics: AlphaMetrics,              // CPU bookkeeping
    update_clock: UpdateClock,          // CPU schedule
}

impl Strategy for AlphaPpoStrategy {
    /// Hot path: called per snapshot. Target p99 ≤ 500μs.
    /// Captured-CUDA-Graph fast path on every call.
    fn on_snapshot(&mut self, snap: &Mbp10Snapshot) -> Option<Signal> {
        // 1. CPU writes the snapshot bytes into model's mapped-pinned slot.
        //    No HtoD copy — the GPU reads directly from this pinned region.
        self.model.snap_pinned_slot_mut().write_volatile(snap);

        // 2. Replay captured CUDA Graph: snap_feature_assemble → CfC step →
        //    multi-horizon heads → projection. Outputs land in ISV[0..13] on-device.
        self.model.perception_forward();

        // 3. CPU control flow only.
        self.decision_counter += 1;
        if self.decision_counter % self.config.decision_stride != 0 {
            return None;
        }

        // 4. Decision point. Captured CUDA Graph: build_state → policy_forward →
        //    sample_action → write (state, action_idx, log_prob, value) to next
        //    replay slot. ONLY the action_idx (i8) crosses DtoH (mapped-pinned).
        let action = self.model.policy_action();
        let target = action.clamp_to(self.config.max_live);

        let signal = if target != self.last_target {
            self.last_target = target;
            Some(Signal::SetTargetPosition { contracts: target })
        } else {
            None
        };

        // 5. Periodic update (slow path).
        if self.update_clock.tick() {
            let batch = self.model.replay_buffer().sample_mixed(8192, /*offline_frac*/ 0.8);
            match self.model.update(&batch) {
                UpdateStatus::Applied { metrics } => self.metrics.update_applied(metrics),
                UpdateStatus::RolledBack { reason, .. } => {
                    self.metrics.update_rolled_back(&reason);
                    self.config.online_updates_enabled = false;  // auto-disable; manual ack
                    tracing::error!(reason, "online update rolled back; auto-updates DISABLED");
                }
            }
        }

        signal
    }

    /// Fill ingress: ALSO via mapped-pinned, then a single GPU kernel
    /// updates position_state, pnl_state, computes ΔPnL for the open segment,
    /// and writes the reward back into the prior replay slot.
    /// No CPU PnL math, no plain HtoD.
    fn on_fill(&mut self, fill: &Fill) {
        self.model.fill_pinned_slot_mut().write_volatile(fill);
        self.model.ingest_fill();  // GPU kernel: updates state + back-fills reward
    }
}
```

### Why this structure satisfies the GPU/CPU contract

| Per-snapshot CPU work | Per-snapshot GPU work |
|---|---|
| Volatile write of snapshot to mapped-pinned slot (≈70 bytes) | snap_feature_assemble kernel (computes Δt, log-returns, OFI per level) |
| `decision_counter += 1` | CfC step kernel (128 cells × per-cell time-constant integration) |
| Modulo branch | Multi-horizon heads (5 linears + sigmoid; block tree-reduce) |
| If decision: read mapped-pinned readback byte for `action` | Projection 128→8 |
| Compare `target != last_target` and emit signal | (Decision-only) build_state kernel → 20-dim AlphaState directly into replay slot |
| (slow path) Schedule check, branch into update | (Decision-only) policy_forward kernel: actor + critic + logit mask + cuRAND sample → writes (action_idx, log_prob, value) to replay slot |
| | DtoH async memcpy of single `action_idx` byte to mapped-pinned readback; one `cudaStreamSynchronize` at end of decision step |

Zero CPU arithmetic on model tensors. Zero plain HtoD. Zero atomicAdd. All reductions are block tree-reduce.

### Persistence and restart semantics

| Element | Persistence | Restart behavior |
|---|---|---|
| Model weights | Atomic write to `/data/ml-alpha/weights-<ts>.bin` on every successful `update()` | Load latest at startup |
| Perception hidden state | Not persisted (v1) | Cold-start: zeros; emit flat signal for first 60s while CfC saturates |
| Replay buffer | Live trajectories written to QuestDB; offline trajectories loaded from PVC | Rebuild: load offline + replay live from QuestDB since last weight snapshot |
| Position state | Owned by `trading_service::position_manager` | Reconcile with broker on startup |
| EWC anchor + Fisher matrix | Saved alongside weights | Loaded with weights |

### Observability

- Per snapshot: latency p50/p99 (Prometheus histogram)
- Per decision: action sampled, alpha probs at each horizon (QuestDB row)
- Per fill: realized P&L, position change, commission (QuestDB row)
- Per update: training metrics, kill-switch margin (Prometheus)
- Alerts: kill-switch fired, daily loss limit hit, margin breach, IBKR rejection

---

## Section 5.5 — CUDA Graphs and max-performance kernel design

Maximum performance is non-negotiable on the hot path. NVIDIA's reference patterns for sub-millisecond inference are: pre-compiled cubins, captured CUDA Graphs for fixed-shape work, persistent pinned-memory rings, cuBLAS Lt with epilogue fusion, and single sync per decision. This section enumerates exactly which kernels exist and how they're orchestrated.

### CUDA Graph capture topology

Three captured graphs replay over a per-stream + per-instance lifetime:

**Graph A — per-snapshot perception (captured once at model load):**
```
[snap_feature_assemble]
        ↓
[CfC step]         ← reads h_old, writes h_new (ping-pong device buffers)
        ↓
[multi-horizon heads (fused: 5 linears into one 5×128 GEMM via cuBLAS Lt)]
        ↓
[projection 128→8]
        ↓
[layer-norm + write to ISV slots 0..13]
```
Single graph replay per snapshot. Inputs pointer-stable (mapped-pinned snapshot slot, ping-pong h buffer); kernel args are device pointers, not values, so the graph stays valid across replays.

**Graph B — per-decision policy + replay write (captured once at model load):**
```
[build_state]                ← reads ISV[0..13] + position/pnl state → 20-dim AlphaState
        ↓
[policy_forward (fused: actor CfC + critic CfC + logit-mask + log_softmax)]
        ↓
[sample_action (cuRAND device API, no host sync)]
        ↓
[write_replay_slot (writes state, action_idx, log_prob, value to ring)]
        ↓
[async DtoH 1 byte → mapped-pinned readback]
```
Single graph replay per decision. The `max_live` mask threshold is read from device memory (ISV slot 28, written by config-update path), so it stays inside the graph.

**Graph C — per-update training step (captured once at model load):**
```
[sample minibatch indices (cuRAND, device-side)]
        ↓
[gather state/action/reward/value/log_prob from replay (device gather)]
        ↓
[compute GAE returns + advantages (sequential prefix scan, single-block cooperative)]
        ↓
[advantage normalization (block tree-reduce mean/std, then z-score)]
        ↓
[policy_forward + critic_forward (fused, batched, cuBLAS Lt GEMMs with bias-epilogue)]
        ↓
[ppo_loss + value_loss + entropy_bonus + EWC_term (fused into one kernel — all reductions block tree-reduce)]
        ↓
[backward through fused forward (cuBLAS Lt grad GEMMs)]
        ↓
[AdamW step (per-parameter-group lr scaling, in-place flat-buffer update)]
        ↓
[update ISV slots 13..24 with diagnostics]
```
One graph per PPO minibatch. Capture once at startup; replay K=4 epochs × N minibatches per update. Mirrors the `fused_training.rs` pattern from `gpu_dqn_trainer.rs` — replaces 2,000+ Candle dispatches per training step with one captured graph.

### Kernel inventory and which one calls cuBLAS

| Kernel (NVRTC pre-compiled cubin) | Purpose | Reduction strategy | cuBLAS path |
|---|---|---|---|
| `snap_feature_assemble.cu` | MBP-10 levels → 32-dim feature vector (mid log-return, spread, depth, OFI per level, trade flow, Δt) | None (element-wise) | None |
| `cfc_step.cu` | One CfC time step: `h_new = h_old·exp(-Δt/τ) + (1-exp(-Δt/τ))·tanh(W_in·x + W_rec·h + b)` | None | **Yes** — `W_in @ x` and `W_rec @ h_old` as fused 2-matrix GEMM via cuBLAS Lt with bias + tanh epilogue |
| `multi_horizon_heads.cu` | 128-dim hidden → 5 logits + sigmoid | None | **Yes** — single 5×128 GEMM via cuBLAS Lt with sigmoid epilogue |
| `projection.cu` | 128 → 8 with layer-norm | Block tree-reduce for mean/std | **Yes** — 8×128 GEMM |
| `build_state.cu` | Compose 20-dim AlphaState from ISV slots + position/PnL state | None (element-wise) | None |
| `policy_forward.cu` | Actor CfC + critic CfC, fused 4 GEMMs | Block tree-reduce for log_softmax denom | **Yes** — 4 fused GEMMs (state→hidden actor, hidden→logits, state→hidden critic, hidden→value) |
| `sample_action.cu` | cuRAND categorical sample with logit mask | Block tree-reduce for cumulative sum | None |
| `write_replay_slot.cu` | Write (state, action_idx, log_prob, value) to ring | None | None |
| `ingest_fill.cu` | Read mapped-pinned fill, update position/PnL state, back-fill reward into prior replay slot | None | None |
| `gae_returns.cu` | Sequential prefix scan over rollout for returns + advantages | Single-block cooperative scan | None |
| `advantage_normalize.cu` | z-score advantages across minibatch | Block tree-reduce mean + std | None |
| `ppo_loss.cu` | Fused: PPO clip + value MSE + entropy + EWC term + backward through policy_forward | Block tree-reduce for each loss term | **Yes** — grad GEMMs via cuBLAS Lt |
| `adamw_step.cu` | Per-parameter-group AdamW with weight decay | None (element-wise on flat buffer) | None |
| `bce_loss_multi_horizon.cu` | Phase A only: 5-horizon BCE + backward | Block tree-reduce | **Yes** — grad GEMMs |
| `kill_switch_check.cu` | Compute σ-distance for each metric against rolling baseline; return verdict | Block tree-reduce | None |

All reductions use the standard NVIDIA cooperative block tree-reduce pattern from `cooperative_groups.h` — single warp per block does the final reduce, no `atomicAdd`. Per `feedback_no_atomicadd.md`.

### cuBLAS Lt epilogue fusion patterns used

| Pattern | Where | Saving |
|---|---|---|
| `GEMM + bias + tanh` | CfC step (`W_in @ x + b` then tanh) | 2 kernels → 1 |
| `GEMM + bias + sigmoid` | Multi-horizon heads | 2 kernels → 1 |
| `GEMM + bias + ReLU` (if used) | Policy hidden layers | 2 kernels → 1 |
| `Batched GEMM` | Phase A perception over `seq_len=64` per batch | N kernel launches → 1 |
| `Strided batched GEMM` | Multi-env rollout during Phase B (`n_par=64` envs) | N kernel launches → 1 |

These mirror the patterns in `pearl_cublas_lt_vs_classic_sgemm.md`.

### Persistent allocations and ring layouts

- **Snapshot ingress slot**: 1 × (32 floats + 1 u64 timestamp) ≈ 136 bytes, mapped-pinned, host-write + device-read
- **Fill ingress slot**: 1 × (8 floats: fill_price, fill_qty, side, commission, ts, ...) ≈ 32 bytes, mapped-pinned
- **Action readback slot**: 1 × i8 ≈ 1 byte, mapped-pinned, device-write + host-read
- **CfC hidden ping-pong**: 2 × 128 × f32 = 1 KB device-resident (no pinned)
- **ISV bus**: 32 × f32 = 128 bytes device-resident, accessible by all kernels
- **Replay ring**: ring of `MAX_LIVE_TRANSITIONS` × `(state[20] + action[1 i8 + 3 pad] + log_prob[1] + value[1] + reward[1] + done[1] = 28 floats + 4 bytes)` ≈ 116 bytes × 10K = 1.16 MB
- **Offline replay**: 50K × 116 = 5.8 MB, GPU-resident, never re-uploaded
- **Per-minibatch scratch (training)**: 8192 × 116 ≈ 950 KB (sampled batch); plus gradient buffers ≈ 280 KB (35K params × 2 for momentum + variance) ≈ 1.5 MB total

All allocated once at `AlphaModel::load`. Replay ring uses a single-writer device-atomic ring index (CAS, no atomicAdd — uses `atomicCAS` for the write head only, which is per-decision overhead ≈ ns).

### Single-stream design (low-latency hot path)

Hot path is serial — perception depends on snapshot, policy depends on perception. No multi-stream benefit. Use a single dedicated `CUDA_STREAM_NON_BLOCKING` stream for the hot path.

Slow path (training updates) uses a separate stream to keep hot-path latency clean during overlapping update windows. The atomic weights swap synchronizes both streams at the boundary.

### Determinism for tests

- `cublasLtMatmulPreference` set to `CUBLAS_GEMM_DEFAULT_DEDUCED` (deterministic algorithm)
- `cublasSetMathMode(handle, CUBLAS_PEDANTIC_MATH)` in test builds
- Bit-equivalence tests target ε ≤ 1e-5 relative error across CPU oracle vs GPU
- Production builds use `CUBLAS_TF32_TENSOR_OP_MATH` for max throughput; accept the bit-divergence (still ε-stable)

### Build-time kernel compilation (no NVRTC)

All .cu files compiled to cubins by `build.rs`:
- `nvcc --gpu-architecture=compute_89 --gpu-code=sm_89,sm_90 --cubin --use_fast_math`
- `sm_86` (RTX 3050) also targeted for local dev
- Cubins embedded in the binary via `include_bytes!` so no runtime file IO
- Per `feedback_no_nvrtc.md` — zero runtime kernel compilation

### Performance budget (target on L40S, after CUDA Graph capture amortized)

| Operation | Budget | Realistic |
|---|---|---|
| Per-snapshot perception (Graph A replay) | ≤100μs | ~30-50μs (small model, single graph replay) |
| Per-decision policy (Graph B replay) | ≤200μs | ~80-120μs (includes single DtoH for action byte) |
| Per-update full PPO step (Graph C × K=4 × N_minibatch) | ≤30s | ~15-25s on L40S for 8192-batch × K=4 |
| Cold start (model load + buffer load) | ≤30s | dominated by offline replay load from PVC |
| Kernel build (build.rs) | ≤2min | one-time per release build |

---

## Section 5.6 — Disconnect, failure, and degraded-mode handling

The hot path has two external dependencies — Databento (snapshot stream) and IBKR (fills, orders). Either can drop. The default safe state is **flat + paused**.

### Databento stream loss

Triggered by: gap in snapshot timestamps > `max_snapshot_gap_ms` (default 1000 ms), connection drop, or malformed payload.

| Step | Action |
|---|---|
| 1 | Mark strategy state `Degraded::DataStale` (no new `on_snapshot` calls processed) |
| 2 | Issue `Signal::SetTargetPosition { contracts: 0 }` to broker gateway immediately |
| 3 | CfC perception state `h` is **frozen** (no zero-reset — discontinuity should not corrupt the memory) |
| 4 | Reconnect with exponential backoff (1s, 2s, 5s, 10s, 30s, cap) |
| 5 | On reconnect: continue feeding `on_snapshot`, but `decision_counter` skips `decision_stride` ticks of warm-up before first action (CfC needs to re-prime) |

If gap > `max_recovery_window_s` (default 300s), force a full model reload (state vector and CfC `h` re-initialized from disk snapshot).

### IBKR API failure modes

| Failure | Detection | Response |
|---|---|---|
| Order rejected (margin, contract spec, throttle) | `OrderRejected` event | Log + Prometheus counter; do NOT retry blindly. If throttle: backoff per IBKR pacing rules (see Section 3.5). If margin: assume risk-system breach, target=0, halt |
| Fill stream stalled (no fills despite known orders) | No `Fill` events for `max_fill_silence_s` (default 60s) | Query IBKR for open orders; reconcile broker state against local `position` |
| TCP socket closed | `Disconnected` event from `ibapi` | Same as Databento: flat + paused + reconnect with backoff |
| Position drift (broker says X, local thinks Y) | Periodic reconcile (every 60s) | Hard error: alert + halt; never silently re-sync (state corruption from a missed fill must be diagnosed, not papered over) |

### What is NEVER done

- **No retry-on-error of the same action** without first reconciling external state
- **No "if disconnected, keep predicting from last known state"** — the policy can't trade if data is stale
- **No silent re-init of CfC hidden state** mid-day — a discontinuity in `h` is a bug to log, not a recovery action

### Restart semantics (operator-driven)

`alpha-control restart` writes a clean shutdown signal:
1. Issue target=0, wait for fill confirmation
2. Persist `(weights, h, position, replay_ring, ewc_anchor, kill_switch_state, isv_state)` to disk
3. Acknowledge shutdown to operator

Cold-start on next launch reloads all of the above. See Section 6 cold-start subsection for what happens before enough live data accumulates.

---

## Section 6 — Online learning mechanics

### EWC anchor — what gets frozen, what gets learned

After Phase B finishes on the training window:
1. Take trained weights `θ_anchor` and freeze them as reference.
2. Compute diagonal Fisher information `F_i = E_val[(∂L_PPO / ∂θ_i)²]` per parameter.
3. Persist `(θ_anchor, F)` to disk alongside live weights.

During online updates:
```
loss = loss_PPO + λ_ewc · Σ_i F_i · (θ_current_i − θ_anchor_i)²
```
`λ_ewc` is the conservatism dial — initial value 1e3 but driven by a controller during operation (see Section 7).

### Replay buffer composition

| Component | Source | Fraction | Lifecycle |
|---|---|---|---|
| Offline trajectories | Saved during Phase B (last ~3 training quarters, ~50K decisions) | 80% (controller-tunable) | Loaded once at startup; GPU-resident; never updated |
| Live trajectories | Collected by `AlphaPpoStrategy::on_snapshot` | 20% (controller-tunable) | Ring buffer of ~10K most recent live decisions; mirror written to QuestDB |

Each minibatch: 8192 transitions = 6553 offline + 1638 live (at default 80/20).

### Update cadence

Whichever fires first:
- Time-based: every `update_interval_secs = 14_400` (4 hours)
- Volume-based: every `update_interval_transitions = 10_000` live decisions

The update lasts ~30s on L40S; `policy_action` calls block momentarily during atomic weights swap (~10ms typical).

### Atomic weights swap with kill-switch

```rust
fn update(&mut self, batch: &MiniBatch) -> UpdateStatus {
    let snapshot_path = self.write_weights_snapshot()?;       // versioned snapshot first

    let candidate_weights = self.weights.clone_gpu();         // GPU-side copy
    let metrics = self.run_ppo_epochs_on(&candidate_weights, batch);

    match self.kill_switch.check(&metrics) {
        KillSwitchVerdict::Ok => {
            std::mem::swap(&mut self.weights, &mut candidate_weights);
            self.kill_switch.record(metrics.clone());
            UpdateStatus::Applied { metrics }
        }
        KillSwitchVerdict::Reject { reason, deviating_metric, sigma } => {
            // candidate_weights dropped — live weights untouched
            UpdateStatus::RolledBack { reason, deviating_metric, sigma }
        }
    }
}
```

### Cold-start bootstrap (first hours after deploy)

The kill-switch baseline needs ≥20 successful updates before its σ-bounds are meaningful. Until then, the system runs in **bootstrap mode**.

| State | Live trajectories collected | Behavior |
|---|---|---|
| **Bootstrap-init** | < 1,000 live decisions | Replay sampler returns 100% offline (live ring is too sparse to mix in without bias). No online updates yet. |
| **Bootstrap-warmup** | 1,000 – 10,000 | Replay sampler ramps from 100% offline → 80/20 linearly. Online updates allowed but kill-switch uses a *conservative* baseline: reject if any metric is 2σ worse than the Phase B walk-forward metrics (not the rolling live baseline, which doesn't exist yet). |
| **Bootstrap-baseline** | 10,000 + 20 successful updates landed | Switch to rolling-baseline kill-switch as documented below. |
| **Steady-state** | After bootstrap-baseline cleared | Full online learning with rolling σ-bounds. |

Bootstrap-mode kill-switch uses **Phase B walk-forward statistics as the reference distribution** — `mean_reward`, `loss_ppo`, `hit_rate`, `kl_div`, `entropy` from the final fold of Phase B CV serve as the pre-deployment baseline. These get written to a `phase_b_baseline.json` artifact alongside the weights snapshot at the end of Phase B training.

ISV updates are **always live** — controllers don't need a warmup period because their per-signal EMAs auto-bootstrap on first observation (per `pearl_first_observation_bootstrap.md`).

### Kill-switch criteria

Rolling baseline tracked from last 20 successful updates.

| Metric | Reject condition | Why |
|---|---|---|
| `loss_ppo` | > baseline_mean + 3σ | Training divergence |
| `mean_reward_per_decision` | < baseline_mean − 2σ | Strategy degrading |
| `hit_rate` | absolute < 0.45 | Below empirical floor |
| `kl_divergence(candidate \|\| anchor)` | > 0.1 | Policy drifting; EWC has failed |
| `entropy` | < 0.1 nats | Policy collapse (deterministic) |

If any fires: weights stay at pre-update state, `online_updates_enabled = false` in config, operator alert via Prometheus + log. Manual ack required to re-enable.

### Snapshot retention

Every successful update creates `weights-<timestamp>-<idx>.bin` (~150 KB). Retain rolling last 100. Rollback CLI: `alpha-control rollback --to <timestamp>`.

The anchor `(θ_anchor, F)` is FROZEN automatically — only updated by explicit operator command after a multi-month track record.

---

## Section 7 — Testing strategy, ISV-driven hyperparameters, risks

### Testing tiers (in order of cost / fidelity)

| Tier | What | When | Pass criterion |
|---|---|---|---|
| Unit tests | CfC cell, EWC loss, GAE, PPO clip, reward formula, action masking, replay buffer | Continuously | All pass |
| Bit-equiv | CfC forward CPU vs GPU on small inputs | Per commit | < 1e-5 relative error |
| Smoke training | Phase A perception on 1Q subset | Before each merge | val AUC > 0.55 at K=6000 |
| **CfC vs Mamba2 gate** | Phase A perception on 2Q | Before Phase B | CfC AUC ≥ Mamba2 AUC at every horizon |
| Walk-forward Sharpe | 6-fold CV with PPO on 9Q | Before paper trading | Mean test Sharpe > 0 at IBKR commission; std < mean |
| **Paper trading** | Live MBP-10 + simulated execution (no broker orders); 5 trading days | Before live capital | Live P&L tracks backtest ±15%; no kill-switch fires; latency p99 < 500μs |
| Live with $1k cap | Real money, `max_live=1`, manual kill armed, 5-day window | Before scaling | Realized Sharpe > 0; zero IBKR rejections; rollover clean if it occurs |

### ISV-driven hyperparameters (per `pearl_controller_anchors_isv_driven.md`)

Most "hyperparameters" are controller outputs, not config constants. The ml-alpha trainer gets its own ISV bus (~32 slots).

#### Controller-driven (signal-driven adaptation)

| Hyperparam | Controller signal | Update rule |
|---|---|---|
| `entropy_coef` | Entropy deficit = (target_entropy − current_entropy), target = 70% of uniform | `entropy_coef ← entropy_coef · exp(η · deficit)` |
| `lr_policy` | Loss-slope EMA + grad-norm stability | Wiener-α blend toward target_loss_slope = -ε per epoch; floor at 1e-5 |
| `λ_ewc` | Weight drift magnitude `‖θ − θ_anchor‖²` vs target | Raise λ if drift > 1.2× target; lower if < 0.5× target |
| `clip_eps` (PPO) | KL divergence trajectory; target KL = 0.01 | Raise clip if KL too low (under-update); lower if too high (over-update) |
| `value_coef` | Ratio of value loss to policy loss; target = 1:5 | Adjust value_coef to match target ratio |
| Replay offline/live mix | Reward gap = `live_reward_mean − offline_reward_mean` | Increase live fraction if gap > Nσ (regime shift signal); decrease if gap < 0 |
| `C_vol` (vol penalty) | Variance of returns vs target | Raise coef if return std > target × 1.5; keep at 0 otherwise |

All controllers use the standard foxhunt pattern: Wiener-α optimal blending + permanent floor (per `pearl_wiener_alpha_floor_for_nonstationary.md` and `pearl_blend_formulas_must_have_permanent_floor.md`).

#### Fixed parameters (genuinely architectural)

| Parameter | Value | Why constant |
|---|---|---|
| `decision_stride` | 50 snapshots | Architecture choice; changing requires retraining |
| `seq_len` (perception) | 64 snapshots | Architecture choice |
| `n_horizons` | 5 (30/100/300/1000/6000) | Architecture choice |
| `max_train` | 10 contracts | Architecture choice (network output dim) |
| `n_par` (parallel envs) | 64 | Hardware fit |
| `batch_size` (perception) | 256 sequences | Hardware fit |
| `batch_size` (PPO update) | 8192 | Hardware fit |
| GAE γ | 0.99 | Standard; controller would risk subtle bugs |
| GAE λ | 0.95 | Standard |
| K (PPO epochs) | 4 | KL early-stop within K catches over-update |
| Kill-switch σ thresholds | 3σ loss, 2σ reward | Safety floor — operator-controlled, not signal-driven |

#### ISV slot allocation (32 slots)

```
ml-alpha ISV bus
─────────────────────────────────
  0..5    Perception output:  5 alpha probs at h={30,100,300,1000,6000}
  5..13   Perception hidden projection (8 dims)

  13..18  Training diagnostics:
          13: loss_ppo EMA
          14: kl_divergence (vs anchor)
          15: entropy_current
          16: grad_norm EMA
          17: weight_drift |θ − θ_anchor|²

  18..24  Controller outputs (effective hyperparams):
          18: lr_policy_eff
          19: lr_perception_eff
          20: entropy_coef_eff
          21: clip_eps_eff
          22: value_coef_eff
          23: λ_ewc_eff

  24..28  Replay buffer state:
          24: mean_live_reward EMA
          25: mean_offline_reward EMA
          26: live_offline_mix_eff (0..1)
          27: buffer_fill_fraction

  28..32  Operational / live:
          28: current_pos_eff
          29: realized_pnl_session_eff
          30: kill_switch_margin (σ-distance from rolling baseline)
          31: rolling_baseline_idx
```

### Known risks

| Risk | Severity | Mitigation |
|---|---|---|
| CfC fails to match Mamba2 | High | Validation gate stops Phase B; perception layer is replaceable in isolation; revert to Mamba2 if needed |
| PPO entropy collapse | Medium | Kill-switch fires on entropy < 0.1; entropy_coef controller raises automatically |
| Replay buffer staleness | Medium | 80% offline data is from fixed historical window; if regime shifts dramatically the offline anchor is obsolete. Manual remix or new offline trajectory generation needed periodically |
| IBKR margin spike on FOMC | Medium | Pre-trade `whatif` rejects oversized orders; daily loss limit catches systemic damage |
| Snapshot CfC compute slower than 500μs | Medium | Profile early; if perception forward > 100μs per snapshot may need TensorRT export for live (not training) |
| Decision-stride / horizon mismatch | Low | Decision_stride=50 << longest horizon 6000; plenty of decision points. Verified this session |
| EWC λ mis-tuning | Low | Now controller-driven, not hardcoded |
| Contract rollover bugs | Low (but high impact) | Existing trading_engine handles symbol changes; audit before go-live |

### What this design does NOT include

- HFT / sub-millisecond latency (target ≤500μs per snapshot, not nanoseconds)
- Cross-asset universe expansion (ES.FUT only)
- Hedging logic (outright ES only)
- Options strategies
- Hyperparameter sweep over architecture (the architecture itself — number of CfC cells, horizons, decision_stride — is fixed)
- Auto-rollback of `θ_anchor` (anchor moves only by explicit operator action)
- Time-of-day filter (overnight trading allowed; operator-controlled risk)

---

## Implementation order

1. **IBKR adapter audit** (1-2 days) — verify ES futures, `whatif`, rollover, halt, PDT counter
2. **ml-alpha library scaffold + ISV slot allocation** (1 day)
3. **CfC perception forward + multi-horizon heads + Phase A trainer** (3-4 days)
4. **Phase A run + CfC-vs-Mamba2 validation gate on 2Q** (1 day cluster) — GATE
5. **CfC policy (actor-critic) + reward + state construction** (2-3 days)
6. **PPO loss + GAE + replay buffer + EWC term + Phase B trainer** (3-4 days)
7. **ISV controllers (entropy, lr, λ_ewc, clip_eps, value_coef, replay mix, C_vol)** (2-3 days)
8. **Online update + kill-switch + atomic swap** (2 days)
9. **AlphaPpoStrategy integration in trading_agent_service** (2-3 days)
10. **Walk-forward CV (6 folds, full 9Q) on cluster** (1-2 days, mostly cluster wall)
11. **Paper trading on live MBP-10 + simulated execution (5 day window)**
12. **Live deploy with `max_live=1` (5 day window)**
13. **Ramp `max_live` per the table in Section 3.5**

Estimated end-to-end timeline to live: **~3-4 weeks of focused work**, including paper trading.

## Appendix A — Memory entries this design embodies

- `feedback_cpu_is_read_only.md` — no CPU compute, GPU is sole arithmetic engine
- `feedback_no_htod_htoh_only_mapped_pinned.md` — snapshot bytes via mapped-pinned, no HtoD
- `feedback_no_atomicadd.md` — block tree-reduce only (relevant for the multi-horizon BCE + EWC term kernels)
- `feedback_no_nvrtc.md` — pre-compiled cubins for any CUDA kernels
- `pearl_controller_anchors_isv_driven.md` — all tunables ISV-controlled
- `feedback_isv_for_adaptive_bounds.md` — bounds live in ISV
- `feedback_adaptive_not_tuned.md` — fixes are signal-driven, not tuned constants
- `pearl_wiener_alpha_floor_for_nonstationary.md` — floor of 0.4 on Wiener-α in non-stationary loops
- `pearl_blend_formulas_must_have_permanent_floor.md` — max-with-floor, not blend
- `pearl_first_observation_bootstrap.md` — EMAs bootstrap from sentinel = 0; first observation replaces directly
- `pearl_bar_abstraction_can_be_alpha_ceiling.md` — bar aggregation can cap alpha; snapshot-level chosen for this reason
- `pearl_state_amplifies_short_horizon_into_long_horizon.md` — stateful encoder (Mamba2/CfC) gives multi-minute alpha from snapshot input
- `pearl_phase1d4_backtest_cost_edge_frontier.md` — frictionless Sharpe ~4.4 baseline; break-even quarter-tick; this session pushed to ~1-tick at stride=200

## Appendix B — Today's session findings that motivated this design

- Per-bar trading = coin-flip on noise. Decision-stride 200 with scaled training flipped Fold A Sharpe at quarter-tick cost from -3.7 to +2.52; 3-fold mean from -4.29 to +0.78; with 16× scaled training to +1.78. The signal exists at minute-horizon; the framing of "decide every bar" was the bug.
- Variance across folds collapsed from ±8.8 (per-bar) to ±0.80 (stride=200) — removing noise-mining made the alpha consistent.
- IBKR commission at $1.70 RT translates to 0.034 price units — well below the new break-even of ~1 tick. Economic deployment is plausible.
- `alpha_baseline.rs` was a smoke trainer; the production DQN trainer (`gpu_dqn_trainer.rs`) is engineered for a different problem regime (per-bar Bellman, branched discrete actions); porting it would inherit the wrong shape. Greenfield rebuild with PPO + multi-horizon perception + CfC is the right move.
