plan(crt-1): unified inventory controller implementation plan
Per spec v3 (commit 9ad76c4df). Supersedes the v2 Phase A plan
(2026-05-20-crt-phase-a-continuous-controller.md) which is now an
artifact — its load-bearing commits (A0.5 forward_step, A1
decision_stride deletion) are preserved; its A2 scalar conviction-EMA
work is REPLACED by the multi-horizon §4.4 formula here.
Six tasks (all under one Gate CRT.1, no inter-task gates):
C1.1 open_trade_state 24→64 byte atomic refactor (spec §7)
C1.2 multi-horizon ISV-weighted conviction (spec §4.4) replacing
scalar EMA approach
C1.3 no-trade band in seed_inflight (delta_floor config field)
C1.4 composite exit_signal safety circuit-breaker (spec §4.3)
C1.5 local compile + tests
C1.6 cluster smoke + Gate CRT.1 validation
Gate CRT.1 acceptance = v2 §9 Gate 2 tiered MUST/WIN structure intact.
What ships in this plan:
- Continuous evaluation (every event, via existing forward_step)
- Multi-horizon ISV-weighted conviction (one unbounded factor =
net_edge / (var + cost²); rest bounded; per pearl_one_unbounded_signal)
- target_lots = direction × |conviction_ema| × envelope_max (no aggregate
rescale; the v2 A2/A2.1 bug is structurally absent)
- No-trade band: kernel skips seed when |target − effective| < delta_floor
- open_trade_state 64-byte expansion for per-trade trajectory tracking
- Composite exit_signal as safety circuit-breaker (primary exit is
emergent target→0)
What stays out of scope (CRT.2 / CRT.3):
- Adaptive max_lots / threshold / vol target (CRT.2)
- Self-tuning percentile gate (CRT.2)
- LoRA + EWC++ + shadow eval (CRT.3)
Status: ready for execution.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,642 @@
|
||||
# CRT.1 — Unified Inventory Controller Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Ship the v3 unified inventory controller per `docs/superpowers/specs/2026-05-20-continuous-reasoning-trader-design.md` v3 ("v3 ARCHITECTURE RESET" section). Replace the empirically-broken scalar conviction-EMA approach (commits `1d889d2de` + `fe2498769`, smokes `vjmwc` + `lkrdf` both RED on Gate 1) with the multi-horizon ISV-weighted conviction formula plus a no-trade band and the open_trade_state expansion. Single integrated milestone — what v2 called Phase A and Phase B are one mechanism.
|
||||
|
||||
**Architecture:** Every event, the trunk produces fresh per-horizon alpha probabilities. The unified conviction formula `weight_h × magnitude_h × direction_h` (weights from per-horizon ISV state's `net_edge / (var + cost²)`) collapses the vector into a single signed scalar. Wiener-α adaptive EMA smooths it. `target_lots = round(direction × |conviction_ema| × envelope_max)`. `seed_inflight_limits_batched` only seeds an order when `|target − effective_position| ≥ delta_floor`. Most events: target moves fractionally, no order. When the signal genuinely shifts or the trade thesis breaks: order seeds, position adjusts. Conviction-degradation exit is emergent from target → 0; the v2 §4.3 composite signal stays as a safety circuit-breaker.
|
||||
|
||||
**Tech Stack:** Rust 1.85, CUDA 12.4, cudarc 0.19, Mamba2/Cfc trunk via PerceptionTrainer, ISV state per-backtest×per-horizon, Tokio.
|
||||
|
||||
**Spec:** `docs/superpowers/specs/2026-05-20-continuous-reasoning-trader-design.md` (v3, commit `9ad76c4df`)
|
||||
|
||||
**Branch:** `ml-alpha-phase-a` (HEAD: `9ad76c4df`; v2 Phase A artifacts at `2e87ed0da` … `fe2498769`)
|
||||
|
||||
---
|
||||
|
||||
## Scope contract
|
||||
|
||||
**Load-bearing v2 commits (KEEP on branch, do not revert):**
|
||||
- `a0e81fbdf` — forward_step incremental SSM (A0.5)
|
||||
- `92f8b10ed` — forward_step_into eliminates GPU↔CPU round-trip
|
||||
- `045850e8f` — delete decision_stride field (A1; greenfields atomic)
|
||||
- `3d8f12deb` — buffer-level seed bit-identity test
|
||||
|
||||
**v2 commits to be superseded in code (NOT reverted via git; rewritten in this plan):**
|
||||
- `1d889d2de` — A2 scalar conviction-EMA with aggregate rescale. The three device slots (`conviction_ema_d`, `conviction_diff_var_ema_d`, `conviction_sample_var_ema_d`) STAY in `LobSimCuda` — the EMA mechanism is reused on the multi-horizon scalar. What gets deleted is the `update_conviction_ema` call site on `max(|p−0.5|)` and the `final_size *= scale` rescale step at the end of both decision kernels.
|
||||
- `fe2498769` — A2.1 amplification clamp. Falls away with the rescale deletion above.
|
||||
|
||||
**CRT.1 scope (ships in this plan):**
|
||||
1. Per-horizon ISV state read in `decision_policy_*` kernels (the ISV state struct already exists per `crates/ml-backtesting/cuda/decision_policy.cu`; A2.1's stop_check_isv reads it for stops — we extend the same reads for sizing)
|
||||
2. Multi-horizon ISV-weighted conviction formula (spec §4.4) replacing both the per-horizon-sum + the broken rescale
|
||||
3. Wiener-α EMA reapplied to the multi-horizon conviction (reuses existing `conviction_*_ema_d` slots)
|
||||
4. `target_lots = round(direction × |conviction_ema| × envelope_max)` — direct formula, no aggregate-rescale
|
||||
5. No-trade band in `seed_inflight_limits_batched`: skip if `|delta| < delta_floor`
|
||||
6. `open_trade_state` 24→64 byte expansion per spec §7 (atomic refactor across `pnl_track`, `decision_policy`, `resting_orders`, `sim/mod.rs`, tests)
|
||||
7. Conviction-degradation safety circuit-breaker per spec §4.3 (composite signal `exit_signal = max(degradation_term, disagreement_term, pnl_decay_term)`)
|
||||
8. Cluster smoke validation against Gate CRT.1 acceptance (= v2 §9 Gate 2 tiered MUST/WIN)
|
||||
|
||||
**Out of scope (deferred to CRT.2 / CRT.3 plans):**
|
||||
- Adaptive `max_lots` / threshold / vol target (CRT.2)
|
||||
- Self-tuning percentile gate (CRT.2)
|
||||
- LoRA adapter + EWC++ + shadow eval (CRT.3)
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
**Modify (CUDA):**
|
||||
- `crates/ml-backtesting/cuda/decision_policy.cu` — replace `update_conviction_ema` call from scalar `max_conv` to multi-horizon `conviction_signed` per spec §4.4; delete `final_size *= scale` rescale step; both `decision_policy_default` AND `decision_policy_program` (bytecode VM)
|
||||
- `crates/ml-backtesting/cuda/resting_orders.cu` — add `delta_floor` param to `seed_inflight_limits_batched`; skip seeding when `|delta| < delta_floor`
|
||||
- `crates/ml-backtesting/cuda/pnl_track.cu` — update `OPEN_TRADE_STATE_BYTES` from 24 to 64; rewrite open/close branches to populate the new fields (conviction_at_entry, conviction_per_horizon_ema, peak_unrealized, degradation_consecutive_events, etc.); reset-on-close zeros all 64 bytes
|
||||
|
||||
**Modify (Rust):**
|
||||
- `crates/ml-backtesting/src/sim/mod.rs` — `open_trade_state_d` allocation grows 24→64 bytes per backtest; add `delta_floor_d: CudaSlice<f32>` (per-backtest), allocator + uploader; thread into seed_inflight launch; `step_decision_*` launches unchanged signature-wise (kernel internals change)
|
||||
- `crates/ml-backtesting/src/sim/batched_config.rs` — add `delta_floor: f32` field (per-variant), default 1.0 lot
|
||||
- `bin/fxt-backtest/src/main.rs` — `SweepBase.delta_floor: Option<f32>` (default 1.0), threaded into `ResolvedSimVariant`
|
||||
- `crates/ml-backtesting/src/harness.rs` — no logic change; SSM forward still every event, decision kernels still every event (target-delta + no-trade-band handles cadence)
|
||||
|
||||
**Modify (config):**
|
||||
- `config/ml/sweep_smoke.yaml` — add `delta_floor: 1.0` to the smoke variant
|
||||
- Other YAMLs only need delta_floor if explicitly trading; otherwise default from base
|
||||
|
||||
**Modify (tests):**
|
||||
- `crates/ml-backtesting/tests/stop_controller.rs` — rewrite or replace these tests if they assert specifics on the old scalar-conviction path: `conviction_ema_smooths_micro_oscillations`, `conviction_ema_does_not_lag_reversals`, `conviction_ema_rescale_never_amplifies_weak_signal`. The first two can adapt to the multi-horizon formula; the third is now meaningless (rescale is gone).
|
||||
- Add `multi_horizon_conviction_sums_to_zero_on_disagreement` — disagreeing horizons cancel.
|
||||
- Add `no_trade_band_blocks_micro_delta` — small target delta produces no order.
|
||||
- Add `open_trade_state_64byte_layout` — byte-offset asserts for the new fields.
|
||||
|
||||
**Read-only (referenced but not modified):**
|
||||
- `crates/ml-backtesting/cuda/decision_policy.cu` `stop_check_isv` — already reads per-horizon ISV state; CRT.1's conviction formula reads the same struct.
|
||||
|
||||
---
|
||||
|
||||
## CRT.1 Tasks
|
||||
|
||||
### Task C1.1: `open_trade_state` 24→64 byte atomic refactor
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/ml-backtesting/cuda/pnl_track.cu` — `OPEN_TRADE_STATE_BYTES` const, open/close branch writes
|
||||
- Modify: `crates/ml-backtesting/cuda/decision_policy.cu` — `stop_check_isv` reads entry_ts at offset 0 (unchanged)
|
||||
- Modify: `crates/ml-backtesting/cuda/resting_orders.cu` — S2.2 max_hold event-rate check reads entry_ts at offset 0 (unchanged)
|
||||
- Modify: `crates/ml-backtesting/src/sim/mod.rs` — allocation grows
|
||||
- Tests: `crates/ml-backtesting/tests/stop_controller.rs` — `OPEN_TRADE_STATE_BYTES` references if any
|
||||
|
||||
- [ ] **Step 1: Write failing layout-assertion test**
|
||||
|
||||
Append to `crates/ml-backtesting/tests/stop_controller.rs`:
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn open_trade_state_64_byte_layout() {
|
||||
use ml_backtesting::sim::OPEN_TRADE_STATE_BYTES;
|
||||
assert_eq!(OPEN_TRADE_STATE_BYTES, 64, "spec §7 — 64-byte single-cache-line layout");
|
||||
}
|
||||
```
|
||||
|
||||
Run: `cargo test -p ml-backtesting open_trade_state_64_byte_layout` → expect FAIL (const is still 24).
|
||||
|
||||
- [ ] **Step 2: Expand the constant + Rust allocation**
|
||||
|
||||
In `crates/ml-backtesting/src/sim/mod.rs`:
|
||||
```rust
|
||||
pub const OPEN_TRADE_STATE_BYTES: usize = 64;
|
||||
```
|
||||
Allocation site: `open_trade_state_d` grows to `n_backtests * 64`.
|
||||
|
||||
- [ ] **Step 3: Rewrite pnl_track open branch**
|
||||
|
||||
In `crates/ml-backtesting/cuda/pnl_track.cu`, the open-branch (`prev_size == 0 && now_size != 0`) currently writes 5 fields at offsets 0/8/12/16/20. Extend to the 64-byte layout per spec §7:
|
||||
|
||||
```
|
||||
Offset Size Field
|
||||
0 8 entry_ts_ns (u64)
|
||||
8 4 entry_px_x100 (i32)
|
||||
12 4 entry_size (i32)
|
||||
16 4 realized_at_open (f32)
|
||||
20 4 conviction_at_entry (f32) ← NEW
|
||||
24 4 conviction_ema (f32) ← NEW (snapshot of conviction_ema_d[b] at entry)
|
||||
28 16 conviction_per_horizon_ema [4 × f32] (NEW — per-horizon EMA snapshot)
|
||||
44 4 pnl_adjusted_conviction_ema (f32) ← NEW
|
||||
48 4 peak_unrealized_pnl (f32) ← NEW (= 0 at open)
|
||||
52 4 degradation_consecutive_events (u32) ← NEW (= 0 at open)
|
||||
56 4 disagreement_consecutive_events (u32) ← NEW (= 0 at open)
|
||||
60 1 horizon_idx_dominant_at_entry (u8) ← NEW
|
||||
61 3 pad
|
||||
```
|
||||
|
||||
`conviction_at_entry` and `conviction_per_horizon_ema` come from the decision kernel's writes — but those happen BEFORE pnl_track in the harness ordering. Reads from `conviction_ema_d[b]` directly (and the per-horizon EMA buffers if they're allocated; if not, this is C1.2 work). For Task C1.1's scope: zero the new fields at open; subsequent tasks will populate them.
|
||||
|
||||
- [ ] **Step 4: Update reset-on-close to zero all 64 bytes**
|
||||
|
||||
In `pnl_track.cu` close branch: replace the existing 24-byte reset loop with a 64-byte loop.
|
||||
|
||||
- [ ] **Step 5: Verify entry_ts offset unchanged for downstream readers**
|
||||
|
||||
`stop_check_isv` (decision_policy.cu) and `resting_orders_step` max_hold path both read entry_ts via `+ 0` indexing into the 24-byte struct. With layout unchanged at offset 0 = entry_ts, those reads still work.
|
||||
|
||||
`grep -n "open_trade_state + (size_t)b \* 24"` should find these existing sites — change `24` to `64`:
|
||||
```bash
|
||||
grep -rn "size_t)b \* 24\|OPEN_TRADE_STATE_BYTES" crates/ml-backtesting/
|
||||
```
|
||||
Update every match to 64 / `OPEN_TRADE_STATE_BYTES` (prefer the named constant where Rust-side, hardcoded literal at CUDA side).
|
||||
|
||||
- [ ] **Step 6: Compile + run the failing test from Step 1**
|
||||
|
||||
```bash
|
||||
SQLX_OFFLINE=true cargo check -p ml-backtesting --tests 2>&1 | tail -5
|
||||
SQLX_OFFLINE=true cargo test -p ml-backtesting open_trade_state_64_byte_layout 2>&1 | tail -5
|
||||
```
|
||||
|
||||
Expected: PASS. Existing stop_controller tests still pass (entry_ts handling unchanged).
|
||||
|
||||
- [ ] **Step 7: Commit**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "$(cat <<'EOF'
|
||||
arch(crt-1): open_trade_state 24→64 byte expansion — atomic refactor
|
||||
|
||||
Per spec §7 (promoted from v2 Phase B to CRT.1 in v3 architecture reset).
|
||||
Every reader and writer of open_trade_state migrates in this commit per
|
||||
feedback_no_partial_refactor. No backwards-compat shim per
|
||||
feedback_no_legacy_aliases.
|
||||
|
||||
New 64-byte layout: existing 24-byte fields preserved at original offsets
|
||||
(entry_ts:0, entry_px_x100:8, entry_size:12, realized_at_open:16) so
|
||||
downstream readers (stop_check_isv, resting_orders max_hold check) need
|
||||
only update OPEN_TRADE_STATE_BYTES from 24 to 64.
|
||||
|
||||
New fields at offsets 20..63 will be populated by subsequent CRT.1
|
||||
tasks (C1.2 multi-horizon conviction, C1.3 degradation tracking).
|
||||
Open branch zeros new fields; close branch zeros all 64 bytes.
|
||||
|
||||
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||||
EOF
|
||||
)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task C1.2: Multi-horizon ISV-weighted conviction (replaces A2 scalar approach)
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/ml-backtesting/cuda/decision_policy.cu` — both kernels
|
||||
- Modify: `crates/ml-backtesting/tests/stop_controller.rs` — replace 3 obsolete tests; add 1 new
|
||||
|
||||
- [ ] **Step 1: Write failing test for multi-horizon disagreement cancellation**
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
#[ignore = "requires CUDA"]
|
||||
fn multi_horizon_conviction_cancels_on_disagreement() -> Result<()> {
|
||||
let dev = match MlDevice::cuda(0) { Ok(d) => d, Err(e) => { eprintln!("skip: {e}"); return Ok(()); }};
|
||||
let mut sim = LobSimCuda::new(1, &dev)?;
|
||||
sim.upload_price_range(&[1000.0], &[20000.0])?;
|
||||
let cfg = cfg_default();
|
||||
|
||||
// Two horizons bullish (0.7), two bearish (0.3) — equal weights would cancel.
|
||||
// With ISV-derived weights starting equal (cold start), conviction should be ~0
|
||||
// and no large target should form.
|
||||
let mixed: [f32; N_HORIZONS] = [0.7, 0.3, 0.7, 0.3];
|
||||
sim.broadcast_alpha(&mixed)?;
|
||||
sim.step_decision_with_latency(1_000_000_000u64, &cfg)?;
|
||||
let (_side, size) = sim.read_market_target(0)?;
|
||||
assert!(size.abs() <= 1, "disagreeing horizons must cancel; got size={size}");
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
Run → expect FAIL (current scalar logic doesn't aggregate per-horizon-signed).
|
||||
|
||||
- [ ] **Step 2: Replace `update_conviction_ema` call to use multi-horizon conviction**
|
||||
|
||||
In `decision_policy.cu`, the current code (post-A2.1) computes `raw_max_conv = max_h |p_h − 0.5| × 2` then calls `update_conviction_ema(b, raw_max_conv, ...)`.
|
||||
|
||||
Replace with the spec §4.4 formula:
|
||||
```c
|
||||
// CRT.1: multi-horizon ISV-weighted conviction.
|
||||
// Per spec §4.4 (promoted from v2 Phase B). Replaces v2 A2 scalar
|
||||
// max(|p-0.5|) which failed Gate 1 catastrophically.
|
||||
const IsvKellyState* isv = reinterpret_cast<const IsvKellyState*>(
|
||||
isv_kelly_base + (size_t)b * N_HORIZONS * sizeof(IsvKellyState)
|
||||
);
|
||||
const float cost = cost_per_lot_per_side[b];
|
||||
const float eps_edge = cost * 0.01f; // ε floor on net_edge per spec §4.4
|
||||
|
||||
float weighted_sum_signed = 0.0f;
|
||||
float total_abs_weight = 0.0f;
|
||||
#pragma unroll
|
||||
for (int h = 0; h < N_HORIZONS; ++h) {
|
||||
const float p_h = alpha_probs[h];
|
||||
const float direction_h = (p_h > 0.5f) ? 1.0f : -1.0f;
|
||||
const float magnitude_h = fabsf(p_h - 0.5f) * 2.0f;
|
||||
const float net_edge_h = fmaxf(isv[h].pnl_ema_win - isv[h].pnl_ema_loss, eps_edge);
|
||||
const float weight_h = net_edge_h / (isv[h].realised_return_var + cost * cost);
|
||||
weighted_sum_signed += magnitude_h * weight_h * direction_h;
|
||||
total_abs_weight += fabsf(weight_h);
|
||||
}
|
||||
const float conviction_signed = (total_abs_weight > 1e-9f)
|
||||
? (weighted_sum_signed / total_abs_weight)
|
||||
: 0.0f;
|
||||
const float conviction_abs = fabsf(conviction_signed);
|
||||
const int conviction_dir = (conviction_signed >= 0.0f) ? 1 : -1;
|
||||
|
||||
// Wiener-α EMA on the multi-horizon conviction (reuses existing slots).
|
||||
const float conv_ema_abs = update_conviction_ema(
|
||||
b, conviction_abs, conviction_ema_per_b,
|
||||
conviction_diff_var_ema_per_b, conviction_sample_var_ema_per_b
|
||||
);
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Replace target sizing path**
|
||||
|
||||
The old per-horizon-summed `final_size` with the broken rescale is replaced. New target formula:
|
||||
|
||||
```c
|
||||
// CRT.1: target_lots = direction × |conviction_ema| × envelope_max.
|
||||
// envelope_max is the per-backtest max_lots (Layer C makes this adaptive).
|
||||
const float envelope_max = (float)max_lots_per_b[b];
|
||||
const float target_lots_f = (float)conviction_dir * conv_ema_abs * envelope_max;
|
||||
const int target_lots = (int)truncf(target_lots_f + (target_lots_f >= 0.0f ? 0.5f : -0.5f));
|
||||
|
||||
if (target_lots == 0) {
|
||||
market_targets[b * 2 + 0] = 3; // force-flat
|
||||
market_targets[b * 2 + 1] = 0;
|
||||
} else if (target_lots > 0) {
|
||||
market_targets[b * 2 + 0] = 0; // buy
|
||||
market_targets[b * 2 + 1] = target_lots;
|
||||
} else {
|
||||
market_targets[b * 2 + 0] = 1; // sell
|
||||
market_targets[b * 2 + 1] = -target_lots;
|
||||
}
|
||||
```
|
||||
|
||||
DELETE the old per-horizon `final_size` accumulation loop and the `final_size *= scale` rescale.
|
||||
|
||||
Apply the same change to BOTH `decision_policy_default` AND `decision_policy_program` (the bytecode VM `OP_WRITE_ORDER` site).
|
||||
|
||||
- [ ] **Step 4: Delete obsolete tests; run new test**
|
||||
|
||||
Remove the three obsolete tests from `stop_controller.rs`:
|
||||
- `conviction_ema_smooths_micro_oscillations` (assertions assume old scalar path)
|
||||
- `conviction_ema_does_not_lag_reversals` (same)
|
||||
- `conviction_ema_rescale_never_amplifies_weak_signal` (rescale is gone)
|
||||
|
||||
Keep `conviction_ema_does_not_lag_reversals` ONLY if you can rewrite it to assert on the multi-horizon formula without ISV bootstrap dependencies; otherwise delete.
|
||||
|
||||
```bash
|
||||
SQLX_OFFLINE=true cargo test -p ml-backtesting --test stop_controller multi_horizon_conviction_cancels_on_disagreement -- --ignored --nocapture 2>&1 | tail -10
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "$(cat <<'EOF'
|
||||
arch(crt-1): multi-horizon ISV-weighted conviction replaces scalar EMA approach
|
||||
|
||||
Per spec §4.4 (promoted from v2 Phase B). Replaces v2 A2 (1d889d2de) +
|
||||
A2.1 (fe2498769) scalar approach that failed Gate 1 catastrophically on
|
||||
smokes vjmwc + lkrdf (155k trades, 9100% drawdown).
|
||||
|
||||
Formula:
|
||||
weight_h = max(isv[h].net_edge, ε) / (isv[h].var + cost²)
|
||||
weighted_h = magnitude_h × weight_h × direction_h
|
||||
conviction_signed = sum(weighted_h) / sum(|weight_h|)
|
||||
conviction_ema = Wiener-α EMA(|conviction_signed|) (reuses A2 slots)
|
||||
target_lots = round(sign(conviction_signed) × conviction_ema × max_lots)
|
||||
|
||||
Direction is computed from the per-horizon signed sum, so disagreeing
|
||||
horizons CANCEL — addresses the v2 Gate-1 root cause (scalar EMA on
|
||||
max(|p-0.5|) couldn't smooth direction jitter).
|
||||
|
||||
Per-horizon weights are ISV-derived per pearl_controller_anchors_isv_driven
|
||||
and pearl_one_unbounded_signal_per_reward (single unbounded factor =
|
||||
net_edge/(var+cost²); magnitude_h × direction_h ∈ [-1, +1]).
|
||||
|
||||
The aggregate `final_size *= scale` rescale step is DELETED — that was the
|
||||
v2 bug that amplified weak signals when conv_ema > raw_max_conv. The new
|
||||
formula multiplies smoothed conviction directly with envelope.
|
||||
|
||||
Tests:
|
||||
- Added: multi_horizon_conviction_cancels_on_disagreement (passes)
|
||||
- Removed: conviction_ema_smooths_micro_oscillations (obsolete scalar path)
|
||||
- Removed: conviction_ema_does_not_lag_reversals (obsolete scalar path)
|
||||
- Removed: conviction_ema_rescale_never_amplifies_weak_signal (rescale gone)
|
||||
|
||||
Three conviction-EMA device slots (conviction_ema_d et al.) STAY in
|
||||
LobSimCuda; the EMA mechanism is reused on the multi-horizon scalar.
|
||||
|
||||
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||||
EOF
|
||||
)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task C1.3: No-trade band in `seed_inflight_limits_batched`
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/ml-backtesting/cuda/resting_orders.cu` — `seed_inflight_limits_batched` signature + body
|
||||
- Modify: `crates/ml-backtesting/src/sim/mod.rs` — add `delta_floor_d` device slot, thread into launch
|
||||
- Modify: `crates/ml-backtesting/src/sim/batched_config.rs` — `delta_floor: Vec<f32>` per-variant + upload helper
|
||||
- Modify: `bin/fxt-backtest/src/main.rs` — `SweepBase.delta_floor: Option<f32>` default 1.0, threaded into `ResolvedSimVariant`
|
||||
- Modify: `config/ml/sweep_smoke.yaml` — `delta_floor: 1.0`
|
||||
- Tests: `crates/ml-backtesting/tests/stop_controller.rs`
|
||||
|
||||
- [ ] **Step 1: Write failing test for no-trade band**
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
#[ignore = "requires CUDA"]
|
||||
fn no_trade_band_blocks_micro_delta() -> Result<()> {
|
||||
let dev = match MlDevice::cuda(0) { Ok(d) => d, Err(e) => { eprintln!("skip: {e}"); return Ok(()); }};
|
||||
let mut sim = LobSimCuda::new(1, &dev)?;
|
||||
sim.upload_price_range(&[1000.0], &[20000.0])?;
|
||||
sim.upload_delta_floor(&[1.0])?;
|
||||
let cfg = cfg_default();
|
||||
|
||||
// Set target=1 lot (from a small alpha bump). With delta_floor=1.0 and
|
||||
// current_position=0, |delta|=1 — at the band boundary. Increment fires.
|
||||
// Then with target=1 (no change), |delta|=0 → no fire.
|
||||
let probs: [f32; N_HORIZONS] = [0.55; N_HORIZONS];
|
||||
sim.broadcast_alpha(&probs)?;
|
||||
sim.step_decision_with_latency(1_000_000_000u64, &cfg)?;
|
||||
let inflight_after_first = sim.read_inflight_count(0)?; // existing or add helper
|
||||
assert!(inflight_after_first >= 1, "first decision must seed");
|
||||
|
||||
// Same target again — delta=0 → no new seeding.
|
||||
sim.broadcast_alpha(&probs)?;
|
||||
sim.step_decision_with_latency(2_000_000_000u64, &cfg)?;
|
||||
let inflight_after_second = sim.read_inflight_count(0)?;
|
||||
assert_eq!(inflight_after_second, inflight_after_first,
|
||||
"no new order should seed when delta=0");
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
If `read_inflight_count` doesn't exist, add it as a public accessor (sums `active != 0` across the orders.limits[] array).
|
||||
|
||||
- [ ] **Step 2: Add `delta_floor` device slot + uploader**
|
||||
|
||||
In `crates/ml-backtesting/src/sim/mod.rs`:
|
||||
```rust
|
||||
pub(crate) delta_floor_d: CudaSlice<f32>, // [n_backtests] — no-trade band, lots
|
||||
```
|
||||
Allocate via `alloc_zeros` in `new()`. Add a method:
|
||||
```rust
|
||||
pub fn upload_delta_floor(&mut self, values: &[f32]) -> Result<()> {
|
||||
anyhow::ensure!(values.len() == self.n_backtests, "delta_floor length mismatch");
|
||||
self.stream.memcpy_htod(values, &mut self.delta_floor_d)?;
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
In `batched_config.rs`: add `delta_floor: Vec<f32>` per-variant + a `from_*` method that sets it.
|
||||
|
||||
In `fxt-backtest/src/main.rs`: add `delta_floor: Option<f32>` to `SweepBase` (default 1.0) and `ResolvedSimVariant`. Thread through `resolve_variants`.
|
||||
|
||||
- [ ] **Step 3: Add the gate to `seed_inflight_limits_batched`**
|
||||
|
||||
In `crates/ml-backtesting/cuda/resting_orders.cu`, the kernel currently computes `delta = target_signed - effective` and seeds an order if `delta != 0`. Add the band check:
|
||||
|
||||
```c
|
||||
const int delta = target_signed - effective;
|
||||
|
||||
// CRT.1: no-trade band. Skip seeding when delta is smaller than the
|
||||
// configured floor — most events produce fractional target changes
|
||||
// that aren't worth the spread. Per spec §4.0 (v3 architecture reset).
|
||||
const float delta_floor = delta_floor_per_b[b];
|
||||
const float abs_delta_f = (delta > 0) ? (float)delta : (float)(-delta);
|
||||
if (abs_delta_f < delta_floor) return;
|
||||
|
||||
if (delta == 0) return;
|
||||
```
|
||||
|
||||
Thread `delta_floor_per_b` param through the kernel signature and the Rust launch in `sim/mod.rs`.
|
||||
|
||||
- [ ] **Step 4: Update the smoke YAML**
|
||||
|
||||
In `config/ml/sweep_smoke.yaml`:
|
||||
```yaml
|
||||
base:
|
||||
...
|
||||
delta_floor: 1.0
|
||||
...
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run the failing test**
|
||||
|
||||
```bash
|
||||
SQLX_OFFLINE=true cargo test -p ml-backtesting --test stop_controller no_trade_band_blocks_micro_delta -- --ignored --nocapture 2>&1 | tail -10
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 6: Compile workspace + full test suite**
|
||||
|
||||
```bash
|
||||
SQLX_OFFLINE=true cargo check --workspace 2>&1 | tail -5
|
||||
SQLX_OFFLINE=true cargo test -p ml-backtesting --test stop_controller -- --ignored 2>&1 | tail -5
|
||||
```
|
||||
|
||||
Expected: all pass.
|
||||
|
||||
- [ ] **Step 7: Commit**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "$(cat <<'EOF'
|
||||
arch(crt-1): no-trade band in seed_inflight — sparse action at signal cadence
|
||||
|
||||
Per spec §4.0 (v3 architecture reset). Without this gate, every fractional
|
||||
change in target_lots seeds an order via target-delta semantics; combined
|
||||
with continuous controller invocation (every event) this generates
|
||||
hyperactivity even with the multi-horizon conviction formula. v2 Gate-1
|
||||
failures confirmed: scalar conviction-EMA + amplification-clamp + per-event
|
||||
controller invocation = 155k trades on a 2M-event smoke.
|
||||
|
||||
The band absorbs fractional target adjustments so most events produce no
|
||||
order. When the signal genuinely shifts (conviction crosses a meaningful
|
||||
threshold OR direction flips OR position needs to scale), |delta| crosses
|
||||
delta_floor and the kernel seeds. Continuous evaluation, sparse action —
|
||||
the core CRT.1 idea.
|
||||
|
||||
New per-backtest config: `delta_floor: f32` (default 1.0 lot). Threaded
|
||||
through SweepBase → ResolvedSimVariant → BatchedSimConfig → device slot.
|
||||
Kernel skips when |target_signed − effective| < delta_floor.
|
||||
|
||||
Test: no_trade_band_blocks_micro_delta verifies same-target repeat does
|
||||
not seed additional orders.
|
||||
|
||||
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||||
EOF
|
||||
)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task C1.4: Conviction-degradation safety circuit-breaker (spec §4.3)
|
||||
|
||||
This task adds the composite exit_signal as a SAFETY layer (not the primary exit mechanism — that's emergent from target → 0 via Tasks C1.2 + C1.3). The composite handles the edge case where conviction stays positive but the trade is deep in unrealized loss.
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/ml-backtesting/cuda/decision_policy.cu` — `stop_check_isv` extended with composite-signal logic
|
||||
- Tests: add `composite_exit_fires_on_pnl_decay`
|
||||
|
||||
- [ ] **Step 1: Write failing test**
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
#[ignore = "requires CUDA"]
|
||||
fn composite_exit_fires_on_pnl_decay() -> Result<()> {
|
||||
// Scenario: position open with positive conviction (no degradation), but
|
||||
// unrealized PnL deep negative. composite exit_signal pnl_decay_term should
|
||||
// dominate and force exit.
|
||||
// ...
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Implement the composite signal in `stop_check_isv`**
|
||||
|
||||
Per spec §4.3, the formula is:
|
||||
```c
|
||||
const float degradation_term = (conviction_ema_at_entry > 1e-6f)
|
||||
? (1.0f - conviction_ema / conviction_ema_at_entry) / degradation_factor
|
||||
: 0.0f;
|
||||
const float disagreement_term =
|
||||
(float)disagreement_consecutive_events / disagreement_factor;
|
||||
const float pnl_decay_term =
|
||||
fmaxf(-pnl_adjusted_conviction_ema, 0.0f) / pnl_decay_factor;
|
||||
|
||||
const float exit_signal = fmaxf(fmaxf(degradation_term, disagreement_term), pnl_decay_term);
|
||||
if (exit_signal >= 1.0f) {
|
||||
market_targets[b * 2 + 0] = 3; // force-flat
|
||||
market_targets[b * 2 + 1] = 0;
|
||||
return 1;
|
||||
}
|
||||
```
|
||||
|
||||
`degradation_factor`, `disagreement_factor`, `pnl_decay_factor` are ISV-derived (spec §4.3). Bootstrap defaults: `degradation_factor = 2.0`, `disagreement_factor = 32.0`, `pnl_decay_factor = 1.0`.
|
||||
|
||||
Read the needed state from the expanded `open_trade_state`:
|
||||
- `conviction_at_entry` at offset 20
|
||||
- `pnl_adjusted_conviction_ema` at offset 44
|
||||
- `disagreement_consecutive_events` at offset 56
|
||||
|
||||
- [ ] **Step 3: Update open_trade_state writes to track these states**
|
||||
|
||||
In `pnl_track.cu` open branch: write `conviction_at_entry = conviction_ema_d[b]` at offset 20. Initialize the trajectory fields to 0.
|
||||
|
||||
In `decision_policy.cu` after computing conviction_signed/conviction_ema: if pos is open, update `pnl_adjusted_conviction_ema` and `disagreement_consecutive_events` in `open_trade_state`. This requires writing to open_trade_state from the decision kernel — extend its signature with `unsigned char* open_trade_state_per_b`.
|
||||
|
||||
- [ ] **Step 4: Run failing test, verify pass**
|
||||
|
||||
```bash
|
||||
SQLX_OFFLINE=true cargo test -p ml-backtesting --test stop_controller composite_exit_fires_on_pnl_decay -- --ignored --nocapture 2>&1 | tail -10
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "fix(crt-1): composite exit_signal safety circuit-breaker per spec §4.3"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task C1.5: Local compile + full test suite
|
||||
|
||||
**Files:** none (verification only)
|
||||
|
||||
- [ ] **Step 1: Full workspace compile**
|
||||
```bash
|
||||
SQLX_OFFLINE=true cargo check --workspace 2>&1 | tail -5
|
||||
```
|
||||
Expected: clean.
|
||||
|
||||
- [ ] **Step 2: Full test suite**
|
||||
```bash
|
||||
SQLX_OFFLINE=true cargo test -p ml-backtesting --lib 2>&1 | tail -5
|
||||
SQLX_OFFLINE=true cargo test -p ml-backtesting --test stop_controller -- --ignored 2>&1 | tail -5
|
||||
SQLX_OFFLINE=true cargo test -p ml-backtesting --test decision_floor_coldstart -- --ignored 2>&1 | tail -5
|
||||
SQLX_OFFLINE=true cargo test -p ml-alpha --test forward_step_golden -- --ignored 2>&1 | tail -5
|
||||
```
|
||||
All pass.
|
||||
|
||||
---
|
||||
|
||||
### Task C1.6: Cluster smoke + Gate CRT.1 validation
|
||||
|
||||
**Files:** none (deployment + monitoring)
|
||||
|
||||
- [ ] **Step 1: Push**
|
||||
```bash
|
||||
git push origin ml-alpha-phase-a
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Submit cluster smoke**
|
||||
```bash
|
||||
SHA=$(git rev-parse --short=9 HEAD)
|
||||
./scripts/argo-lob-sweep.sh --branch ml-alpha-phase-a --sha $SHA --grid config/ml/sweep_smoke.yaml --sweep-tag sweep_smoke-$SHA
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Monitor to completion** (~13-16 min)
|
||||
|
||||
- [ ] **Step 4: Pull metrics + validate against Gate CRT.1 (= v2 §9 Gate 2 tiered MUST/WIN)**
|
||||
|
||||
| Tier | Criterion | Threshold |
|
||||
|---|---|---|
|
||||
| MUST | hold-time emergent (max_hold circuit-breaker fires < 5% of closes) | yes |
|
||||
| MUST | Sharpe ≥ baseline (tgg4l) − 0.5 units | ≥ −7.14 |
|
||||
| MUST | PF ≥ baseline × 0.95 | ≥ 0.38 |
|
||||
| MUST | drawdown ≤ baseline × 1.10 | ≤ 58.9% |
|
||||
| MUST | conviction-degradation accounts for ≥ 60% of closes | yes (via attribution) |
|
||||
| MUST | no new NaN counters | nan_counters all 0 |
|
||||
| WIN | Sharpe lift ≥ +0.5 units | OR |
|
||||
| WIN | PF lift ≥ 30% | OR (PF ≥ 0.52) |
|
||||
| WIN | alpha-realization ratio lift ≥ +20% | OR |
|
||||
| WIN | drawdown reduction ≥ 20% at equal PnL | OR |
|
||||
|
||||
- [ ] **Step 5: Write project memory entry**
|
||||
|
||||
Save `project_crt_1_gate1.md` with verdict + numbers + commit SHAs.
|
||||
|
||||
- [ ] **Step 6: Decide outcome**
|
||||
- All MUST + ≥ 1 WIN → Gate CRT.1 GREEN. Plan CRT.2.
|
||||
- All MUST but no WIN → STOP, re-evaluate whether the signal can carry this anchor.
|
||||
- Any MUST fails → Gate RED. Diagnose with per-event instrumentation. Do NOT advance.
|
||||
|
||||
---
|
||||
|
||||
## Notes for the Implementer
|
||||
|
||||
- **No fallbacks** per user direction. If a test helper doesn't exist, add it. If the kernel needs new params, add them. If the YAML schema needs a field, add it. Greenfields atomic refactor.
|
||||
- **No memcpy_htod/dtoh/dtov/synchronize in the per-event hot path.** Only `memcpy_dtod_async`. Test accessors that do one-shot DtoH at end of run are OK.
|
||||
- **Each task is one commit.** No mid-task commits.
|
||||
- **TDD discipline.** Each task starts with a failing test.
|
||||
- **rust-analyzer's "expected 7 args, found 2" is stale cache** — verify with `cargo check` directly.
|
||||
|
||||
### Pearl conformance
|
||||
|
||||
- C1.1: [feedback_single_source_of_truth_no_duplicates](../../../../home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/feedback_single_source_of_truth_no_duplicates.md), [feedback_no_partial_refactor](../../../../home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/feedback_no_partial_refactor.md)
|
||||
- C1.2: [pearl_controller_anchors_isv_driven](../../../../home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/pearl_controller_anchors_isv_driven.md), [pearl_one_unbounded_signal_per_reward](../../../../home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/pearl_one_unbounded_signal_per_reward.md), [pearl_zscore_normalization_for_magnitude_asymmetric_signals](../../../../home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/pearl_zscore_normalization_for_magnitude_asymmetric_signals.md), [pearl_trade_level_vol_for_stop_distance](../../../../home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/pearl_trade_level_vol_for_stop_distance.md), [pearl_wiener_optimal_adaptive_alpha](../../../../home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/pearl_wiener_optimal_adaptive_alpha.md), [pearl_wiener_alpha_floor_for_nonstationary](../../../../home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/pearl_wiener_alpha_floor_for_nonstationary.md), [pearl_blend_formulas_must_have_permanent_floor](../../../../home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/pearl_blend_formulas_must_have_permanent_floor.md)
|
||||
- C1.3: spec §4.0 (v3 reset note on no-trade band)
|
||||
- C1.4: [pearl_event_driven_reward_density_alignment](../../../../home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/pearl_event_driven_reward_density_alignment.md) — exit decisions match event density
|
||||
- C1.6: [feedback_push_before_deploy](../../../../home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/feedback_push_before_deploy.md), [feedback_stop_on_anomaly](../../../../home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/feedback_stop_on_anomaly.md)
|
||||
|
||||
---
|
||||
|
||||
## Phase CRT.2 / CRT.3 forward references
|
||||
|
||||
After Gate CRT.1 closes green:
|
||||
|
||||
- **`docs/superpowers/plans/2026-05-XX-crt-phase-2-adaptive-envelope.md`** — ISV-derived envelope, conviction-percentile threshold gate (3-arm bandit), regime-vol-inverse vol target.
|
||||
|
||||
- **`docs/superpowers/plans/2026-05-XX-crt-phase-3-online-adaptation.md`** — LoRA + EWC++ + trajectory-batch updates + shadow eval + circuit-breaker revert.
|
||||
|
||||
Each gate is a stop. Plan next phase only after prior phase passes.
|
||||
Reference in New Issue
Block a user