From 235961987d09a21f4153d9bbb231b284fbb0ccaf Mon Sep 17 00:00:00 2001 From: jgrusewski Date: Wed, 20 May 2026 17:33:16 +0200 Subject: [PATCH] plan(crt-a): Phase A continuous controller implementation plan MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per docs/superpowers/specs/2026-05-20-continuous-reasoning-trader-design.md (v2, commit 8ba8755b4). One plan per phase per writing-plans guidance — Phase B/C/D plans written after each prior gate closes. Phase A scope: A0 - Investigate forward_only cost model (read-only research, writes memo to docs/superpowers/memos/). STOP condition if trunk forward is stateless K-window per call (Case 2) — requires user approval to expand scope. A1 - Atomic delete of decision_stride (greenfields, no compat). Removes from SweepBase, ResolvedSimVariant, BatchedSimConfig, BacktestHarness, all YAMLs. A2 - Minimal Wiener-α conviction-EMA smoothing in decision_policy_* kernels. Three new device slots: conviction_ema_d, conviction_diff_var_ema_d, conviction_sample_var_ema_d. Floor at 0.4 per pearl_wiener_alpha_floor_for_nonstationary. First-observation bootstrap. Direction stays from raw alpha; magnitude/sizing uses smoothed value. A3 - Local compile + tests. A4 - Cluster smoke + Gate 1 validation against spec §3.6 acceptance. Memory entry on closure (green/red/yellow). A5 - Conditional hyperactivity mitigation (raise floor 0.4→0.6 OR add target-delta hysteresis). Fires only if Gate 1 reveals trade-count balloon. Out of scope (deferred): full §4.4 multi-horizon conviction formula, §4.3 conviction-degradation exit, §5 envelope, §6 LoRA adapter, §7 open_trade_state expansion — all Phase B/C/D. Pearl conformance: - feedback_no_partial_refactor (atomic delete) - feedback_no_legacy_aliases (no compat shim) - feedback_push_before_deploy - pearl_wiener_optimal_adaptive_alpha - pearl_wiener_alpha_floor_for_nonstationary - pearl_first_observation_bootstrap - feedback_stop_on_anomaly (Gate 1 RED → diagnose, don't advance) Co-Authored-By: Claude Opus 4.7 (1M context) --- ...05-20-crt-phase-a-continuous-controller.md | 672 ++++++++++++++++++ 1 file changed, 672 insertions(+) create mode 100644 docs/superpowers/plans/2026-05-20-crt-phase-a-continuous-controller.md diff --git a/docs/superpowers/plans/2026-05-20-crt-phase-a-continuous-controller.md b/docs/superpowers/plans/2026-05-20-crt-phase-a-continuous-controller.md new file mode 100644 index 000000000..d2ffb7967 --- /dev/null +++ b/docs/superpowers/plans/2026-05-20-crt-phase-a-continuous-controller.md @@ -0,0 +1,672 @@ +# Continuous-Reasoning Trader — Phase A Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Deliver Layer A from the Continuous-Reasoning Trader spec — continuous (event-rate) controller invocation, replacing the current `decision_stride`-gated discrete policy. Validate via cluster smoke that infrastructure works at ≤ 2× wall-time with no signal-quality regression. + +**Architecture:** Replace the `if event_count % stride == 0` gate in `BacktestHarness::run` with unconditional per-event invocation. Investigate whether `PerceptionTrainer::forward_only` supports incremental state advance (cheap per-event) or requires a refactor (per-event K-window reprocessing would be 200× more expensive — unacceptable per spec §3.4). Apply minimal anti-hyperactivity smoothing (conviction-EMA in the controller) so Phase A is behaviorally sensible, then validate Gate 1. + +**Tech Stack:** Rust 1.85, CUDA 12.4, Tokio, sqlx (offline), Mamba2/Cfc trunk via `PerceptionTrainer`, ISV state per-horizon, cudarc 0.19, Argo Workflows for cluster validation. + +**Spec:** `docs/superpowers/specs/2026-05-20-continuous-reasoning-trader-design.md` (v2, commit `8ba8755b4`) + +**Branch:** `ml-alpha-phase-a` (HEAD: `8ba8755b4`, S2 closed at `8828e8ab1`) + +--- + +## Scope contract + +**In scope (Phase A):** +- Investigate `forward_only` cost model +- Remove `decision_stride` field (greenfields atomic) per spec §3.3 + §8 +- Remove stride gate from `BacktestHarness::run` +- Minimal anti-hyperactivity smoothing (conviction-EMA in controller — small subset of spec §4.2 that ships now to keep Gate 1 measurable) +- Cluster smoke validation against Gate 1 acceptance criteria + +**Out of scope (deferred to Phase B/C/D plans):** +- Full multi-horizon conviction formula (spec §4.4 — only the EMA smoothing of the existing scalar conviction is in Phase A) +- Conviction-degradation exit (spec §4.3) +- Adaptive envelope/threshold/vol-target (spec §5) +- Online weight adaptation (spec §6) +- `open_trade_state` 24→64 byte expansion (spec §7) + +--- + +## File Structure + +**Modify:** +- `crates/ml-backtesting/src/harness.rs` — remove stride gate; remove `decision_stride` field; remove `stride` local at line 213 +- `crates/ml-backtesting/src/sim/batched_config.rs` — remove `decision_stride` field from `UniformSimParams`, `BatchedSimConfig`, `ResolvedSimVariant` +- `bin/fxt-backtest/src/main.rs` — remove `decision_stride` from `SweepBase`, `SweepVariant`, `ResolvedSimVariant`; remove `default_decision_stride` fn; update `resolve_variants` accordingly +- `crates/ml-backtesting/cuda/decision_policy.cu` — extend `stop_check_isv` and/or `decision_policy_default` to accept a conviction-EMA slot and apply Wiener-α smoothing before producing `market_targets` +- `crates/ml-backtesting/src/sim/mod.rs` — add `conviction_ema_d: CudaSlice`, alloc + accessor + threading into decision kernel launches +- `config/ml/sweep_smoke.yaml` — remove `decision_stride: 200` line +- `config/ml/*.yaml` — remove `decision_stride` from EVERY YAML that has it (one sweep, atomic) +- `crates/ml-backtesting/tests/stop_controller.rs` — adjust any test that constructs configs with `decision_stride` + +**Create:** +- `docs/superpowers/memos/2026-05-20-crt-a-forward-cost-investigation.md` — output of Task A0 investigation (created by the implementer with their findings) + +**Read-only (referenced but not modified):** +- `crates/ml-perception-trainer/src/lib.rs` (or wherever `PerceptionTrainer::forward_only` lives) — read to understand state model +- `docs/superpowers/specs/2026-05-20-continuous-reasoning-trader-design.md` — the spec + +--- + +## Phase A Tasks + +### Task A0: Investigate trunk forward pass cost model (research, no code changes) + +**Files:** +- Read: `crates/ml-perception-trainer/src/**/*.rs` (find `PerceptionTrainer::forward_only`) +- Read: `crates/ml-alpha/src/**/*.rs` (Mamba2 trunk + Cfc trunk forward path) +- Create: `docs/superpowers/memos/2026-05-20-crt-a-forward-cost-investigation.md` + +- [ ] **Step 1: Locate `PerceptionTrainer::forward_only` and trace its compute path** + +Run: +```bash +grep -rn "fn forward_only" crates/ml-perception-trainer/ crates/ml-alpha/ +``` +Read the function. Understand what it does with the input window. Note specifically: +- Does it maintain trunk state across calls, or rebuild from window each time? +- What is the per-call FLOPs cost dominated by? (Mamba2 SSM scan? Cfc gates? Output head?) +- What is the relationship between input window length and output length? + +- [ ] **Step 2: Trace the Mamba2 SSM state path** + +Find the Mamba2 forward kernel(s) in `crates/ml-alpha/cuda/` or wherever they live. Determine if state is maintained across `forward_only` calls or re-initialized each call. + +```bash +grep -rn "mamba2\|ssm_state\|forward_sequential" crates/ml-alpha/ crates/ml-perception-trainer/ | head -20 +``` + +- [ ] **Step 3: Estimate the cost ratio** + +Compute the actual cost ratio if `forward_only` is called every event vs every 200 events. Three possible cases: + +| Case | If forward_only is | Per-event cost vs stride-200 | +|---|---|---| +| 1 | Stateful, advances 1 step per call | ~1× (cheap incremental) | +| 2 | Stateless, processes K-window per call, state derived from window | ~200× (expensive — runs full K window every event) | +| 3 | Stateful but with periodic state reset | Between 1× and 200× depending on reset frequency | + +- [ ] **Step 4: Write the investigation memo** + +Create `docs/superpowers/memos/2026-05-20-crt-a-forward-cost-investigation.md` with: +- Which case (1/2/3) applies +- The relevant files and key functions +- Estimated cost ratio +- Recommended Phase A implementation path: + - **If Case 1:** simple — Phase A just removes the stride gate. Cost target ≤ 2× should hold. + - **If Case 2:** Phase A needs `forward_only_incremental` first (advance trunk state by 1 step using cached prior state). This becomes Task A1.5 added to this plan. + - **If Case 3:** Phase A needs to relax cost target OR add state caching. Recommend approach. + +- [ ] **Step 5: Commit the memo** + +```bash +git add docs/superpowers/memos/2026-05-20-crt-a-forward-cost-investigation.md +git commit -m "memo(crt-a): forward pass cost investigation — chose [Case 1/2/3]" +``` + +**Stop condition:** If the memo recommends adding `forward_only_incremental` (Case 2), pause and notify the user — they should approve the scope expansion before proceeding. If Case 1 or Case 3-with-trivial-mitigation, proceed to Task A1. + +--- + +### Task A1: Delete `decision_stride` field (greenfields atomic) + +**Files:** +- Modify: `bin/fxt-backtest/src/main.rs` (SweepBase, SweepVariant, ResolvedSimVariant, default_decision_stride, resolve_variants) +- Modify: `crates/ml-backtesting/src/harness.rs` (BacktestHarness::cfg field, BacktestHarness::run line 213 `stride` local, line 242 conditional) +- Modify: `crates/ml-backtesting/src/sim/batched_config.rs` (UniformSimParams, BatchedSimConfig, ResolvedSimVariant if present) +- Modify: `config/ml/sweep_smoke.yaml` and any other YAML referencing the field +- Modify: `crates/ml-backtesting/tests/stop_controller.rs` if any test constructs configs with decision_stride + +- [ ] **Step 1: Find every reference to decision_stride in the workspace** + +```bash +grep -rn "decision_stride\|default_decision_stride" --include="*.rs" --include="*.yaml" --include="*.cu" . +``` + +Note every occurrence. Per [feedback_no_partial_refactor](../../../../home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/feedback_no_partial_refactor.md): every consumer migrates atomically in one commit. + +- [ ] **Step 2: Remove from `SweepBase` and `SweepVariant` structs** + +In `bin/fxt-backtest/src/main.rs`: +- Find the `SweepBase` struct. Remove the `decision_stride: usize` field (or `Option` if optional). +- Find the `SweepVariant` struct. Remove any `decision_stride` override field. +- Find `default_decision_stride()`. Delete the function entirely. +- Find `resolve_variants` (or the equivalent merge function around line 410). Remove the line that copies decision_stride. + +- [ ] **Step 3: Remove from `BatchedSimConfig`** + +In `crates/ml-backtesting/src/sim/batched_config.rs`: +- Find `UniformSimParams` — remove `decision_stride` field. +- Find `BatchedSimConfig` — remove `decision_stride: Vec` field if present (it shouldn't be per-backtest, but check). +- Find `ResolvedSimVariant` — remove `decision_stride` field. +- Update the `new`/`from_*` constructors to not initialize the removed field. + +- [ ] **Step 4: Remove from harness loop** + +In `crates/ml-backtesting/src/harness.rs`: +- Find the `BacktestHarness::cfg` field reference around line 138: `decision_stride: cfg.decision_stride`. Remove from the cfg builder. +- In `run()` line 213: remove `let stride = self.cfg.decision_stride.max(1) as u64;` +- In `run()` line 242: change `if self.event_count % stride == 0 && self.snapshot_window.len() == self.seq_len {` to `if self.snapshot_window.len() == self.seq_len {` + +- [ ] **Step 5: Remove from YAML configs** + +```bash +for f in config/ml/*.yaml; do + grep -l "decision_stride" "$f" && sed -i '/^ decision_stride:/d' "$f" +done +``` + +Verify with: +```bash +grep -rn "decision_stride" config/ml/ +``` +Expected: no output. + +- [ ] **Step 6: Update tests if needed** + +Run: +```bash +grep -rn "decision_stride" crates/ml-backtesting/tests/ +``` +For each match: remove the field initializer or update the test helper to not pass it. + +- [ ] **Step 7: Compile workspace** + +```bash +SQLX_OFFLINE=true cargo check --workspace 2>&1 | tail -20 +``` +Expected: clean compile, no warnings about unused `decision_stride`. If there are warnings, you missed a reference — go back to Step 1. + +- [ ] **Step 8: Run existing tests to confirm nothing broke** + +```bash +SQLX_OFFLINE=true cargo test -p ml-backtesting --lib 2>&1 | tail -10 +SQLX_OFFLINE=true cargo test -p ml-backtesting --test stop_controller -- --ignored --nocapture 2>&1 | tail -10 +``` + +Expected: all pass. The stride was set to 200 in smoke YAML but that was just the gate frequency — removing it changes the loop to fire every event, which means the existing tests (which use small event counts) still work, just with more decisions per test event. + +- [ ] **Step 9: Commit** + +```bash +git add -A +git commit -m "$(cat <<'EOF' +arch(crt-a): delete decision_stride field — greenfields atomic refactor + +Per spec 2026-05-20-continuous-reasoning-trader-design.md §3.3 and §8: +decision_stride is REMOVED, not deprecated. No backwards-compat shim. +Every consumer migrates in this commit per +feedback_no_partial_refactor. + +Removed from: + - bin/fxt-backtest SweepBase / SweepVariant / ResolvedSimVariant / default_decision_stride + - crates/ml-backtesting/src/sim/batched_config.rs UniformSimParams / BatchedSimConfig / ResolvedSimVariant + - crates/ml-backtesting/src/harness.rs BacktestHarness.cfg.decision_stride + run loop stride gate + - config/ml/*.yaml every decision_stride: line + - tests: stop_controller suite updated where applicable + +After this commit step_decision fires every event whenever the snapshot +window is full. forward_only cost behaviour determined by Task A0 memo; +mitigation applied in subsequent tasks if needed. + +Co-Authored-By: Claude Opus 4.7 (1M context) +EOF +)" +``` + +--- + +### Task A2: Add conviction-EMA anti-hyperactivity smoothing (minimal Layer B subset) + +This task ships a SMALL part of spec §4.2 to keep Phase A behaviorally sensible. Without smoothing, removing the stride gate likely produces hyperactive target oscillation as alpha jiggles event-to-event. The full multi-horizon conviction formula is Phase B — this task implements ONLY the scalar conviction EMA on whatever the controller currently uses (typically max conviction across horizons). + +**Files:** +- Modify: `crates/ml-backtesting/cuda/decision_policy.cu` (`decision_policy_default` and/or `decision_policy_program`) +- Modify: `crates/ml-backtesting/src/sim/mod.rs` (add `conviction_ema_d: CudaSlice` field, alloc, thread into decision launches) +- Modify: `crates/ml-backtesting/tests/stop_controller.rs` (add test) + +- [ ] **Step 1: Write failing test for the EMA smoothing** + +Append to `crates/ml-backtesting/tests/stop_controller.rs`: + +```rust +#[test] +#[ignore = "requires CUDA"] +fn conviction_ema_smooths_micro_oscillations() -> Result<()> { + let dev = match MlDevice::cuda(0) { + Ok(d) => d, + Err(e) => { eprintln!("skipping: cuda unavailable ({e})"); return Ok(()); } + }; + let mut sim = LobSimCuda::new(1, &dev)?; + sim.upload_price_range(&[1000.0], &[20000.0])?; + + // Drive 10 alternating high/low convictions. Without EMA, target lots + // would oscillate. With EMA (Wiener-α floor = 0.4), target should + // damp toward the mean of the two. + let alphas_high = [0.8_f32, 0.8, 0.8, 0.8]; // strongly bullish all horizons + let alphas_low = [0.55_f32, 0.55, 0.55, 0.55]; // weakly bullish + + let mut prev_target: Option = None; + let mut flips = 0; + + for i in 0..10 { + let probs = if i % 2 == 0 { alphas_high } else { alphas_low }; + sim.broadcast_alpha(&probs)?; + sim.step_decision_with_latency(1_000_000_000u64 * (i + 1) as u64, &/*sim_config*/test_sim_config())?; + let target = sim.read_market_target(0)?; + if let Some(p) = prev_target { + if (target > 0) != (p > 0) && target != 0 && p != 0 { flips += 1; } + } + prev_target = Some(target); + } + // With Wiener-α floor 0.4, EMA on alternating high/low produces a + // smooth trajectory — flip count should be 0 or 1, NOT 10. + assert!(flips <= 2, "conviction EMA must smooth oscillations: flips={}", flips); + Ok(()) +} + +fn test_sim_config() -> BatchedSimConfig { + // Minimal config for the test. Take from existing helpers if present. + todo!("use existing test config helper if available; otherwise hand-build") +} +``` + +**Note:** If existing test helpers like `read_market_target` and config builders aren't present in `LobSimCuda`'s public API, **omit this unit test and rely on the cluster smoke at Task A4 as the validation gate**. Don't invent helpers — that's scope creep. + +```bash +SQLX_OFFLINE=true cargo test -p ml-backtesting --test stop_controller conviction_ema_smooths_micro_oscillations -- --ignored --nocapture 2>&1 | tail -10 +``` + +Expected (if test exists): FAIL with "conviction EMA must smooth oscillations". + +- [ ] **Step 2: Add `conviction_ema_d` device slot in `LobSimCuda`** + +In `crates/ml-backtesting/src/sim/mod.rs`: + +```rust +// Find the LobSimCuda struct fields. Add (alphabetical/logical position): +pub(crate) conviction_ema_d: CudaSlice, // [n_backtests] — Wiener-α EMA of |max conviction| +pub(crate) conviction_diff_var_ema_d: CudaSlice, // [n_backtests] — for adaptive α +pub(crate) conviction_sample_var_ema_d: CudaSlice, // [n_backtests] — for adaptive α +``` + +In `LobSimCuda::new()`, add allocations: + +```rust +let conviction_ema_d = stream + .alloc_zeros::(n_backtests) + .context("alloc conviction_ema_d")?; +let conviction_diff_var_ema_d = stream + .alloc_zeros::(n_backtests) + .context("alloc conviction_diff_var_ema_d")?; +let conviction_sample_var_ema_d = stream + .alloc_zeros::(n_backtests) + .context("alloc conviction_sample_var_ema_d")?; +``` + +Add to the struct initializer in `Self { ... }`. Add to any `read_*_state` helpers if a regression test needs to read it. + +- [ ] **Step 3: Add Wiener-α EMA logic to `decision_policy_default`** + +In `crates/ml-backtesting/cuda/decision_policy.cu`: + +Add to `decision_policy_default` (and `decision_policy_program` if it exists separately) **before** the target sizing logic: + +```c +// S2/CRT-A: Wiener-α adaptive EMA on max conviction to smooth event-rate +// noise without losing responsiveness to genuine signal changes. +// alpha = diff_var / (diff_var + sample_var + eps); floor at 0.4 per +// pearl_wiener_alpha_floor_for_nonstationary. + +// 1. Compute current max conviction across horizons. +float max_conv = 0.0f; +#pragma unroll +for (int h = 0; h < N_HORIZONS; ++h) { + const float c = fabsf(alpha_probs[h] - 0.5f) * 2.0f; + if (c > max_conv) max_conv = c; +} + +// 2. Update diff_var EMA and sample_var EMA (fixed α=0.1 for these +// second-order EMAs; we only need adaptive α on the top-level signal). +const float prev_ema = conviction_ema[b]; +const float diff = max_conv - prev_ema; +const float diff_var_prev = conviction_diff_var_ema[b]; +const float diff_var_new = (diff_var_prev == 0.0f) + ? (diff * diff) + : (0.9f * diff_var_prev + 0.1f * diff * diff); +conviction_diff_var_ema[b] = diff_var_new; + +const float sample_var_prev = conviction_sample_var_ema[b]; +const float sample_var_new = (sample_var_prev == 0.0f) + ? (max_conv * max_conv) + : (0.9f * sample_var_prev + 0.1f * max_conv * max_conv); +conviction_sample_var_ema[b] = sample_var_new; + +// 3. Compute Wiener α with floor at 0.4 for non-stationary policy loop. +const float eps = 1e-6f; +const float alpha_raw = diff_var_new / (diff_var_new + sample_var_new + eps); +const float alpha_active = fmaxf(alpha_raw, 0.4f); + +// 4. Update EMA. First-observation bootstrap: prev_ema == 0 → replace directly. +const float new_ema = (prev_ema == 0.0f) + ? max_conv + : (alpha_active * max_conv + (1.0f - alpha_active) * prev_ema); +conviction_ema[b] = new_ema; + +// 5. Use smoothed_conviction for downstream gating instead of raw alpha +// magnitudes. Direction stays from the dominant horizon's sign. +const float smoothed_conviction = new_ema; +// ... existing logic uses smoothed_conviction in place of |alpha - 0.5| +``` + +Add the 3 new pointer params (`conviction_ema`, `conviction_diff_var_ema`, `conviction_sample_var_ema`) to the kernel signature. + +**Critical:** the existing logic that decides direction + size needs to be updated to use `smoothed_conviction` instead of recomputing from `alpha_probs` directly. Read the existing code carefully and substitute the smoothed value in the right place. Direction can still come from raw alpha (since direction flips are what we WANT to respond to fast), but magnitude/sizing should use the smoothed value. + +- [ ] **Step 4: Thread the new slots into the decision kernel launches** + +In `crates/ml-backtesting/src/sim/mod.rs`, find the `step_decision_with_latency` method (or wherever the decision kernels are launched). Add the three new `.arg(&self.conviction_ema_d).arg(&self.conviction_diff_var_ema_d).arg(&self.conviction_sample_var_ema_d)` to both decision kernel launches (default + program if both exist). + +Match exactly the order in the kernel signature defined in Step 3. + +- [ ] **Step 5: Compile and run the test** + +```bash +SQLX_OFFLINE=true cargo check -p ml-backtesting --tests 2>&1 | tail -5 +SQLX_OFFLINE=true cargo test -p ml-backtesting --test stop_controller -- --ignored --nocapture 2>&1 | tail -10 +``` + +Expected: previously failing test passes (or remained skipped if helpers unavailable). 20+ existing stop_controller tests still pass. + +- [ ] **Step 6: Commit** + +```bash +git add -A +git commit -m "$(cat <<'EOF' +arch(crt-a): conviction-EMA smoothing in decision_policy — minimal Phase B subset + +Removing the decision_stride gate (previous commit) makes the +controller fire every event. Without smoothing, target_lots would +oscillate as alpha probabilities jitter event-to-event, generating +hyperactive cost from spread-paying back-and-forth trades. + +Ships the smallest Layer B feature that makes Layer A behaviorally +sensible: Wiener-α adaptive EMA on max-conviction-across-horizons +with α floor at 0.4 per pearl_wiener_alpha_floor_for_nonstationary. +First-observation bootstrap per pearl_first_observation_bootstrap. + +Three new device slots: conviction_ema_d, conviction_diff_var_ema_d, +conviction_sample_var_ema_d. Threaded through both decision_policy_* +kernels' signatures and launches in sim/mod.rs. + +Direction still comes from raw alpha (genuine reversals respond fast). +Magnitude/sizing uses smoothed conviction. + +Full multi-horizon conviction formula from spec §4.4 deferred to +Phase B; this commit ships ONLY the EMA on the existing scalar +conviction proxy. + +Co-Authored-By: Claude Opus 4.7 (1M context) +EOF +)" +``` + +--- + +### Task A3: Local compile + test suite + smoke fixture + +**Files:** none (verification only) + +- [ ] **Step 1: Full workspace compile** + +```bash +SQLX_OFFLINE=true cargo check --workspace 2>&1 | tail -5 +``` + +Expected: clean, no warnings related to decision_stride or conviction_ema. + +- [ ] **Step 2: Full ml-backtesting test suite** + +```bash +SQLX_OFFLINE=true cargo test -p ml-backtesting --lib 2>&1 | tail -10 +SQLX_OFFLINE=true cargo test -p ml-backtesting --test stop_controller -- --ignored --nocapture 2>&1 | tail -10 +SQLX_OFFLINE=true cargo test -p ml-backtesting --test decision_floor_coldstart -- --ignored --nocapture 2>&1 | tail -10 +``` + +Expected: all pass. + +- [ ] **Step 3: Smoke test if available locally** + +If local GPU has enough memory for the smoke (per memory `user_dev_environment.md` it's RTX 3050 Ti 4GB — may not): + +```bash +FOXHUNT_TEST_DATA=test_data/futures-baseline cargo test -p ml --lib -- smoke_tests --ignored --nocapture 2>&1 | tail -20 +``` + +If local GPU insufficient: skip and rely on cluster smoke at Task A4. + +- [ ] **Step 4: No commit (verification only)** + +--- + +### Task A4: Cluster smoke + Gate 1 validation + +**Files:** none (deployment + monitoring) + +- [ ] **Step 1: Push the branch** + +```bash +git push origin ml-alpha-phase-a 2>&1 | tail -3 +``` + +Per [feedback_push_before_deploy](../../../../home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/feedback_push_before_deploy.md): always push before argo-lob-sweep. + +- [ ] **Step 2: Capture the HEAD commit SHA** + +```bash +SHA=$(git rev-parse --short=9 HEAD) +echo "deploying SHA=$SHA" +``` + +- [ ] **Step 3: Submit cluster smoke** + +```bash +./scripts/argo-lob-sweep.sh --branch ml-alpha-phase-a --sha $SHA --grid config/ml/sweep_smoke.yaml --sweep-tag sweep_smoke-$SHA 2>&1 | tail -3 +``` + +Capture the workflow name (`lob-backtest-sweep-XXXXX`). + +- [ ] **Step 4: Monitor workflow to completion** + +Use a Monitor task or poll loop. Expected ~13-15 min total (compile + run-sweep + aggregate). Wait for the `nan_counters` and `nan_counters_v2` log lines plus the `=== sweep complete ===` marker. + +- [ ] **Step 5: Pull smoke metrics from the run-sweep pod log** + +```bash +POD=$(kubectl -n foxhunt get pods --no-headers | awk '/lob-backtest-sweep-XXXXX-run-sweep/{print $1; exit}') +kubectl -n foxhunt logs "$POD" -c main 2>&1 | grep -E "progress:|nan_counters|stop_ctrl_counters|sweep complete|elapsed=" | tail -20 +``` + +Note: with no `decision_stride`, the `decisions=` counter on the progress line should equal `events=` (or very close — only events with full snapshot_window count). + +- [ ] **Step 6: Fetch summary.json + trade CSV via inspect pod** + +Use the same pattern as the S2 verdict pod (see `docs/superpowers/plans/2026-05-19-isv-driven-stop-controller.md` Task 11 Step 5): + +```bash +cat > /tmp/crt-a-summary-read.yaml < +15%) | 53.5% | ≤ 61.5% | [Y/N] | +| nan_counters all 0 | yes | yes | [Y/N] | +| nan_counters_v2 all 0 | yes | yes | [Y/N] | + +Spec §3.6 alpha lag criterion ("≥ 30% reduction in mean signal-to-entry lag") is hard to measure without additional instrumentation. **Defer** this criterion to a follow-up if Gate 1 otherwise passes — log it as an open observation, not a blocker. + +- [ ] **Step 8: Decide gate outcome** + +- **All MUST criteria pass** → Gate 1 is GREEN. Update spec status (annotate v2 with "Gate 1 validated commit XXX"), commit a project memory entry, and prepare to write Phase B plan. +- **Any MUST criterion fails** → Gate 1 is RED. Diagnose with per-event instrumentation (same SP21-style approach used for S2). Do NOT advance to Phase B. +- **MUST passes but observed hyperactivity** (e.g., trade count balloons > 2× baseline despite Sharpe being roughly stable) → Task A5 fires. + +- [ ] **Step 9: Write Gate 1 outcome memory entry** + +If Gate 1 GREEN: + +```bash +cat > /home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/project_crt_phase_a_gate1.md <<'EOF' +--- +name: project-crt-phase-a-gate1 +description: CRT Phase A (continuous controller) Gate 1 validation result — commit, smoke, metrics. +metadata: + type: project +--- + +# CRT Phase A Gate 1 — [GREEN | RED | YELLOW] + +Date: [today] +Commit: [SHA] +Smoke: [workflow name] + +## Results vs baseline (commit 8828e8ab1, smoke tgg4l) +[table] + +## Open observations +- Alpha lag criterion deferred — needs per-event entry-vs-signal-spike instrumentation; tracked separately. + +## Next +- Phase B plan to be written: docs/superpowers/plans/2026-05-XX-crt-phase-b-signal-driven-position-mgmt.md +EOF +``` + +Update `MEMORY.md` to link the new entry. + +- [ ] **Step 10: Commit memory + close Phase A** + +```bash +cd /home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt +# Memory is per-user, not in the git repo. Just save the file (Write tool). +``` + +Back in repo: +```bash +cd /home/jgrusewski/Work/foxhunt +git log --oneline -5 +``` + +Confirm clean state. Phase A complete. + +--- + +### Task A5: Hyperactivity mitigation (conditional — only if Gate 1 reveals churn) + +This task fires ONLY if Step 8 detects hyperactivity (trade count balloon, e.g., > 2× baseline, despite Sharpe staying within ±15%). + +**Files:** +- Modify: `crates/ml-backtesting/cuda/decision_policy.cu` (tighten EMA α floor OR add hysteresis on target deltas) + +- [ ] **Step 1: Quantify hyperactivity** + +Pull the trade CSV. Compare: +- Mean trade hold time (should be similar to baseline — small or matches max_hold setting) +- Trade count distribution (per-hour, per-day) +- Fees paid (proportional to round-trip count) + +- [ ] **Step 2: Choose mitigation** + +Two options: + +**Option A** (preferred if minor): raise the Wiener-α floor from 0.4 to 0.6 in §4.2's `alpha_active = fmaxf(alpha_raw, 0.6f);`. Damps further. Lower responsiveness. + +**Option B** (stronger): add target-delta hysteresis in `seed_inflight_limits_batched`. Don't seed a new order unless `|new_target − effective_position| ≥ delta_floor` where delta_floor is e.g. 0.3 lots (rounded). Skips micro-rebalances. + +- [ ] **Step 3: Apply mitigation, re-test, re-validate Gate 1** + +Follow A4 steps with the chosen mitigation. If Gate 1 passes after mitigation, document the chosen value as a tuned constant in the spec amendment (acknowledge as tuned, not derived — pearl_adaptive_not_tuned says "fixes are signal-driven not tuned constants" so any tuned floor should be a temporary stopgap with a follow-up to make it ISV-derived). + +- [ ] **Step 4: Commit** + +If mitigation chosen: +```bash +git add -A +git commit -m "fix(crt-a): tighten conviction-EMA floor 0.4→0.6 — mitigates hyperactivity seen in [smoke]" +``` + +If mitigation NOT needed (hyperactivity wasn't a real problem), no commit — leave task A5 as documentation that it was considered and ruled out. + +--- + +## Notes for the Implementer + +- **TDD discipline.** Each task starts with a failing test where possible (Tasks A2, A5 explicitly do); Tasks A0 and A1 are refactor/research and don't fit the failing-test mold. Per [feedback_no_quickfixes](../../../../home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/feedback_no_quickfixes.md) — every issue gets a proper fix; tests where a test makes sense. + +- **Each task is one commit.** No mid-task commits. + +- **Greenfields atomic refactor (Task A1).** Every `decision_stride` consumer migrates in one commit per [feedback_no_partial_refactor](../../../../home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/feedback_no_partial_refactor.md). Don't leave a "compat" branch around. + +- **Pearl conformance:** + - Task A2 uses [pearl_wiener_optimal_adaptive_alpha](../../../../home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/pearl_wiener_optimal_adaptive_alpha.md) and [pearl_wiener_alpha_floor_for_nonstationary](../../../../home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/pearl_wiener_alpha_floor_for_nonstationary.md). + - Task A2 uses [pearl_first_observation_bootstrap](../../../../home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/pearl_first_observation_bootstrap.md) for EMA init. + - Task A1 follows [feedback_no_legacy_aliases](../../../../home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/feedback_no_legacy_aliases.md) — no compat shim. + - Task A4 follows [feedback_push_before_deploy](../../../../home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/feedback_push_before_deploy.md). + +- **Local GPU is RTX 3050 Ti 4GB** ([user_dev_environment](../../../../home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/user_dev_environment.md)) — may not be able to run the full smoke locally; cluster smoke at Task A4 is the real validation. + +- **If Task A0 returns Case 2 (`forward_only` is stateless K-window per call):** STOP and notify the user. This expands Phase A scope significantly (need `forward_only_incremental` first). Don't proceed without explicit approval. + +- **If Task A4 Gate 1 fails:** apply [feedback_stop_on_anomaly](../../../../home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/feedback_stop_on_anomaly.md) — terminate, diagnose, fix, re-run. Use SP20/SP21-style per-event instrumentation if needed. Do NOT advance to Phase B with a red Gate 1. + +--- + +## Phase B / C / D — Forward Reference + +After Gate 1 closes (green), the next implementation plan will be written: + +- **`docs/superpowers/plans/2026-05-XX-crt-phase-b-signal-driven-position-mgmt.md`** — Layer B: multi-horizon conviction (spec §4.4), conviction-degradation exit (§4.3), continuous sizing (§4.2 full), `open_trade_state` 24→64 byte atomic refactor (§7). + +- **`docs/superpowers/plans/2026-05-XX-crt-phase-c-adaptive-envelope.md`** — Layer C: max_lots additive bounded envelope (§5.1), conviction-percentile threshold gate with 3-arm bandit (§5.2), regime-vol-inverse target vol (§5.3). + +- **`docs/superpowers/plans/2026-05-XX-crt-phase-d-online-adaptation.md`** — Layer D: LoRA adapter, value-regression loss + EWC++ (§6.2), shadow eval (§6.2.1), circuit-breaker revert (§6.2.2), regime-overfit mitigation (§10.5). + +**Each gate is a stop.** Per spec §12: do not write the next plan until the prior gate closes. This avoids the spec-velocity-vs-validation mismatch that bit us in S2.