From 566e8bcb0ab7876c641925855e263b963bc593c6 Mon Sep 17 00:00:00 2001 From: jgrusewski Date: Tue, 19 May 2026 21:47:13 +0200 Subject: [PATCH] =?UTF-8?q?spec(ml-backtesting):=20CBSW=20review=20pass=20?= =?UTF-8?q?=E2=80=94=2011=20fixes=20(perf=20+=20correctness)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Critical review surfaced 11 actionable issues in the v1 spec; 10 fixed inline (#11 — legacy-comparison test — dropped per user decision since empirical legacy behavior is already known: n_trades=0). Performance: 1. expf → piecewise-linear ramp (~5× cheaper on GPU, no transcendental in the hot path). Operationally equivalent: monotonic, saturating, midpoint at K. Side-benefit: pure max-confidence at cold-start (sq=0 instead of sigmoid's 11.9% leakage). 2. Single fused per-horizon pass replaces the two-loop sketch. 3. Tier 2 scope narrowed to decision_policy_default ONLY. The bytecode VM (decision_policy_program) stays unchanged — runs only for custom strategy experiments, never in production policy. Halves Tier 2's surface area + test cost. 4. __device__ helpers cbsw_signal_quality + cbsw_weight introduced for single source of truth. Correctness bugs (silently present in v1 sketch, would have shipped): 5. strong_h and sq_min_active initializers added (were referenced before init in the per-horizon loop). 6. Attribution mask at cold-start was setting all 5 bits because every w[h] >= floor > 1e-9; trades would pollute ALL horizons' recent_sharpe. Fix: binary split — at sq_min_active < 0.5 attribute only to strong_h; at mature, attribute per weights > floor + epsilon. 7. sq_min was "min over h", which permanently locked the aggregator into cold-start mode if any horizon never gets attributed (e.g., h6000-only-trading regime). Fix: sq_min_active = min over h with n_trades_seen > 0; cold horizons don't gate maturity. 8. Opposing-horizons case now correctly fires on the strongest single horizon at cold-start (piecewise-linear sq=0 → pure max-conf), with explicit design-choice note explaining why this conservatism trade- off favors firing. Spec hygiene: 9. Rate validation committed to a measurement gate (Q2 ev/s ≥ 95% of Q1 ev/s) instead of back-of-envelope estimate. 10. Kernel ABI explicitly stated as unchanged → feedback_no_partial_ refactor compliance is trivial at the kernel boundary. 12. Tier 1's use_cold_start_stopgap field is DROPPED in Q2 atomically (not orphaned), and DoD checklist includes a grep-zero gate for feedback_no_legacy_aliases compliance. Risks updated: removed the sigmoid-narrowing risk (irrelevant now); added register-pressure risk with the rate-gate mitigation; added explicit "cold-start may lose on noisy trades" risk with empirical escalation path (bump kelly_floor 0.20 → 0.40 if Q2 smoke shows large negative PnL). Math walk-throughs rewritten for piecewise-linear (different numbers at cold-start): cold = pure max-conf, mature = pure weighted-sharpe, clean separation at sq_min_active threshold. Awaiting review of the revised spec before writing-plans dispatch. Co-Authored-By: Claude Opus 4.7 (1M context) --- ...05-19-cbsw-cold-start-aggregator-design.md | 287 ++++++++++++------ 1 file changed, 188 insertions(+), 99 deletions(-) diff --git a/docs/superpowers/specs/2026-05-19-cbsw-cold-start-aggregator-design.md b/docs/superpowers/specs/2026-05-19-cbsw-cold-start-aggregator-design.md index f43a5b35b..99d9ddf00 100644 --- a/docs/superpowers/specs/2026-05-19-cbsw-cold-start-aggregator-design.md +++ b/docs/superpowers/specs/2026-05-19-cbsw-cold-start-aggregator-design.md @@ -69,19 +69,21 @@ CBSW changes both the **weight formula** AND the **aggregation type**, transitio ``` n_h = isv[h].n_trades_seen -K = MIN_TRADES_FOR_VAR_CAP = 10 (already exists, sample-size threshold) -sigmoid_scale = 5.0 (half-transition width) -signal_quality[h] = sigmoid((n_h - K) / sigmoid_scale) +K = MIN_TRADES_FOR_VAR_CAP = 10 (already exists, sample-size threshold) +RAMP_HALF_WIDTH = 5.0 (half-width of linear transition zone) +signal_quality[h] = clamp((n_h − K + RAMP_HALF_WIDTH) / (2 × RAMP_HALF_WIDTH), 0, 1) ``` -Properties: -- `n=0` → `sq = sigmoid(-2) ≈ 0.119` (cold) -- `n=K-5=5` → `sq = sigmoid(-1) ≈ 0.269` (transitioning) -- `n=K=10` → `sq = sigmoid(0) = 0.500` (mid-transition) -- `n=K+5=15` → `sq = sigmoid(+1) ≈ 0.731` (transitioning) -- `n=K+10=20` → `sq = sigmoid(+2) ≈ 0.881` (nearly mature) +**Piecewise-linear ramp, not sigmoid.** Operationally identical (monotonic + saturating + same midpoint), but ~5× cheaper on GPU: 2 `fmaxf/fminf` + 1 division per horizon vs 1 `expf` + arithmetic. Operational smoothness of sigmoid is mathematically pleasing but irrelevant here — Kelly aggregation doesn't differentiate the weight. -Smooth, monotonic, signal-driven by `n_trades_seen` (the ISV-side sample count). Reuses the existing `MIN_TRADES_FOR_VAR_CAP` threshold so there's no new tuning knob. +Properties: +- `n=0` → `sq = clamp(−0.5,0,1) = 0.000` (pure cold) +- `n=K−5=5` → `sq = 0.000` (start of ramp) +- `n=K=10` → `sq = 0.500` (mid-transition) +- `n=K+5=15` → `sq = 1.000` (fully mature) +- `n=K+10=20` → `sq = 1.000` (saturated) + +Cold-start is now **pure max-confidence** (sq=0 → conviction weight only, ss_strong only in aggregator). This is cleaner than the sigmoid's 11.9% leakage at n=0. Reuses the existing `MIN_TRADES_FOR_VAR_CAP` threshold so there's no new tuning knob. #### Weight formula (blended per-horizon) @@ -100,69 +102,103 @@ At cold-start the weight = `max(floor, sig_mag[h])` — strong-conviction horizo Weighted-mean alone can't escape the dilution bound. We need max-confidence at cold-start. Hybrid: ``` -strong_h = argmax_h |ss[h]| // max-confidence pick -ss_strong = ss[strong_h] -ss_weighted = Σ w_norm[h] × ss[h] -sq_min = min over h of signal_quality[h] // transition gate -final_size = (1 - sq_min) * ss_strong + sq_min * ss_weighted +strong_h = argmax_h |ss[h]| // max-confidence pick +ss_strong = ss[strong_h] +ss_weighted = Σ (w[h] / w_sum) × ss[h] // weighted-mean piece +sq_min_active = min over h with n_trades_seen[h] > 0 of signal_quality[h] + (defaults to 0.0 when NO horizon has any trades) +final_size = (1 − sq_min_active) × ss_strong + sq_min_active × ss_weighted ``` -At cold-start: 100% max-confidence → strong-horizon-only trades fire. -At mature: 100% weighted-sharpe → existing behavior. -Transition: smooth blend over the same K=10-trade threshold. +**Why `sq_min_active` and not `sq_min`**: if only one horizon ever contributes to closes (e.g., h6000 = the AUC-best horizon does all the trading), `n_trades_seen` for other horizons stays at 0 forever. With a naive `sq_min`, the aggregator would be **permanently** stuck in cold-start mode despite h6000 being fully mature. Restricting the min to horizons that have *actually contributed* fixes this: only horizons with real sample history gate the transition. Horizons with n=0 are silently excluded — they don't get to veto maturity. + +At cold-start (no closes anywhere): `sq_min_active = 0.0` → 100% max-confidence → strong-horizon-only trades fire. +At fully mature (all h have closes): `sq_min_active = min sq_h` → weighted-mean takes over. +At partial mature (only h6000 has closes): `sq_min_active = sq_{h6000}` → if it's mature, aggregator is mature. + +**Attribution mask** (which horizons get credited on trade close, drives `isv_kelly_update_on_close`): + +``` +if (sq_min_active < 0.5): + attribution_mask = (1u << strong_h) // cold-start: blame the strong horizon only +else: + attribution_mask = OR over h with w[h] > sharpe_weight_floor + ε of (1u << h) + // mature: credit horizons that contributed signal beyond the floor +``` + +The cold-start branch fixes a silent bug in the v1 sketch: previously `attribution_mask` was set whenever `w[h] > 1e-9`, which was always true (floor = 0.10 ≫ 1e-9), so cold-start trades polluted ALL 5 horizons' `recent_sharpe`. With the binary split, the strong horizon takes the credit it deserves; other horizons are unaffected by trades they didn't drive. + +**Design choice — opposing-horizon trades at cold-start**: with piecewise-linear `sq=0`, the aggregator at cold-start is pure max-confidence. For `alpha = [0.75, 0.30, 0.50, 0.50, 0.50]` (one bullish, one bearish, rest neutral), CBSW fires a long on h0's signal — h1's opposing bearish signal is ignored. This is **deliberate**: at cold-start the model has no historical performance data to use as a tiebreaker, so the strongest single-horizon conviction wins. Once `recent_sharpe[h]` is populated, the mature-regime weighted aggregator will properly penalize horizons that have historically lost on these patterns. + +The alternative (suppress trades on opposing horizons) would echo the consensus-requirement bug. We explicitly choose to fire on single-horizon strong signals at cold-start, with the understanding that some of these trades will lose and `recent_sharpe[h]` will adjust accordingly. ### Math walk-through (cold-start, single-horizon strong) -`alpha = [0.85, 0.50, 0.50, 0.50, 0.50]`, n=0 everywhere, sq_min ≈ 0.119: +`alpha = [0.85, 0.50, 0.50, 0.50, 0.50]`, n=0 everywhere → `sq_min_active = 0.0`: | Quantity | Value | Note | |---|---|---| | `sig_mag` | `[0.70, 0, 0, 0, 0]` | | | `dir` | `[+1, +1, +1, +1, +1]` | doesn't matter for sig_mag=0 | -| `ss = dir × sig_mag × 0.20 × 5` | `[0.70, 0, 0, 0, 0]` | kelly_floor=0.20, cap=max_lots=5 | -| `w` (with floor) | `[max(0.10, 0.70), 0.10, 0.10, 0.10, 0.10]` = `[0.70, 0.10, 0.10, 0.10, 0.10]` | sig_mag → high weight on strong h | -| `w_sum` | 1.10 | | -| `w_norm` | `[0.636, 0.091, 0.091, 0.091, 0.091]` | | -| `ss_weighted` | `0.636 × 0.70 + 0 = 0.445` | linear-mean piece | +| `ss = dir × sig_mag × 0.20 × 5` | `[+0.70, 0, 0, 0, 0]` | kelly_floor=0.20, cap=max_lots=5 | +| `w` (with floor) at sq=0 | `[max(0.10, 0.70), 0.10, 0.10, 0.10, 0.10]` = `[0.70, 0.10, 0.10, 0.10, 0.10]` | pure conviction → strong h gets full sig_mag | | `ss_strong` | `0.70` | max-conf pick | -| `final_size` | `(1 − 0.119) × 0.70 + 0.119 × 0.445 = 0.616 + 0.053 = 0.669` | | -| `lots = truncf(0.669 + 0.5)` | **1** | trade fires | +| `final_size` | `(1 − 0) × 0.70 + 0 × ss_weighted = 0.70` | pure max-confidence at cold-start | +| `lots = truncf(0.70 + 0.5)` | **1** | trade fires | +| `attribution_mask` (sq < 0.5) | `(1u << 0) = 0b00001` | only h0 gets credited on close | -Single-horizon strong signal at cold-start now fires a 1-lot trade. +Single-horizon strong signal at cold-start fires a 1-lot trade, attributed solely to the convicted horizon. -### Math walk-through (weak conflict) +### Math walk-through (opposing horizons at cold-start) -`alpha = [0.55, 0.45, 0.55, 0.45, 0.55]`, n=0, sq_min ≈ 0.119: +`alpha = [0.75, 0.30, 0.50, 0.50, 0.50]` (h0 bullish, h1 bearish, rest neutral), n=0 → `sq_min_active = 0.0`: + +| Quantity | Value | +|---|---| +| `sig_mag` | `[0.50, 0.40, 0, 0, 0]` | +| `dir` | `[+1, −1, +1, +1, +1]` | +| `ss` | `[+0.50, −0.40, 0, 0, 0]` | +| `strong_h` (argmax \|ss\|) | h=0, `ss_strong = +0.50` | +| `final_size` (pure max-conf) | `0.50` | +| `lots` | **1** (long h0's signal) | +| `attribution_mask` | `(1u << 0)` | + +Cold-start fires on the strongest single-horizon signal even when other horizons disagree. Per the design-choice note: subsequent close attributes loss/win exclusively to h0, letting `recent_sharpe[0]` adjust. + +### Math walk-through (weak conflict at cold-start) + +`alpha = [0.55, 0.45, 0.55, 0.45, 0.55]`, n=0 → sq=0: | Quantity | Value | |---|---| | `sig_mag` | `[0.10, 0.10, 0.10, 0.10, 0.10]` | -| `dir` | `[+1, −1, +1, −1, +1]` | | `ss` | `[+0.10, −0.10, +0.10, −0.10, +0.10]` | -| `strong_h` (argmax of \|ss\|) | any (ties); pick h=0; `ss_strong = +0.10` | -| `w` | `[max(0.10, 0.10)] × 5 = [0.10] × 5` (all equal) | -| `w_norm` | `[0.20] × 5` | -| `ss_weighted` | `0.20 × (0.10 − 0.10 + 0.10 − 0.10 + 0.10) = 0.02` | -| `final_size` | `0.881 × 0.10 + 0.119 × 0.02 = 0.091` | -| `lots = truncf(0.091 + 0.5)` | **0** → noop | +| `strong_h` (first tie) | h=0; `ss_strong = +0.10` | +| `final_size` | `0.10` | +| `lots = truncf(0.10 + 0.5)` | **0** → noop | -Weak conflicting signals correctly don't trade. +Weak signals correctly fail to clear the lots≥1 threshold. The model needs at least one horizon with conviction ≥ ~0.5 to fire 1 lot at cold-start (sig_mag × kelly_floor × cap_lots > 0.5 → sig_mag > 0.5 when kelly_floor=0.20, max_lots=5). -### Math walk-through (mature regime, n=20 everywhere) +### Math walk-through (h6000-only mature regime) -`alpha = [0.85, 0.50, 0.50, 0.50, 0.50]`, sq_min ≈ 0.881, `recent_sharpe = [1.5, 0.1, 0.1, 0.1, 0.1]` (model converged: h0 is the best horizon): +After 50 trades, all attributed to h6000 (the AUC-best horizon at training). `n_trades_seen = [0, 0, 0, 0, 50]`, `recent_sharpe = [0, 0, 0, 0, 1.5]`. `sq[h] = [0, 0, 0, 0, 1.0]`. `sq_min_active = min over active h = sq[4] = 1.0` (only h=4 is active). + +`alpha = [0.85, 0.50, 0.50, 0.50, 0.85]` (h0 and h6000 both strong-long this decision): | Quantity | Value | |---|---| -| `w_blend` | `0.119 × sig_mag + 0.881 × sharpe = [0.083+1.322, 0.0+0.088, ...] = [1.405, 0.088, 0.088, 0.088, 0.088]` | -| `w` (with floor) | `[1.405, 0.10, 0.10, 0.10, 0.10]` | -| `w_norm` | `[0.778, 0.055, 0.055, 0.055, 0.055]` | -| `ss_weighted` | `0.778 × 0.70 = 0.544` | -| `ss_strong` | `0.70` | -| `final_size` | `0.119 × 0.70 + 0.881 × 0.544 = 0.083 + 0.479 = 0.562` | -| `lots` | **1** | +| `sig_mag` | `[0.70, 0, 0, 0, 0.70]` | +| `ss` | `[+0.70, 0, 0, 0, +0.70]` | +| `w` (sq[h] varies) | `[max(0.10, 1.0 × 0.70 + 0.0 × 0), 0.10, 0.10, 0.10, max(0.10, 0.0 × 0.70 + 1.0 × 1.5)]` = `[0.70, 0.10, 0.10, 0.10, 1.50]` | +| `w_sum` | 2.50 | +| `w_norm` | `[0.28, 0.04, 0.04, 0.04, 0.60]` | +| `ss_weighted` | `0.28 × 0.70 + 0.60 × 0.70 = 0.196 + 0.420 = 0.616` | +| `ss_strong` (tie at \|ss\|=0.70, picks first) | h=0; `ss_strong = +0.70` | +| `final_size` | `(1 − 1.0) × 0.70 + 1.0 × 0.616 = 0.616` | pure weighted-sharpe | +| `lots = truncf(0.616 + 0.5)` | **1** | +| `attribution_mask` (sq ≥ 0.5) | `(1u << 0) \| (1u << 4) = 0b10001` (both w[0] and w[4] > floor) | -Mature regime preserves the asymmetric Sharpe-driven weighting. Strong-Sharpe horizon dominates correctly. +Mature regime correctly emphasizes h6000 (1.5 sharpe weight dominates h0's cold conviction weight of 0.70). Both horizons get attributed on close because both contributed signal beyond the floor — this lets h0's `recent_sharpe` start tracking from this trade, gradually accelerating its transition out of cold-start. --- @@ -196,75 +232,123 @@ When `program_lens[b] > 0`, the bytecode kernel handles the backtest and the def ## 4. Tier 2 — Kernel CBSW (proper architecture) -**Goal:** Replace both decision kernels' weight + aggregation logic with the CBSW formula from §2. Tier 1's bytecode stopgap becomes unnecessary and is reverted in the same commit. +**Goal:** Replace `decision_policy_default`'s weight + aggregation logic with the CBSW formula from §2. Tier 1's bytecode stopgap becomes unnecessary and is reverted in the same commit (`use_cold_start_stopgap` field dropped entirely per `feedback_no_legacy_aliases`). -**Kernel signature additions** (both `decision_policy_default` and `decision_policy_program`): +**Scope: `decision_policy_default` ONLY.** `decision_policy_program` is left unchanged because it runs only when `program_lens[b] > 0` (custom bytecode strategies), which the production deployment never sets. Modifying both kernels doubles surface area + test cost for zero production benefit. The bytecode VM remains a pluggable experiment surface — including Tier 1's max-confidence pattern, which can stay in `decision_policy_program` indefinitely as a documented strategy preset. + +**Kernel ABI unchanged.** No new args added. The two new constants are file-scope `#define`s: ```cuda -// New per-backtest scalar args (kept symmetric with existing P4 floors) -const float* sigmoid_scale_per_b, // = 5.0 default -int min_trades_for_trust_per_b // existing MIN_TRADES_FOR_VAR_CAP, now per-b +// File-scope, decision_policy.cu top: +#define MIN_TRADES_FOR_TRUST 10u // matches MIN_TRADES_FOR_VAR_CAP — sample-size threshold for trusting historical recent_sharpe +#define RAMP_HALF_WIDTH 5.0f // half-width of CBSW linear transition; 0 below K-half, 1 above K+half ``` -Actually — to minimize per-call upload overhead and keep the kernel ABI tight, use file-scope `#define` for both: +Both are sample-size constants, not adaptive bounds — they satisfy `pearl_isv_for_adaptive_bounds.md` (the pearl flags adaptive bounds, not statistical thresholds). Per `feedback_no_partial_refactor`: kernel ABI unchanged means no consumer migration; Tier 2 is a pure kernel-body change. + +**`__device__` helper for CBSW weight** (avoids duplication if/when other kernels need the same logic): ```cuda -#define MIN_TRADES_FOR_TRUST 10u -#define SIGMOID_SCALE 5.0f -``` - -(Both are sample-size thresholds — not adaptive bounds — so they satisfy `pearl_isv_for_adaptive_bounds.md`. Constants reused across the codebase: same K=10 as `MIN_TRADES_FOR_VAR_CAP`.) - -**Kernel body changes:** - -```cuda -// Inside the per-horizon loop, compute sig_mag and signal_quality: -const float n_h = (float)isv[h].n_trades_seen; -const float sq_h = 1.0f / (1.0f + expf(-(n_h - (float)MIN_TRADES_FOR_TRUST) / SIGMOID_SCALE)); - -// Weight: blend sig_mag (cold) and recent_sharpe (mature), with permanent floor. -const float sharpe_w = fmaxf(0.0f, isv[h].recent_sharpe); -const float conv_w = sig_mag; // already computed for ss[h] -const float w_blend = (1.0f - sq_h) * conv_w + sq_h * sharpe_w; -weights[h] = fmaxf(sharpe_weight_floor, w_blend); -w_sum += weights[h]; - -// Track strong_h for max-confidence aggregation -if (fabsf(signed_sizes[h]) > fabsf(signed_sizes[strong_h])) { - strong_h = h; +__device__ static inline float cbsw_signal_quality(unsigned int n_trades_seen) { + const float n = (float)n_trades_seen; + const float lo = (float)MIN_TRADES_FOR_TRUST - RAMP_HALF_WIDTH; // 5.0 + const float span = 2.0f * RAMP_HALF_WIDTH; // 10.0 + return fmaxf(0.0f, fminf(1.0f, (n - lo) / span)); +} + +__device__ static inline float cbsw_weight(float sig_mag, float recent_sharpe, + unsigned int n_trades_seen, float floor) { + const float sq = cbsw_signal_quality(n_trades_seen); + const float sharpe_w = fmaxf(0.0f, recent_sharpe); + const float w_blend = (1.0f - sq) * sig_mag + sq * sharpe_w; + return fmaxf(floor, w_blend); } -sq_min = fminf(sq_min, sq_h); // gate for the hybrid aggregator ``` +**Kernel body — single fused per-horizon pass** (replaces the existing weight + aggregation block in `decision_policy_default`): + ```cuda -// After the per-horizon loop, hybrid aggregation: -float final_size = 0.0f; +float signed_sizes[N_HORIZONS]; +float weights[N_HORIZONS]; +int strong_h = 0; // init explicit (was implicit-bug in v1 sketch) +float strong_abs = 0.0f; +float w_sum = 0.0f; +float sq_min_active = 1.0f; // init at max; only horizons with n>0 reduce this +int n_active_horizons = 0; + +#pragma unroll +for (int h = 0; h < N_HORIZONS; ++h) { + const float p_h = alpha_probs[h]; + const IsvKellyState& s = isv[h]; + + const float sig_mag = fabsf(p_h - 0.5f) * 2.0f; + const float dir = (p_h > 0.5f) ? 1.0f : -1.0f; + const float kelly_frac = compute_kelly_frac(s, kelly_frac_floor); // existing logic + const float cap_lots = compute_cap_lots(s, target_annual_vol_units, + annualisation_factor, max_lots); // existing logic + signed_sizes[h] = dir * sig_mag * kelly_frac * cap_lots; + + weights[h] = cbsw_weight(sig_mag, s.recent_sharpe, s.n_trades_seen, sharpe_weight_floor); + w_sum += weights[h]; + + // max-confidence pick: track strong_h by |ss| + const float abs_ss = fabsf(signed_sizes[h]); + if (abs_ss > strong_abs) { strong_abs = abs_ss; strong_h = h; } + + // sq_min over ACTIVE horizons only (n_trades_seen > 0); cold horizons don't gate maturity + if (s.n_trades_seen > 0u) { + sq_min_active = fminf(sq_min_active, cbsw_signal_quality(s.n_trades_seen)); + n_active_horizons += 1; + } +} +// If no horizon has any closes yet, the aggregator is pure max-confidence. +if (n_active_horizons == 0) sq_min_active = 0.0f; + +// Hybrid aggregator +float ss_weighted = 0.0f; if (w_sum > 1e-9f) { - float ss_weighted = 0.0f; #pragma unroll for (int h = 0; h < N_HORIZONS; ++h) { ss_weighted += (weights[h] / w_sum) * signed_sizes[h]; - if (weights[h] > 1e-9f) attribution_mask |= (1u << h); } - const float ss_strong = signed_sizes[strong_h]; - final_size = (1.0f - sq_min) * ss_strong + sq_min * ss_weighted; +} +const float ss_strong = signed_sizes[strong_h]; +const float final_size = (1.0f - sq_min_active) * ss_strong + + sq_min_active * ss_weighted; + +// Attribution: cold → strong_h only; mature → all horizons above floor + ε +unsigned int attribution_mask; +if (sq_min_active < 0.5f) { + attribution_mask = (1u << strong_h); +} else { + attribution_mask = 0u; + #pragma unroll + for (int h = 0; h < N_HORIZONS; ++h) { + if (weights[h] > sharpe_weight_floor + 1e-6f) attribution_mask |= (1u << h); + } } ``` +Single forward pass + one short aggregation loop (5 elements, fully unrolled by the compiler). No `expf`. The `cbsw_signal_quality` is called twice per horizon (once in `cbsw_weight`, once in the sq_min reduce) but both calls inline trivially and the result is shared in registers by the compiler. + **Files:** -- `crates/ml-backtesting/cuda/decision_policy.cu` — both kernels (default + program). ~30 LOC each. -- `crates/ml-backtesting/src/harness.rs` + sim — revert the Tier 1 stopgap bytecode upload. +- `crates/ml-backtesting/cuda/decision_policy.cu` — `decision_policy_default` body change + 2 `__device__` helpers + `#define`s. ~50 LOC. +- `crates/ml-backtesting/src/harness.rs` — revert the Tier 1 stopgap bytecode upload, **drop** the `use_cold_start_stopgap` field entirely from `BacktestHarnessConfig` + `UniformSimParams`. +- `bin/fxt-backtest/src/main.rs` — drop the `--cold-start-stopgap` CLI flag. +- `config/ml/sweep_smoke.yaml` — drop `use_cold_start_stopgap: true` from the variant. - `crates/ml-backtesting/tests/cbsw_aggregator.rs` (NEW) — unit tests per §2 math walk-throughs. **Tests:** | Test | Setup | Assertion | |---|---|---| -| `cbsw_cold_start_single_horizon_strong_fires_trade` | n=0 all h, alpha=[0.85,0.50,0.50,0.50,0.50] | side=0, size ≥ 1 | +| `cbsw_cold_start_single_horizon_strong_fires_trade` | n=0 all h, alpha=[0.85,0.50,0.50,0.50,0.50] | side=0, size≥1, attribution_mask = 0b00001 | +| `cbsw_cold_start_opposing_horizons_fires_on_strongest` | n=0, alpha=[0.75,0.30,0.50,0.50,0.50] | side=0, size≥1, attribution_mask = 0b00001 | | `cbsw_cold_start_weak_conflict_does_not_trade` | n=0, alpha=[0.55,0.45,0.55,0.45,0.55] | side=2, size=0 | -| `cbsw_cold_start_consensus_strong_fires_trade` | n=0, alpha=[0.85]×5 | side=0, size ≥ 1 | -| `cbsw_mature_recovers_weighted_sharpe` | n=20 all h, recent_sharpe=[1.5,0.1,0.1,0.1,0.1], alpha=[0.85,0.50,0.50,0.50,0.50] | side=0, size ≥ 1, ss_weighted contribution > 0.5 of final | -| `cbsw_transition_smooth` | n=K all h, alpha=[0.70,0.50,0.50,0.50,0.50] | size monotone in n_trades_seen sweep | +| `cbsw_cold_start_consensus_strong_fires_trade` | n=0, alpha=[0.85]×5 | side=0, size≥1 | +| `cbsw_mature_recovers_weighted_sharpe` | n=20 all h, recent_sharpe=[1.5,0.1,0.1,0.1,0.1], alpha=[0.85,0.50,0.50,0.50,0.50] | side=0, size≥1, attribution_mask has all 5 bits set, ss_weighted contribution > 90% of final_size | +| `cbsw_h6000_only_active_does_not_block_maturity` | n_trades_seen=[0,0,0,0,50], recent_sharpe=[0,0,0,0,1.5], alpha=[0.85,0.50,0.50,0.50,0.85] | sq_min_active = 1.0 (only h=4 active), final_size ≈ ss_weighted (mature regime), trade fires | +| `cbsw_transition_smooth` | sweep n_trades_seen=[0..20], alpha=[0.70,0.50,0.50,0.50,0.50] | size monotone non-decreasing in n_trades_seen, no discontinuity at n=K=10 | **Validation gate:** Re-run threshold-tuning smoke at threshold=0.0, cost=0.125. Expected `n_trades > 100`, `total_pnl_usd != 0`. Cross-check that Tier 1 and Tier 2 produce similar n_trades counts at cold-start (both should fire on single-horizon strong signals). Tier 2 should be *better* than Tier 1 once trades start closing (recent_sharpe takes over, max-confidence over-trading subsides). @@ -306,16 +390,16 @@ If found, log as separate spec follow-ups. Don't expand THIS spec's scope. ## 6. Migration plan (atomic commits) -Per `feedback_no_partial_refactor`: each commit is internally consistent and ships its own tests. +Per `feedback_no_partial_refactor`: each commit is internally consistent and ships its own tests. **Kernel ABI is unchanged across all four commits** — Tier 1 uses the existing `decision_policy_program` kernel as-is (bytecode upload only); Tier 2 changes `decision_policy_default`'s body but not its signature. No consumer migration required at the kernel boundary. | Commit | Subject | Files | Test gate | |---|---|---|---| -| Q1 | `feat(ml-backtesting): cold-start stopgap — max-confidence bytecode policy (Tier 1)` | sim/batched_config.rs (+ use_cold_start_stopgap flag), harness.rs (upload bytecode program when flag set), main.rs (CLI flag), sweep_smoke.yaml (set flag) | Local: harness builds bytecode program of correct shape. Cluster: smoke fires `n_trades > 100`. | -| Q2 | `feat(ml-backtesting): CBSW kernel aggregator (Tier 2)` | decision_policy.cu (both kernels), tests/cbsw_aggregator.rs (5 new tests), harness.rs (revert Tier 1 stopgap upload, drop flag) | Local: all 5 cbsw_* tests pass. Cluster: smoke fires `n_trades > 100`. | +| Q1 | `feat(ml-backtesting): cold-start stopgap — max-confidence bytecode policy (Tier 1)` | sim/batched_config.rs (+ `use_cold_start_stopgap: bool` field on `UniformSimParams` and `BatchedSimConfig`), harness.rs (when flag set, build 7-instr bytecode program via `Strategy::flatten()` and upload via existing `sim.upload_program()` API), main.rs (CLI flag), sweep_smoke.yaml (set flag) | Local: existing `parallel_sim_correctness::*` + `threshold_and_cost::*` continue to pass (smoke-flag default false, no regression). Cluster: smoke fires `n_trades > 100`, `total_pnl_usd != 0`. | +| Q2 | `feat(ml-backtesting): CBSW kernel aggregator (Tier 2)` | decision_policy.cu (`decision_policy_default` body + 2 new `__device__` helpers + 2 `#define`s; `decision_policy_program` UNCHANGED), tests/cbsw_aggregator.rs (7 new tests), sim/batched_config.rs (drop `use_cold_start_stopgap` field entirely), harness.rs (drop the upload-when-flag-set branch), main.rs (drop the CLI flag), sweep_smoke.yaml (drop the flag setting) | Local: all 7 cbsw_* tests pass; pre-existing `decision_floor_coldstart`, `parallel_sim_correctness`, `threshold_and_cost`, `forward_graph_capture`, `save_load_roundtrip_preserves_all_weights` all still pass. Cluster: smoke fires `n_trades > 100`, `total_pnl_usd != 0`. | | Q3 | `docs(memory): pearl_conviction_bootstrap_for_kelly_aggregation` | new memory pearl file + MEMORY.md index update | Manual review. | | Q4 | `spec(ml-alpha): deployability sweep depends on CBSW` | append §3.4 cross-reference to the parallelism spec | n/a (doc-only) | -Q1 and Q2 land **before** running the deployability sweep on the cluster. Q3 and Q4 can land anytime after Q2 (documentation, no functional impact). +Q1 and Q2 land **before** running the deployability sweep on the cluster. Q3 and Q4 can land anytime after Q2 (documentation, no functional impact). The Tier 1 → Tier 2 transition is **atomic**: Q2 deletes the stopgap field and config-side wiring in the same commit that lands the proper kernel fix. No half-state where both paths exist. --- @@ -346,9 +430,11 @@ Pre-existing tests that MUST continue to pass: ## 8. Rate target validation -Tier 2 adds a sigmoid per horizon per decision = 5 extra `expf()` calls per decision. At the existing 2100 ev/s rate and stride=4, this is ~500k decisions/quarter × 5 expf ≈ 2.5M expf operations per quarter — negligible (~0.01% of forward cost on L40S). +Tier 2 replaces the `max(floor, recent_sharpe)` weight with a CBSW blend. Per-horizon per-decision cost added: ~6 arithmetic ops (subtraction, division, two `fmaxf/fminf`, one FMA, one comparison + branch) — no transcendentals, no memory reads beyond what was already done. At stride=4 with N_HORIZONS=5, that's ~30 ops × 500k decisions/quarter = 15M ops/quarter per backtest. Effectively free on L40S relative to the ~2 ms/decision forward cost. -No rate regression expected. Re-bench at Q2's smoke if curious; do not gate on it. +**Validation is a measurement, not an estimate.** Q2 smoke must produce `ev/s` rate within ±5% of the Q1 smoke's rate at matching `n_parallel` and stride. If the post-CBSW rate drops by more than 5%, something is wrong (likely the compiler didn't unroll the loop, or there's a register-spill from the extra locals). Gate Q2's commit on the measured rate ratio, not on the back-of-envelope estimate. + +Procedure: run a 1M-event Q1-mode smoke (Tier 1 active) on cluster; record ev/s. Then run the same 1M-event smoke at Q2's SHA (Tier 1 reverted, CBSW kernel live); record ev/s. Ratio must be ≥ 0.95. --- @@ -357,10 +443,10 @@ No rate regression expected. Re-bench at Q2's smoke if curious; do not gate on i | Risk | Likelihood | Mitigation | |---|---|---| | Tier 1 stopgap fires too many trades, blowing through max_lots constraints | Medium — max-confidence at MATURE regime is over-aggressive | Tier 1 is brief (one validation run); Tier 2 lands immediately after and reverts the stopgap. Tier 1's max_events stays capped at 2M to keep blast radius small. | -| Tier 2's sigmoid transition introduces non-monotonic sizing during the 5-15 trade window | Low | The transition is monotone in n_trades_seen by construction (sigmoid is monotone, both regimes' weights are monotonic in their drivers). The `cbsw_transition_smooth` test gates this. | +| Tier 2's piecewise-linear transition has a non-differentiable kink at n=K±half_width | Low | Mathematically clean: the transition is monotone, saturating, and continuous (linear pieces meet at the endpoints). Kelly aggregation doesn't differentiate weights, so non-differentiability is irrelevant. `cbsw_transition_smooth` test confirms monotonicity by sweeping n_trades_seen. | | Tier 2's tests pass but real-data behavior still has 0 trades for some unrelated reason | Medium | If Q2 smoke shows `n_trades = 0`, do NOT proceed to deployability. Diagnose first — likely a downstream sim-side bug (cap_lots collapse, fill failure, OCO trap). | -| The Tier 2 changes to `decision_policy_program` break the bytecode VM's other op-codes (OP_AGG_MEAN, OP_AGG_WEIGHTED_SHARPE) | Low | The CBSW logic only modifies how OP_AGG_WEIGHTED_SHARPE's final size is computed; other op-codes (EMIT_PER_HORIZON_SIZE, OP_AGG_MEAN, OP_AGG_MAX_CONFIDENCE) unchanged. Add a regression smoke for each op-code if confidence is low. | -| CBSW's max-confidence component over-emphasizes one horizon mature too | Medium — at sq=0.881 we still blend in 11.9% max-conf | Acceptable; the 11.9% blend ensures the model can fire on extreme single-horizon signals even after maturity. If problematic, narrow the sigmoid scale (5.0 → 2.0) to make the transition sharper. | +| Register pressure from the new locals (signed_sizes, weights, strong_h, strong_abs, sq_min_active, n_active_horizons) spills to local memory and tanks the kernel rate | Low — N_HORIZONS=5, total new locals are ~16 floats + 2 ints = 72 bytes, well within per-thread register budget on sm_89 (255 32-bit regs) | The §8 measurement gate (rate ratio ≥ 0.95) catches this. If it fires, refactor to reduce live ranges (compute strong_h via separate scan, drop ss_weighted accumulator and recompute post-loop). | +| Cold-start fires trades on noisy single-horizon convictions, losing on most | Medium — fundamental trade-off | This IS the design choice (§2 design-choice note). The empirical question is whether the model has enough alpha to break even on noisy cold-start trades while `recent_sharpe` is bootstrapping. The Q2 smoke's `total_pnl_usd` answers this empirically. If negative + large, escalate to a higher-conviction floor (`kelly_frac_floor` 0.20 → 0.40) to reduce trade size during the cold-start window. | --- @@ -379,7 +465,10 @@ No rate regression expected. Re-bench at Q2's smoke if curious; do not gate on i - Commits Q1–Q4 landed with all gating tests green. - Threshold-tuning smoke at threshold=0.0, cost=0.125 produces `n_trades > 100` and finite Sharpe/max-dd. - A second smoke at threshold=0.55 (≈ p60 from the threshold-tuning distribution), cost=0.125 produces a non-trivial trade count — proves the threshold gate + CBSW interact correctly. +- **Rate gate**: post-Q2 cluster smoke ev/s rate ≥ 95% of pre-Q2 (Tier 1 active) ev/s rate at matching 1M-event setup. Confirms CBSW's per-decision overhead is negligible. +- **All 7 cbsw_* unit tests pass on local RTX 3050 Ti** (per §4 test table); pre-existing CUDA tests still pass. +- **`feedback_no_legacy_aliases` compliance**: Q2 deletes `use_cold_start_stopgap` and `--cold-start-stopgap` entirely; grep for both terms returns zero hits post-Q2. - Pearl `pearl_conviction_bootstrap_for_kelly_aggregation.md` committed under memory/ and linked from MEMORY.md. - Parallelism spec (`2026-05-19-deployability-sweep-parallelism-design.md`) cross-references this spec. -- All pearls hold: permanent floor preserved, ISV-driven bootstrap, no version suffixes, no legacy aliases. +- All pearls hold: permanent floor preserved, ISV-driven bootstrap, no version suffixes, no legacy aliases, no host branches in captured graph (untouched), no `expf` in the new aggregator hot path. - Out-of-scope items are explicitly listed (above), not silently deferred.