gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

spec(ml-backtesting): CBSW cold-start aggregator design

Browse Source 
The post-trunk-grows threshold-tuning smoke (81decf40f) produced
n_trades=0 despite the model having a HEALTHY max-conviction
distribution (74.6% of decisions ≥ 0.30, 26.7% ≥ 0.70, full spread
across [0,1]). Diagnosis: the linear-weighted-mean aggregator in
decision_policy_default is structurally dilution-bound at cold-start
— single-horizon strong signals get washed out when uniform-floor
weights produce mean-over-horizons aggregation.

Solution: Conviction-Bootstrapped Sharpe Weighting (CBSW). Hybrid
max-confidence × weighted-sharpe with a per-horizon sigmoid transition
keyed on n_trades_seen vs MIN_TRADES_FOR_VAR_CAP. Cold-start: sig_mag
(decision-time conviction) drives weights AND max-confidence aggregator
fires single-horizon trades. Mature: recent_sharpe (historical) drives
weights AND linear-mean aggregator emphasizes strong-Sharpe horizons.
Permanent floor preserved per pearl_blend_formulas_must_have_permanent_floor.

Mirrors DQN bootstrap pattern (pearl_thompson_for_distributional_action_
selection): when historical estimates are uncertain, use available
signal as the bootstrap. Sig_mag is the ISV signal at decision time;
recent_sharpe is the ISV signal at trade-close time. Transition
self-terminates based on data accumulation, not time constants.

3-tier delivery (one spec, atomic commits):
  Q1: Bytecode VM stopgap — upload max-confidence 7-instruction
      program per backtest. Validates diagnosis; zero kernel work.
  Q2: Kernel CBSW — replace weight + aggregator in both decision
      kernels. 5 new regression tests covering cold/mature/transition.
  Q3: New memory pearl pearl_conviction_bootstrap_for_kelly_aggregation
      capturing the lesson.
  Q4: Cross-reference from parallelism spec (deployability sweep
      depends on CBSW being live to produce meaningful verdict).

The parallelism work (P1-P6) is fully working — confirmed by the
threshold-tuning smoke completing end-to-end (Succeeded status, 500k
decisions, artifacts written, aggregator parquet emitted). What's
blocked is the deployability VERDICT, because the dilution bug means
all variants would show n_trades=0 regardless of cost/latency/threshold.
CBSW unblocks the verdict.

Awaiting review before transitioning to writing-plans for Q1-Q4
implementation plan.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-05-19 21:38:55 +02:00
parent 81decf40f8
commit 1271d03931
1 changed files with 385 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

385
docs/superpowers/specs/2026-05-19-cbsw-cold-start-aggregator-design.md Normal file
View File

@@ -0,0 +1,385 @@
# CBSW Cold-Start Aggregator — Design
**Author:** brainstormed live 2026-05-19 with claude-opus-4-7
**Status:** Draft (awaiting review)
**Depends on:** `81decf40f` (P1–P6 of deployability-sweep parallelism landed + threshold-tuning smoke completed)
**Related specs:** `docs/superpowers/specs/2026-05-19-deployability-sweep-parallelism-design.md` (the parallelism work this builds on)
---
## 1. Problem
The 4-quarter × 140-variant deployability sweep is blocked on a **policy-aggregator structural defect**, not on infrastructure. The post-trunk-grows checkpoint (`dbd500ecf`, AUC h6000=0.76) produces zero trades over 500k decisions on real ES.FUT 2025-Q1 data, even with the threshold gate disabled (threshold=0.0) and frictionless cost.
### Evidence (from threshold-tuning smoke at commit `81decf40f`)
The model's max-over-horizons conviction distribution is **healthy**:
```
n_decisions = 499,992
mean = 0.498, std = 0.256, median = 0.460
p60 = 0.549, p70 = 0.661, p80 = 0.773, p90 = 0.869, p95 = 0.925
P(max_conv >= 0.30) = 74.6%
P(max_conv >= 0.50) = 45.2%
P(max_conv >= 0.70) = 26.7%
P(max_conv >= 0.90) = 7.0%
```
The model produces strong directional convictions on roughly half its decisions. Yet `n_trades = 0`.
### Diagnosis
`decision_policy_default` computes per-horizon signed sizes then aggregates linearly:
```
ss[h] = dir × sig_mag[h] × kelly_frac × cap_lots
w[h] = max(sharpe_weight_floor, recent_sharpe[h])
w_norm[h] = w[h] / Σ w[h]
final_size = Σ w_norm[h] × ss[h]
```
At cold-start (no closed trades), `recent_sharpe[h] = 0` for all h, so `w[h] = sharpe_weight_floor = 0.10` uniformly, so `w_norm[h] = 1/N_HORIZONS = 0.20`. The aggregator becomes **mean over horizons**.
**Linear weighted mean with weights summing to 1 has a hard upper bound**: `final_size ≤ max_h |ss[h]|`. For a single-horizon-strong signal (alpha = `[0.85, 0.50, 0.50, 0.50, 0.50]`), only one `ss[h] = 0.70`, others zero. `final_size = mean = 0.14`. After rounding, `lots = truncf(0.14 + 0.5) = 0` → noop.
To get `lots ≥ 1` from this case, the structural cost of the dilution requires either:
- `kelly_frac_floor × cap_lots × max_sig_mag > 5 × 0.5 = 2.5` (an oversized PERMANENT floor — violates `pearl_blend_formulas_must_have_permanent_floor.md` since post-bootstrap trades would be inflated), or
- A **non-linear aggregator** that doesn't dilute toward the mean
Parameter tunes cannot escape this; only the **aggregator type** can.
### Why we should care: this isn't a smoke quirk
In real ES.FUT data, single-horizon strong convictions are *more common* than consensus-strong signals. The model is trained per-horizon (h30, h100, h300, h1000, h6000) and each horizon has its own AUC (0.71-0.76). The horizons disagree often — by design, they predict different forward windows. A weighted-mean aggregator that requires near-consensus before firing trades structurally underuses the model.
Once `recent_sharpe[h]` is populated post-bootstrap, weighted-mean aggregation is *correct* because `recent_sharpe[h]` is asymmetric `[0, +∞)` — strong-Sharpe horizons get high weight, weak ones near-zero, and the linear sum correctly emphasizes the dominant horizons. **The bug is exclusively cold-start.**
---
## 2. Architecture — Conviction-Bootstrapped Sharpe Weighting (CBSW)
The fix mirrors the DQN-bootstrap pattern (per `pearl_thompson_for_distributional_action_selection` + `pearl_c51_thompson_closed_phase_e3_gap`): **when historical estimates are uncertain, use available signal as the bootstrap**. For DQN that's Thompson-sampling over the Q distribution. For our policy that's using the decision-time conviction (`sig_mag`) as the per-horizon weight signal until `recent_sharpe[h]` becomes trustworthy.
### Two-axis change
CBSW changes both the **weight formula** AND the **aggregation type**, transitioning smoothly from cold-start to mature regimes based on a per-horizon signal-quality factor.
#### Per-horizon signal-quality (drives the transition)
```
n_h = isv[h].n_trades_seen
K = MIN_TRADES_FOR_VAR_CAP = 10 (already exists, sample-size threshold)
sigmoid_scale = 5.0 (half-transition width)
signal_quality[h] = sigmoid((n_h - K) / sigmoid_scale)
```
Properties:
- `n=0` → `sq = sigmoid(-2) ≈ 0.119` (cold)
- `n=K-5=5` → `sq = sigmoid(-1) ≈ 0.269` (transitioning)
- `n=K=10` → `sq = sigmoid(0) = 0.500` (mid-transition)
- `n=K+5=15` → `sq = sigmoid(+1) ≈ 0.731` (transitioning)
- `n=K+10=20` → `sq = sigmoid(+2) ≈ 0.881` (nearly mature)
Smooth, monotonic, signal-driven by `n_trades_seen` (the ISV-side sample count). Reuses the existing `MIN_TRADES_FOR_VAR_CAP` threshold so there's no new tuning knob.
#### Weight formula (blended per-horizon)
```
sharpe_weight_h = max(0, recent_sharpe[h]) // asymmetric [0, +∞)
conviction_weight_h = sig_mag[h] // symmetric [0, 1]
w_blend = (1 - signal_quality[h]) * conviction_weight_h
+ signal_quality[h] * sharpe_weight_h
w[h] = max(sharpe_weight_floor, w_blend) // permanent floor per pearl
```
At cold-start the weight = `max(floor, sig_mag[h])` — strong-conviction horizons get high weight automatically. At mature regime the weight = `max(floor, recent_sharpe[h])` — existing behavior preserved.
#### Aggregator (hybrid max-confidence + weighted-sharpe)
Weighted-mean alone can't escape the dilution bound. We need max-confidence at cold-start. Hybrid:
```
strong_h = argmax_h |ss[h]| // max-confidence pick
ss_strong = ss[strong_h]
ss_weighted = Σ w_norm[h] × ss[h]
sq_min = min over h of signal_quality[h] // transition gate
final_size = (1 - sq_min) * ss_strong + sq_min * ss_weighted
```
At cold-start: 100% max-confidence → strong-horizon-only trades fire.
At mature: 100% weighted-sharpe → existing behavior.
Transition: smooth blend over the same K=10-trade threshold.
### Math walk-through (cold-start, single-horizon strong)
`alpha = [0.85, 0.50, 0.50, 0.50, 0.50]`, n=0 everywhere, sq_min ≈ 0.119:
| Quantity | Value | Note |
|---|---|---|
| `sig_mag` | `[0.70, 0, 0, 0, 0]` | |
| `dir` | `[+1, +1, +1, +1, +1]` | doesn't matter for sig_mag=0 |
| `ss = dir × sig_mag × 0.20 × 5` | `[0.70, 0, 0, 0, 0]` | kelly_floor=0.20, cap=max_lots=5 |
| `w` (with floor) | `[max(0.10, 0.70), 0.10, 0.10, 0.10, 0.10]` = `[0.70, 0.10, 0.10, 0.10, 0.10]` | sig_mag → high weight on strong h |
| `w_sum` | 1.10 | |
| `w_norm` | `[0.636, 0.091, 0.091, 0.091, 0.091]` | |
| `ss_weighted` | `0.636 × 0.70 + 0 = 0.445` | linear-mean piece |
| `ss_strong` | `0.70` | max-conf pick |
| `final_size` | `(1 − 0.119) × 0.70 + 0.119 × 0.445 = 0.616 + 0.053 = 0.669` | |
| `lots = truncf(0.669 + 0.5)` | **1** | trade fires |
Single-horizon strong signal at cold-start now fires a 1-lot trade.
### Math walk-through (weak conflict)
`alpha = [0.55, 0.45, 0.55, 0.45, 0.55]`, n=0, sq_min ≈ 0.119:
| Quantity | Value |
|---|---|
| `sig_mag` | `[0.10, 0.10, 0.10, 0.10, 0.10]` |
| `dir` | `[+1, −1, +1, −1, +1]` |
| `ss` | `[+0.10, −0.10, +0.10, −0.10, +0.10]` |
| `strong_h` (argmax of \|ss\|) | any (ties); pick h=0; `ss_strong = +0.10` |
| `w` | `[max(0.10, 0.10)] × 5 = [0.10] × 5` (all equal) |
| `w_norm` | `[0.20] × 5` |
| `ss_weighted` | `0.20 × (0.10 − 0.10 + 0.10 − 0.10 + 0.10) = 0.02` |
| `final_size` | `0.881 × 0.10 + 0.119 × 0.02 = 0.091` |
| `lots = truncf(0.091 + 0.5)` | **0** → noop |
Weak conflicting signals correctly don't trade.
### Math walk-through (mature regime, n=20 everywhere)
`alpha = [0.85, 0.50, 0.50, 0.50, 0.50]`, sq_min ≈ 0.881, `recent_sharpe = [1.5, 0.1, 0.1, 0.1, 0.1]` (model converged: h0 is the best horizon):
| Quantity | Value |
|---|---|
| `w_blend` | `0.119 × sig_mag + 0.881 × sharpe = [0.083+1.322, 0.0+0.088, ...] = [1.405, 0.088, 0.088, 0.088, 0.088]` |
| `w` (with floor) | `[1.405, 0.10, 0.10, 0.10, 0.10]` |
| `w_norm` | `[0.778, 0.055, 0.055, 0.055, 0.055]` |
| `ss_weighted` | `0.778 × 0.70 = 0.544` |
| `ss_strong` | `0.70` |
| `final_size` | `0.119 × 0.70 + 0.881 × 0.544 = 0.083 + 0.479 = 0.562` |
| `lots` | **1** |
Mature regime preserves the asymmetric Sharpe-driven weighting. Strong-Sharpe horizon dominates correctly.
---
## 3. Tier 1 — Bytecode VM stopgap (validation)
**Goal:** Validate the diagnosis with zero kernel work. The existing `decision_policy_program` kernel has `OP_AGG_MAX_CONFIDENCE` already implemented and exposed via the bytecode VM. Upload a 7-instruction program per backtest at harness construction:
```
EMIT_PER_HORIZON_SIZE(h=0, leaf_max_lots=max_lots)
EMIT_PER_HORIZON_SIZE(h=1, leaf_max_lots=max_lots)
EMIT_PER_HORIZON_SIZE(h=2, leaf_max_lots=max_lots)
EMIT_PER_HORIZON_SIZE(h=3, leaf_max_lots=max_lots)
EMIT_PER_HORIZON_SIZE(h=4, leaf_max_lots=max_lots)
AGG_MAX_CONFIDENCE(n=5)
WRITE_ORDER
```
When `program_lens[b] > 0`, the bytecode kernel handles the backtest and the default kernel skips. Single-horizon strong signals immediately fire trades at cold-start; mature behavior degrades to pure max-confidence (which over-emphasizes single-horizon signals once recent_sharpe is populated — but only for the *brief* stopgap window before Tier 2 replaces this).
**Files:**
- `crates/ml-backtesting/src/harness.rs` — at construction, generate the 3-instruction bytecode program for each backtest if `cfg.use_cold_start_stopgap` (new bool, default false). Compile via `Strategy::flatten()` (existing API) or directly emit `Vec<Instruction>` matching the cuda struct layout.
- `crates/ml-backtesting/src/sim/batched_config.rs` + `UniformSimParams` — `use_cold_start_stopgap: bool` flag.
- `bin/fxt-backtest/src/main.rs` — CLI `--cold-start-stopgap` flag → passes through to BacktestHarnessConfig.
- `config/ml/sweep_smoke.yaml` — `sim_variants:` entry gains `use_cold_start_stopgap: true` for the next validation run.
**Validation gate:** smoke at threshold=0.0, cost=0.125, max_events=2M, use_cold_start_stopgap=true → `n_trades > 100` AND `total_pnl_usd != 0`. Proves trades are firing through the GPU pipeline.
**Out of scope for Tier 1:** mature-regime correctness. The bytecode max-confidence over-trades after recent_sharpe is populated. Tier 1 is *purely* a diagnosis confirmation.
---
## 4. Tier 2 — Kernel CBSW (proper architecture)
**Goal:** Replace both decision kernels' weight + aggregation logic with the CBSW formula from §2. Tier 1's bytecode stopgap becomes unnecessary and is reverted in the same commit.
**Kernel signature additions** (both `decision_policy_default` and `decision_policy_program`):
```cuda
// New per-backtest scalar args (kept symmetric with existing P4 floors)
const float* sigmoid_scale_per_b, // = 5.0 default
int min_trades_for_trust_per_b // existing MIN_TRADES_FOR_VAR_CAP, now per-b
```
Actually — to minimize per-call upload overhead and keep the kernel ABI tight, use file-scope `#define` for both:
```cuda
#define MIN_TRADES_FOR_TRUST 10u
#define SIGMOID_SCALE 5.0f
```
(Both are sample-size thresholds — not adaptive bounds — so they satisfy `pearl_isv_for_adaptive_bounds.md`. Constants reused across the codebase: same K=10 as `MIN_TRADES_FOR_VAR_CAP`.)
**Kernel body changes:**
```cuda
// Inside the per-horizon loop, compute sig_mag and signal_quality:
const float n_h = (float)isv[h].n_trades_seen;
const float sq_h = 1.0f / (1.0f + expf(-(n_h - (float)MIN_TRADES_FOR_TRUST) / SIGMOID_SCALE));
// Weight: blend sig_mag (cold) and recent_sharpe (mature), with permanent floor.
const float sharpe_w = fmaxf(0.0f, isv[h].recent_sharpe);
const float conv_w = sig_mag; // already computed for ss[h]
const float w_blend = (1.0f - sq_h) * conv_w + sq_h * sharpe_w;
weights[h] = fmaxf(sharpe_weight_floor, w_blend);
w_sum += weights[h];
// Track strong_h for max-confidence aggregation
if (fabsf(signed_sizes[h]) > fabsf(signed_sizes[strong_h])) {
strong_h = h;
}
sq_min = fminf(sq_min, sq_h); // gate for the hybrid aggregator
```
```cuda
// After the per-horizon loop, hybrid aggregation:
float final_size = 0.0f;
if (w_sum > 1e-9f) {
float ss_weighted = 0.0f;
#pragma unroll
for (int h = 0; h < N_HORIZONS; ++h) {
ss_weighted += (weights[h] / w_sum) * signed_sizes[h];
if (weights[h] > 1e-9f) attribution_mask |= (1u << h);
}
const float ss_strong = signed_sizes[strong_h];
final_size = (1.0f - sq_min) * ss_strong + sq_min * ss_weighted;
}
```
**Files:**
- `crates/ml-backtesting/cuda/decision_policy.cu` — both kernels (default + program). ~30 LOC each.
- `crates/ml-backtesting/src/harness.rs` + sim — revert the Tier 1 stopgap bytecode upload.
- `crates/ml-backtesting/tests/cbsw_aggregator.rs` (NEW) — unit tests per §2 math walk-throughs.
**Tests:**
| Test | Setup | Assertion |
|---|---|---|
| `cbsw_cold_start_single_horizon_strong_fires_trade` | n=0 all h, alpha=[0.85,0.50,0.50,0.50,0.50] | side=0, size ≥ 1 |
| `cbsw_cold_start_weak_conflict_does_not_trade` | n=0, alpha=[0.55,0.45,0.55,0.45,0.55] | side=2, size=0 |
| `cbsw_cold_start_consensus_strong_fires_trade` | n=0, alpha=[0.85]×5 | side=0, size ≥ 1 |
| `cbsw_mature_recovers_weighted_sharpe` | n=20 all h, recent_sharpe=[1.5,0.1,0.1,0.1,0.1], alpha=[0.85,0.50,0.50,0.50,0.50] | side=0, size ≥ 1, ss_weighted contribution > 0.5 of final |
| `cbsw_transition_smooth` | n=K all h, alpha=[0.70,0.50,0.50,0.50,0.50] | size monotone in n_trades_seen sweep |
**Validation gate:** Re-run threshold-tuning smoke at threshold=0.0, cost=0.125. Expected `n_trades > 100`, `total_pnl_usd != 0`. Cross-check that Tier 1 and Tier 2 produce similar n_trades counts at cold-start (both should fire on single-horizon strong signals). Tier 2 should be *better* than Tier 1 once trades start closing (recent_sharpe takes over, max-confidence over-trading subsides).
---
## 5. Tier 3 — Pearl + spec annotation
**New pearl:** `~/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/pearl_conviction_bootstrap_for_kelly_aggregation.md`
```markdown
---
name: pearl-conviction-bootstrap-for-kelly-aggregation
description: Linear-weighted-mean aggregation over per-horizon ss[] is structurally dilution-bound at cold-start (single-horizon strong → final_size ≤ max_h ss[h] but typically much less). Fix: hybrid max-confidence × weighted-sharpe with sigmoid transition keyed on n_trades_seen — let conviction (sig_mag) bootstrap weights until recent_sharpe converges.
metadata:
type: pearl
---
[content covering: the diagnosis, the math (linear mean ≤ max ss; uniform weights → mean), the DQN analog (Thompson-sample uncertain estimates), CBSW design, when to apply (multi-source aggregation where one source can be "convicted" alone and others lag).]
Canonical instance: ml-alpha decision_policy on dbd500ecf checkpoint (2026-05-19).
The model produced 26.7% of decisions with max_conv ≥ 0.70 yet n_trades=0 because
the weighted-mean aggregator diluted single-horizon strong signals at cold-start.
Related: [[pearl-blend-formulas-must-have-permanent-floor]] (floor preserved),
[[pearl-thompson-for-distributional-action-selection]] (DQN parallel),
[[pearl-first-observation-bootstrap]] (signal_quality transition).
```
**Spec annotation in `2026-05-19-deployability-sweep-parallelism-design.md`**: add a §3.4 noting that the deployability sweep on checkpoint `dbd500ecf` requires CBSW (this spec) to be live, otherwise the verdict is uninformative due to the cold-start dilution bug. Reference this spec by commit SHA.
**Broader audit:** scan the codebase for other linear-weighted-mean aggregations over heterogeneous sources where one source can be uncertain. Candidate sites (to investigate, not to fix in this spec):
- `crates/ml-alpha/src/...` — multi-horizon training loss weighting (`Σ λ_h × BCE[h]`)
- Any controller's `recent_sharpe`-keyed weighting elsewhere in the sim/policy stack
If found, log as separate spec follow-ups. Don't expand THIS spec's scope.
---
## 6. Migration plan (atomic commits)
Per `feedback_no_partial_refactor`: each commit is internally consistent and ships its own tests.
| Commit | Subject | Files | Test gate |
|---|---|---|---|
| Q1 | `feat(ml-backtesting): cold-start stopgap — max-confidence bytecode policy (Tier 1)` | sim/batched_config.rs (+ use_cold_start_stopgap flag), harness.rs (upload bytecode program when flag set), main.rs (CLI flag), sweep_smoke.yaml (set flag) | Local: harness builds bytecode program of correct shape. Cluster: smoke fires `n_trades > 100`. |
| Q2 | `feat(ml-backtesting): CBSW kernel aggregator (Tier 2)` | decision_policy.cu (both kernels), tests/cbsw_aggregator.rs (5 new tests), harness.rs (revert Tier 1 stopgap upload, drop flag) | Local: all 5 cbsw_* tests pass. Cluster: smoke fires `n_trades > 100`. |
| Q3 | `docs(memory): pearl_conviction_bootstrap_for_kelly_aggregation` | new memory pearl file + MEMORY.md index update | Manual review. |
| Q4 | `spec(ml-alpha): deployability sweep depends on CBSW` | append §3.4 cross-reference to the parallelism spec | n/a (doc-only) |
Q1 and Q2 land **before** running the deployability sweep on the cluster. Q3 and Q4 can land anytime after Q2 (documentation, no functional impact).
---
## 7. Tests
### Unit (CUDA `--ignored`, run locally on RTX 3050 Ti)
All five `cbsw_*` tests from §4 land in Q2. Each is one-decision, deterministic — fast.
Pre-existing tests that MUST continue to pass:
| Test | Rationale |
|---|---|
| `decision_floor_coldstart::*` (×3) | Mature behavior unchanged when isv state is seeded |
| `parallel_sim_correctness::parallel_sim_equivalence_with_uniform_config` | n=8 uniform config still produces 8 bit-identical results under CBSW |
| `threshold_and_cost::threshold_gate_*` (×3) | Threshold gate is upstream of CBSW; behavior unchanged |
| `forward_graph_capture::forward_captured_matches_uncaptured` | Forward path unchanged; capture still bit-near-equivalent |
| `save_load_roundtrip_preserves_all_weights` | Checkpoint unchanged |
### Integration (cluster smoke)
| Gate | Config | Threshold |
|---|---|---|
| After Q1 lands | `--cold-start-stopgap` smoke at threshold=0.0, cost=0.125, max_events=2M | `n_trades > 100`, total_pnl finite |
| After Q2 lands | Same as Q1 but without `--cold-start-stopgap` flag (CBSW kernel handles it natively) | `n_trades > 100`, total_pnl finite, `n_trades_Q2 >= n_trades_Q1 × 0.5` (Q2 may be more conservative than Q1's pure max-confidence but should fire comparable order-of-magnitude trades) |
---
## 8. Rate target validation
Tier 2 adds a sigmoid per horizon per decision = 5 extra `expf()` calls per decision. At the existing 2100 ev/s rate and stride=4, this is ~500k decisions/quarter × 5 expf ≈ 2.5M expf operations per quarter — negligible (~0.01% of forward cost on L40S).
No rate regression expected. Re-bench at Q2's smoke if curious; do not gate on it.
---
## 9. Risks
| Risk | Likelihood | Mitigation |
|---|---|---|
| Tier 1 stopgap fires too many trades, blowing through max_lots constraints | Medium — max-confidence at MATURE regime is over-aggressive | Tier 1 is brief (one validation run); Tier 2 lands immediately after and reverts the stopgap. Tier 1's max_events stays capped at 2M to keep blast radius small. |
| Tier 2's sigmoid transition introduces non-monotonic sizing during the 5-15 trade window | Low | The transition is monotone in n_trades_seen by construction (sigmoid is monotone, both regimes' weights are monotonic in their drivers). The `cbsw_transition_smooth` test gates this. |
| Tier 2's tests pass but real-data behavior still has 0 trades for some unrelated reason | Medium | If Q2 smoke shows `n_trades = 0`, do NOT proceed to deployability. Diagnose first — likely a downstream sim-side bug (cap_lots collapse, fill failure, OCO trap). |
| The Tier 2 changes to `decision_policy_program` break the bytecode VM's other op-codes (OP_AGG_MEAN, OP_AGG_WEIGHTED_SHARPE) | Low | The CBSW logic only modifies how OP_AGG_WEIGHTED_SHARPE's final size is computed; other op-codes (EMIT_PER_HORIZON_SIZE, OP_AGG_MEAN, OP_AGG_MAX_CONFIDENCE) unchanged. Add a regression smoke for each op-code if confidence is low. |
| CBSW's max-confidence component over-emphasizes one horizon mature too | Medium — at sq=0.881 we still blend in 11.9% max-conf | Acceptable; the 11.9% blend ensures the model can fire on extreme single-horizon signals even after maturity. If problematic, narrow the sigmoid scale (5.0 → 2.0) to make the transition sharper. |
---
## 10. Out of scope
- Training-time policy updates (this spec is inference-only).
- Bf16 mixed precision (deferred from parallelism spec).
- Per-horizon `recent_sharpe` priming from training-time AUC (alternative bootstrap approach — could be a Tier 4 in a follow-up; CBSW is simpler).
- Code audit for other linear-weighted-mean aggregations (flagged in §5, scoped as separate spec follow-ups).
- Replacing OP_AGG_WEIGHTED_SHARPE in the bytecode VM with CBSW (only the default kernel is migrated; the VM remains pluggable for future policy experiments).
---
## 11. Definition of done
- Commits Q1–Q4 landed with all gating tests green.
- Threshold-tuning smoke at threshold=0.0, cost=0.125 produces `n_trades > 100` and finite Sharpe/max-dd.
- A second smoke at threshold=0.55 (≈ p60 from the threshold-tuning distribution), cost=0.125 produces a non-trivial trade count — proves the threshold gate + CBSW interact correctly.
- Pearl `pearl_conviction_bootstrap_for_kelly_aggregation.md` committed under memory/ and linked from MEMORY.md.
- Parallelism spec (`2026-05-19-deployability-sweep-parallelism-design.md`) cross-references this spec.
- All pearls hold: permanent floor preserved, ISV-driven bootstrap, no version suffixes, no legacy aliases.
- Out-of-scope items are explicitly listed (above), not silently deferred.