Files
foxhunt/docs/superpowers/specs
jgrusewski 1271d03931 spec(ml-backtesting): CBSW cold-start aggregator design
The post-trunk-grows threshold-tuning smoke (81decf40f) produced
n_trades=0 despite the model having a HEALTHY max-conviction
distribution (74.6% of decisions ≥ 0.30, 26.7% ≥ 0.70, full spread
across [0,1]). Diagnosis: the linear-weighted-mean aggregator in
decision_policy_default is structurally dilution-bound at cold-start
— single-horizon strong signals get washed out when uniform-floor
weights produce mean-over-horizons aggregation.

Solution: Conviction-Bootstrapped Sharpe Weighting (CBSW). Hybrid
max-confidence × weighted-sharpe with a per-horizon sigmoid transition
keyed on n_trades_seen vs MIN_TRADES_FOR_VAR_CAP. Cold-start: sig_mag
(decision-time conviction) drives weights AND max-confidence aggregator
fires single-horizon trades. Mature: recent_sharpe (historical) drives
weights AND linear-mean aggregator emphasizes strong-Sharpe horizons.
Permanent floor preserved per pearl_blend_formulas_must_have_permanent_floor.

Mirrors DQN bootstrap pattern (pearl_thompson_for_distributional_action_
selection): when historical estimates are uncertain, use available
signal as the bootstrap. Sig_mag is the ISV signal at decision time;
recent_sharpe is the ISV signal at trade-close time. Transition
self-terminates based on data accumulation, not time constants.

3-tier delivery (one spec, atomic commits):
  Q1: Bytecode VM stopgap — upload max-confidence 7-instruction
      program per backtest. Validates diagnosis; zero kernel work.
  Q2: Kernel CBSW — replace weight + aggregator in both decision
      kernels. 5 new regression tests covering cold/mature/transition.
  Q3: New memory pearl pearl_conviction_bootstrap_for_kelly_aggregation
      capturing the lesson.
  Q4: Cross-reference from parallelism spec (deployability sweep
      depends on CBSW being live to produce meaningful verdict).

The parallelism work (P1-P6) is fully working — confirmed by the
threshold-tuning smoke completing end-to-end (Succeeded status, 500k
decisions, artifacts written, aggregator parquet emitted). What's
blocked is the deployability VERDICT, because the dilution bug means
all variants would show n_trades=0 regardless of cost/latency/threshold.
CBSW unblocks the verdict.

Awaiting review before transitioning to writing-plans for Q1-Q4
implementation plan.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 21:38:55 +02:00
..