Files
foxhunt/crates/ml-backtesting/tests/decision_floor_coldstart.rs
jgrusewski fef5939556 feat(ml-backtesting): cold-start stopgap — max-confidence bytecode policy (Q1/Tier1)
The threshold-tuning smoke at 81decf40f produced n_trades=0 despite
74.6% of decisions having max_conv ≥ 0.30 — the linear-weighted-mean
aggregator in decision_policy_default is structurally dilution-bound
at cold-start (per spec §1).

Q1 stopgap: when sim_variants[i].use_cold_start_stopgap = true, the
harness uploads a max-confidence Strategy bytecode program for that
backtest, routing decisions through decision_policy_program with
OP_AGG_MAX_CONFIDENCE. Existing kernel; zero CUDA changes.

Field additions (atomically across BatchedSimConfig + UniformSimParams
+ ResolvedSimVariant + SweepBase.SimVariant) — every UniformSimParams
literal migrated to include use_cold_start_stopgap: false (default).
The sweep YAML's sim_variants entry sets it to true only for the
validation run; production deployability uses Q2's kernel fix instead.

Sweep YAML (config/ml/sweep_smoke.yaml) flipped to use_cold_start_stopgap=true
at threshold=0.0, cost=0.125 — same anchor as the threshold-tuning
smoke that produced n_trades=0, for direct comparison.

This is a VALIDATION step. Cluster smoke at this commit MUST produce
n_trades > 100 + finite metrics. Q2's kernel CBSW immediately follows
and deletes this entire stopgap atomically (field, harness branch,
YAML setting, every literal).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 22:00:52 +02:00

120 lines
5.2 KiB
Rust

//! Regression test for the cold-start sentinel-skip bug surfaced during
//! the trunk-grows smoke (2026-05-19). Before the kernel floor was
//! added, `step_decision*` would observe sentinel `isv_kelly_d` (all
//! zeros from `alloc_zeros`) and skip every horizon → `market_target =
//! (noop, 0)` forever, which prevented any trade from ever firing.
//!
//! Per `pearl_blend_formulas_must_have_permanent_floor.md` and
//! `pearl_kelly_cap_signal_driven_floors.md`: the decision policy uses
//! `max(floor, computed)` on Kelly fraction AND on the recent-Sharpe
//! aggregation weight so cold-start produces a non-zero target.
//!
//! Asserts: with sentinel isv_kelly_d, strong directional alpha
//! (p_h=0.8 ⇒ long) + floors > 0 ⇒ market_target side=0 (buy),
//! size >= 1 lot.
use anyhow::Result;
use ml_backtesting::policy::IsvKellyStateHost;
use ml_backtesting::sim::{BatchedSimConfig, LobSimCuda, UniformSimParams};
use ml_core::device::MlDevice;
fn cfg_uniform(n: usize, kelly: f32, sharpe: f32) -> BatchedSimConfig {
BatchedSimConfig::from_uniform(n, &UniformSimParams {
target_annual_vol_units: 50.0,
annualisation_factor: 825.0,
max_lots: 5,
latency_ns: 0,
kelly_frac_floor: kelly,
sharpe_weight_floor: sharpe,
threshold: 0.0,
cost_per_lot_per_side: 0.0,
use_cold_start_stopgap: false,
})
}
#[test]
#[ignore = "requires CUDA"]
fn cold_start_sentinel_state_still_fires_a_trade() -> Result<()> {
let dev = match MlDevice::cuda(0) {
Ok(d) => d,
Err(e) => {
eprintln!("skipping: cuda device unavailable ({e})");
return Ok(());
}
};
let mut sim = LobSimCuda::new(1, &dev)?;
// Do NOT seed isv_kelly — leave at zeros (alloc_zeros' sentinel).
// Strong directional alpha across all horizons → conviction-driven
// sig_mag = 0.6 for every horizon, dir = +1.
sim.broadcast_alpha(&[0.8, 0.8, 0.8, 0.8, 0.8])?;
sim.step_decision_with_latency(0, &cfg_uniform(1, 0.20, 0.10))?;
let (side, size) = sim.read_market_target(0)?;
assert_eq!(side, 0, "cold-start with p_h=0.8 must produce a long; got side={side}");
assert!(size >= 1, "cold-start size {size} < 1 — the kernel floor isn't firing");
Ok(())
}
/// After the first trade closes as a loss, the original kernel set
/// `realised_return_var = ret²` which collapses `cap_units` to ~0 and
/// permanently locks the policy out of further trading despite strong
/// alpha signal. The fix gates the variance-derived cap behind a
/// sample-size threshold (`n_trades_seen >= MIN_TRADES_FOR_VAR_CAP`)
/// so cap_lots falls back to `max_lots` while statistics are unreliable.
///
/// Test: write an IsvKellyState with n_trades_seen=1 and a large
/// realised_return_var (mimicking the post-loss state from the smoke),
/// then prove that the decision kernel still produces a non-zero trade.
#[test]
#[ignore = "requires CUDA"]
fn post_first_loss_state_does_not_lock_out_further_trades() -> Result<()> {
let dev = match MlDevice::cuda(0) {
Ok(d) => d,
Err(e) => {
eprintln!("skipping: cuda device unavailable ({e})");
return Ok(());
}
};
let mut sim = LobSimCuda::new(1, &dev)?;
// Seed isv_kelly_d with the exact state pattern the smoke produced:
// one closed-loss trade, large realised_return_var. Pre-fix, the
// variance-derived cap collapses to ~0.
let post_loss: [IsvKellyStateHost; 5] = std::array::from_fn(|_| IsvKellyStateHost {
pnl_ema_win: 0.0,
pnl_ema_loss: 10.18, // magnitude of the lone loss return
win_rate_ema: 0.0,
n_trades_seen: 1, // exactly 1 closed trade — under MIN_TRADES_FOR_VAR_CAP
realised_return_var: 103.6, // ret² from the smoke (10.18²)
recent_sharpe: -1.0, // very negative — would have starved the weight side too
});
sim.write_isv_kelly(0, &post_loss)?;
sim.broadcast_alpha(&[0.8, 0.8, 0.8, 0.8, 0.8])?;
sim.step_decision_with_latency(0, &cfg_uniform(1, 0.20, 0.10))?;
let (side, size) = sim.read_market_target(0)?;
assert_eq!(side, 0, "post-loss state must still fire a long with strong alpha (got side={side})");
assert!(size >= 1, "post-loss size {size} < 1 — n_trades_seen gate isn't bypassing the variance cap");
Ok(())
}
#[test]
#[ignore = "requires CUDA"]
fn cold_start_with_zero_floor_reproduces_old_bug() -> Result<()> {
// Mirror of the test above but with floors = 0 — the kernel must
// then behave like the old sentinel-skip pre-fix code: no trade ever
// fires. Lets us prove the fix actually changes behaviour (and not
// some other unrelated code path).
let dev = match MlDevice::cuda(0) {
Ok(d) => d,
Err(e) => {
eprintln!("skipping: cuda device unavailable ({e})");
return Ok(());
}
};
let mut sim = LobSimCuda::new(1, &dev)?;
sim.broadcast_alpha(&[0.8, 0.8, 0.8, 0.8, 0.8])?;
sim.step_decision_with_latency(0, &cfg_uniform(1, 0.0, 0.0))?;
let (side, size) = sim.read_market_target(0)?;
assert_eq!(side, 2, "with zero floors, sentinel state must still skip (side=noop)");
assert_eq!(size, 0);
Ok(())
}