Files
foxhunt/crates/ml-backtesting/cuda/book_update.cu
jgrusewski c03cf9aa38 fix(ml-backtesting): parameterized price-range sanitization replaces hardcoded 1e8
Audit (fxt-data-audit on ES.FUT_2024-Q1) revealed real source-data
outliers that the hardcoded < 1e8 threshold didn't catch:
- bid_min = -$4.85 (negative bid)
- bid_p1 = $64.15 (1% of bids in sub-$100 range, far below ES)
- ask_max = $53,012 (10x above any plausible ES price)

The < 1e8 threshold = $100M was useless: never triggered on real ES.

Fix: parameterize the range. min_reasonable_px / max_reasonable_px as
per-backtest fields in UniformSimParams, ResolvedSimVariant, and
BatchedSimConfig (defaults 0.0 / f32::INFINITY = no-check, preserves
existing test fixtures). Sweep_smoke.yaml sets 1000/20000 for ES
futures — catches all observed outliers without rejecting any plausible
price. BacktestHarness calls upload_price_range() once after creating
the sim so the bounds are active before the first apply_snapshot.

CUDA kernel book_update_apply_snapshot gains two new args
(min_reasonable_px[n_backtests], max_reasonable_px[n_backtests]) that
replace the hardcoded > 0.0f && < 1.0e8f checks at both the top-of-book
gate and the per-level sanitization pass.

Test price_range_rejection_skips_snapshot validates: sub-$1000
snapshot, super-$20000 snapshot, and negative bid all skip with
snapshots_skipped counter increment; valid ES snapshot passes through.
17/17 stop_controller tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 12:31:43 +02:00

97 lines
4.6 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
// book_update.cu — MBP-10 snapshot apply.
//
// Replaces the per-block book state with the broadcast snapshot. The
// full top-10 image overwrites the previous state (MBP-10 semantics).
// One block per backtest; thread tid handles one of 10 levels per side
// (10 threads used out of 32 in the warp; remainder idle).
//
// Per feedback_no_atomicadd.md: no atomics — single-writer per level.
// Per spec §2: same snapshot broadcast to every backtest in v1.
#include "lob_state.cuh"
// Books are laid out [n_backtests, 4 fields × 10 levels].
// Field order: bid_px, bid_sz, ask_px, ask_sz (one contiguous Book per backtest).
extern "C" __global__ void book_update_apply_snapshot(
const float* __restrict__ bid_px_in, // [10] — broadcast
const float* __restrict__ bid_sz_in, // [10]
const float* __restrict__ ask_px_in, // [10]
const float* __restrict__ ask_sz_in, // [10]
Book* __restrict__ books_out, // [n_backtests]
float* __restrict__ prev_mid, // [n_backtests]
float* __restrict__ atr_mid_ema, // [n_backtests]
unsigned int* __restrict__ snapshots_skipped, // [n_backtests] — corrupt-top-of-book counter
const float* __restrict__ min_reasonable_px, // [n_backtests] — 0.0 = no lower bound
const float* __restrict__ max_reasonable_px, // [n_backtests] — +inf = no upper bound
int n_backtests
) {
int b = blockIdx.x;
int tid = threadIdx.x;
if (b >= n_backtests) return;
// Per-backtest price-range bounds (S1.19: instrument-aware sanitization).
// Defaults 0.0 / +inf passed from LobSimCuda::new; callers that set
// instrument-specific limits (e.g. ES: 1000/20000) catch real outliers.
const float min_px = min_reasonable_px[b];
const float max_px = max_reasonable_px[b];
// Top-of-book validity check: if best bid/ask is unhealthy, skip the
// entire snapshot. Otherwise per-level sanitization (Task 15) would
// zero ALL levels at session boundaries, leaving vwap_entry / fill
// computations to read 0 prices and produce zero-sentinel trade
// records (Bug C-b from smoke v74v4 2026-05-20).
const bool top_ok = isfinite(bid_px_in[0]) && isfinite(ask_px_in[0])
&& isfinite(bid_sz_in[0]) && isfinite(ask_sz_in[0])
&& bid_sz_in[0] > 0.0f && ask_sz_in[0] > 0.0f
&& bid_px_in[0] >= min_px && bid_px_in[0] <= max_px
&& ask_px_in[0] >= min_px && ask_px_in[0] <= max_px;
if (!top_ok) {
if (tid == 0) snapshots_skipped[b] += 1u;
return; // Leave book/prev_mid/atr_mid_ema unchanged.
}
if (tid < 10) {
// Per-level sanitization for levels 0..9. Top-of-book (tid==0) is
// already validated above; below-top levels can still be NaN/Inf and
// get zeroed so walk_*_for_* (which skips lvl_sz <= 0) treats them
// as empty. Real MBP-10 data at session boundaries / gap-fills can
// contain NaN or Inf at deeper levels even when top-of-book is valid.
Book& bk = books_out[b];
const float bp = bid_px_in[tid];
const float bs = bid_sz_in[tid];
const float ap = ask_px_in[tid];
const float az = ask_sz_in[tid];
const bool bid_ok = isfinite(bp) && isfinite(bs) && bs > 0.0f
&& bp >= min_px && bp <= max_px;
const bool ask_ok = isfinite(ap) && isfinite(az) && az > 0.0f
&& ap >= min_px && ap <= max_px;
bk.bid_px[tid] = bid_ok ? bp : 0.0f;
bk.bid_sz[tid] = bid_ok ? bs : 0.0f;
bk.ask_px[tid] = ask_ok ? ap : 0.0f;
bk.ask_sz[tid] = ask_ok ? az : 0.0f;
}
// Thread 0 handles per-backtest ATR-EMA update against the broadcast mid.
// Top-of-book is validated above so mid is guaranteed finite + positive.
if (tid == 0) {
const float mid = 0.5f * (bid_px_in[0] + ask_px_in[0]);
const float prev = prev_mid[b];
if (prev == 0.0f) {
// First-observation bootstrap (pearl_first_observation_bootstrap).
prev_mid[b] = mid;
// atr_mid_ema stays 0 until event 2.
} else {
const float delta = fabsf(mid - prev);
const float ema = atr_mid_ema[b];
if (ema == 0.0f) {
// First non-zero delta: replace directly.
atr_mid_ema[b] = delta;
} else {
// Wiener-α=0.4 EMA (pearl_wiener_alpha_floor_for_nonstationary).
atr_mid_ema[b] = 0.4f * delta + 0.6f * ema;
}
prev_mid[b] = mid;
}
}
}