Files
foxhunt/crates/ml-backtesting
jgrusewski b44a97ff94 diag(crt): empirical measurement battery for signal × controller × cost analysis
Pure-instrumentation commit. Zero behavior change. The Gate CRT.1 smoke
p9cnk (commit 1e656948b) produced 222k trades and 29688% drawdown,
worse than the failed CRT.A. The structural fixes (multi-horizon
conviction, no-trade band, composite exit) all work as designed — but
the trade cadence is fundamentally too fast for the signal/cost ratio.
Before another round of threshold tuning, measure the structural truth.

Adds device-side counters dumped at smoke close (single end-of-run
memcpy_dtoh batch in read_diagnostics(), NOT per-event):

Group A — per-horizon signal persistence:
  - flip_count[h]:        total direction-sign flips
  - sum_run_length[h]:    cumulative events between flips (u64)
  - run_length_hist[h]:   5-bucket log-scale histogram (1-9, 10-99,
                          100-999, 1k-9.9k, 10k+)
  Output: mean_run_len per horizon = signal half-life proxy

Group B — conviction smoothed-EMA distribution:
  - conv_hist[10]:        10-bucket histogram over [0, 1)
  Output: shape of conviction distribution at decision time

Group C — hold-time distribution:
  - hold_hist[6]:         <1s, 1-10s, 10-60s, 1-10m, 10-60m, >1h
  Output: empirical distribution of trade hold times

Group D — outcome by entry-conviction:
  - outcome_n[10]:        trades per conviction bucket
  - outcome_sum_pnl[10]:  cumulative realized PnL per bucket (price-units)
  - outcome_n_wins[10]:   wins per bucket
  Output: validates "high conviction = better trades" hypothesis OR
          surfaces signal miscalibration

Per-backtest memory: ~330 bytes. Negligible vs existing state.

All counters single-writer-per-block (threadIdx.x == 0 convention; no
atomicAdd per feedback_no_atomicadd). All updates kernel-internal (no
host roundtrip per event). The end-of-run dump uses memcpy_dtoh in
read_diagnostics() — called once at harness termination from the
post-stop_ctrl_counters block, never per-event. Per
feedback_no_htod_htoh_only_mapped_pinned, the hot path is untouched.

N_HORIZONS = 5 (heads.rs canonical {30, 100, 300, 1000, 6000}); the
plan text says 4 but the workspace constant is 5 — used the code value.

Output at end of cluster smoke as a `crt_diag` log block:
  crt_diag h30: flips=X mean_run_len=Y events
  crt_diag h30 run_length_hist: 1-9:N 10-99:N 100-999:N ...
  ...
  crt_diag conv_ema_hist: [0.0-0.1]:N [0.1-0.2]:N ...
  crt_diag hold_time_hist: <1s:N 1-10s:N ...
  crt_diag outcome_by_entry_conv[0.0-0.1]: n=N win_rate=X.X% mean_pnl_pu=Y.YYY
  ...

The output drives the next round of CRT.1 threshold design (delta_floor,
composite exit factors, min-hold cooldown) from data instead of guesses.

Per pearl_adaptive_not_tuned and pearl_controller_anchors_isv_driven:
thresholds will become ISV-derived in CRT.2; CRT.1 must first establish
defensible defaults from the empirical signal characteristics.

Verified: stop_controller (22/22 pass), decision_floor_coldstart (3/3
pass), threshold_and_cost (3/3 pass), parallel_sim_correctness (1/1
pass), lob_sim_integrated_fuzz (3/3 pass). Two lob_sim_fixtures
failures (fix_decision_alpha_buy_close, fix_decision_program_h4_only)
were pre-existing at HEAD 1e656948b — not caused by this commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 21:56:07 +02:00
..