Closes the literal/const drift gap that F.5 introduced. Per
feedback_isv_for_adaptive_bounds + feedback_single_source_of_truth_no_duplicates:
adaptive bounds belong in ISV (or in a single canonical const that
ISV references), never duplicated as literals across modules.
Single canonical source: `crate::rl::common::FRD_HORIZON_TICKS` +
`FRD_BUCKET_RANGE_SIGMA` (already declared in F.5).
Producer-side fixes:
* Trainer ISV bootstrap (integrated.rs): the seed values for slots
500-503 now dereference the canonical consts instead of hardcoded
60.0/300.0/1800.0/3.0 literals. Future tuning of the consts
automatically propagates to both ISV seeds and loader-side
labels — no manual sync required, no drift possible.
* compute_frd_labels (loader.rs): takes `horizon_ticks` and
`range_sigma` as parameters instead of reading consts directly.
Caller (the file-load closure) sources them from the new
MultiHorizonLoaderConfig fields.
Consumer-side fixes — 8 MultiHorizonLoaderConfig literal sites now
provide the two new fields, all defaulting to the canonical consts:
* crates/ml-alpha/src/data/loader.rs (2 internal test-fixture sites)
* crates/ml-alpha/tests/multi_horizon_loader.rs (2 sites)
* crates/ml-alpha/examples/alpha_train.rs (2 sites)
* crates/ml-alpha/examples/alpha_rl_train.rs (2 sites)
* crates/ml-backtesting/src/harness.rs (1 site)
* crates/ml-backtesting/tests/{trainer_parity,ring3_replay}.rs (2 sites)
The "optimal by default" property is preserved: every caller that
doesn't explicitly override gets the spec-recommended 60/300/1800
ticks + ±3σ. Callers that need to retune set the config fields, and
the trainer's ISV slots provide a runtime knob for the same numerics.
Verification (RTX 3050 Ti):
* cargo check -p ml-alpha -p ml-backtesting --examples --tests → clean
* cargo test --lib (6/6 unit tests for FRD label gen + loss_balance) → pass
* frd_head 10/10 + integrated_trainer_smoke 1/1 + trade_mgmt 5/5 → pass
* audit-rust-consts → 0 flags
The two new MultiHorizonLoaderConfig fields are required (no Default
impl) — callers MUST opt in to the FRD label-generation contract by
naming the fields. This is the same discipline applied across other
config consumers; making them Option<...> would silently default to
"no FRD labels" and break F.4's expected supervised signal.
The Phase 1 multi-resolution layout (10 raw + 10 agg@30 + 12 agg@100)
regressed ALL horizons in alpha-perception-htpp6 (2026-05-22):
- auc_h100: 0.681 -> 0.512 (-0.169)
- auc_h300: 0.617 -> 0.506 (-0.111)
- auc_h1000: 0.576 -> 0.526 (-0.050)
Hypothesis falsified. Root cause: Mamba2+CfC SSM encoder already used
all 32 raw ticks effectively via state-recurrence; replacing 22 raw
ticks with arithmetic-mean aggregates destroyed within-window
microstructure variance (the actual h100 signal) AND broke temporal
continuity that the recurrence relies on. Δt Fourier encoder
couldn't compensate.
Architectural pearl: SSM/RNN/CfC + multi-resolution input is
incompatible without separate-encoder-per-scale or explicit scale
tokens. Transformer-style positional encoding tolerates scale-mixing;
recurrent state updates assume consecutive positions.
Reverts default to '1:32'. Adds explicit single_scale_32() constructor
for callers (harness, tests). Keeps default_three_scale() in code with
deprecation note for future sub-variant experiments. Production
defaults across alpha_train CLI, Argo template, dispatcher script,
ml-backtesting harness now match the proven baseline.
Backtest harness + tests use default_three_scale config (1:10,30:10,
100:12 = 1510 ticks of context). Replaces the previous seq_len=32
single-scale path which is now deleted.
Replaces Option<u32> instrument_id_filter with InstrumentFilter enum {All,
Id(u32), FrontMonth}. FrontMonth runs a two-pass detect over the DBN
stream: pass 1 counts instrument_ids and collects SymbolMapping records,
picks the dominant id, validates it resolves to an ES contract via regex
ES[FGHJKMNQUVXZ]\d{1,2}; pass 2 streams the filtered records.
Motivated by alpha-perception-k54wd: a single-id filter on parent-symbol
ES.FUT data caught Q1 2024 (kept=73M) but kept=0 for Q2-Q9 because ES
front-month rolls quarterly (ESH4 -> ESM4 -> ESU4 -> ESZ4 ...). FrontMonth
self-tunes across the rolls without needing a per-file id table.
Sidecar keys distinguish modes: mbp10 / mbp10_instr<id> / mbp10_front_month.
CLI flag renamed --instrument-id -> --instrument-mode {all,id=N,front-month}
with matching parameter rename in argo-alpha-perception.sh + template.
Extends MultiHorizonLoaderConfig with `outcome_label_cost` (default
DEFAULT_OUTCOME_LABEL_COST_ES = 0.5 = 2 ticks for ES round-trip).
LabeledSequence + LoadedFile gain outcome_long + outcome_short arrays
of [Vec<f32>; N_HORIZONS] populated by generate_outcome_labels_d during
LoadedFile construction (same !inference_only branch as BCE labels).
CLI exposure: alpha_train.rs gains --outcome-label-cost flag with the
ES default. All 6 MultiHorizonLoaderConfig consumers updated atomically
per feedback_no_partial_refactor: alpha_train (train + val loaders),
in-file test fixtures (×2), multi_horizon_loader.rs (×2), harness.rs,
trainer_parity.rs, ring3_replay.rs.
cargo check --workspace clean; ml-alpha lib tests 41 passed.
B3+ tasks not yet touched: outcome labels are computed and propagated to
LabeledSequence but no consumer reads them yet (aux trunk Tasks B3-B5
will wire that).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Anti-calibration evidence (smoke ckpts across architecture variants
showed higher reported conviction → MORE negative PnL, -15.6 → -145.2)
traced to a magnitude/direction mismatch in the sizing formula:
conv_ema = EMA(|conviction_signed|) # magnitude smoothing
target_lots = sign(conviction_signed) # INSTANTANEOUS sign
× conv_ema × max_lots # × SMOOTHED magnitude
When per-horizon directions disagree event-to-event, the magnitude EMA
stays high (|x| EMA can't cancel) but the sign flips on noise. Result:
big trades in random direction whenever the 5 horizons disagree.
Replace with a single signed-conviction EMA:
conv_signed_ema = EMA(conviction_signed)
target_lots = sign(conv_signed_ema)
× |conv_signed_ema| × max_lots
Now BOTH sign AND magnitude come from the same smoothed signal. When
directions disagree, signed EMA → 0 collapses size to zero naturally.
Atomic migration across decision_policy.cu (2 kernels), pnl_track.cu
(open-branch reads the SIGNED buffer and takes fabsf for the magnitude-
semantics open_trade_state offsets that composite_exit_check forms ratios
from), and src/sim/mod.rs (renamed device buffers + accessor + 4 launch
sites). No legacy aliases per feedback_no_legacy_aliases; no parallel
buffers per feedback_single_source_of_truth_no_duplicates; α floor 0.4
preserved per pearl_wiener_alpha_floor_for_nonstationary; first-
observation bootstrap preserved per pearl_first_observation_bootstrap.
ml-backtesting lib: 33 passed.
GPU oracle: signed_conviction_ema_collapses_on_sign_flips passes —
observed steady-state shows |signed_ema| ≈ 0.205 (post-fix) producing
|target_lots| = 2 every tail event, vs the pre-fix bug's predicted
|target_lots| = 7 every tail event. Existing
multi_horizon_conviction_cancels_on_disagreement still passes (zero-
cancellation regime is even cleaner under signed EMA).
ml-alpha lib: 33 passed (no regression).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per spec §2.5 and feedback_no_partial_refactor. Removes:
- LoaderMode enum (single-variant after Sequential removal → deleted entirely)
- next_sequence_sequential + sequential_file_idx + sequential_anchor_idx + last_call_was_file_boundary + is_file_boundary
- last_seen_file_boundary, train_graph_boundary_state fields
- notify_file_boundary method + boundary-state graph recapture logic
- need_attn_pool_bootstrap match — always true now (random mode always bootstraps)
- --loader-mode CLI flag + notify_file_boundary() call
- loader-mode Argo template param
Additional consumers migrated atomically (beyond the 4 files listed in
the plan): ml-alpha/tests/perception_overfit.rs,
ml-alpha/tests/multi_horizon_loader.rs, ml-backtesting/src/harness.rs,
ml-backtesting/tests/trainer_parity.rs,
ml-backtesting/tests/ring3_replay.rs — all referenced LoaderMode or the
removed config fields.
Workspace builds clean at this commit (pre-existing cudarc-cupti example
and ml-crate test errors are unrelated to MTER removal — they fail at
HEAD too). New per-horizon CfC arch lands in subsequent tasks of the
same plan.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Loader can now serve sequences in temporal order (Sequential mode);
when active, the trainer skips attn_pool reset at non-file-boundary
sequence boundaries (next commit will use this for stateful CfC + MTER).
Random mode preserved as default; ZERO behavioral change for existing
runs. Phased commit 1/8 of intervention B per
docs/superpowers/specs/2026-05-21-crt-train-intervention-b-multi-timescale-readout.md
Per spec §4.3 + plan C1.4. Safety backup for the primary exit path
(target → 0 from collapsed multi-horizon conviction, C1.2). Catches
the edge case where conviction stays bounded above zero but the trade
is deep in unrealized loss, or short/long horizons disagree for many
events while threshold-gate logic still permits sizing.
Three terms, max-of-three composite:
degradation_term = (1 − conv_ema_now / conviction_at_entry) / 2.0
disagreement_term = disagree_consecutive_events / 32.0
pnl_decay_term = max(−pnl_adjusted_conv_ema, 0) / 1.0
exit_signal = max(deg, dis, pnl_decay)
Fires force-flat (market_targets = (3, 0)) when exit_signal >= 1.0.
State-update flow:
- pnl_track open branch: write conviction_at_entry (offset 20),
conviction_ema (offset 24, initialised to entry value),
pnl_adjusted_conviction_ema (offset 44, = 0),
peak_unrealized_pnl (offset 48, = 0). Takes new conviction_ema_per_b
read-only arg to snapshot at open.
- decision kernel intra-event (when pos open): update offsets
24/44/48/56 (disagreement counter) via update_open_trade_trajectory.
- composite_exit_check device helper: reads offsets 20/24/44/56,
writes (3, 0) on fire. Sibling of stop_check_isv with same return
semantics; called after trajectory update so all four reads see
the freshly-written values.
Fields not read in CRT.1 (offsets 28-43 per-horizon EMA, 52 degradation
counter, 60 horizon_idx_dominant) are NOT written by the open branch —
alloc_zeros covers init. Per feedback_no_hiding: don't populate unused
fields. Also fixes a latent C1.1 bug: pnl_track close branch read
horizon_idx from offset 20 (now conviction_at_entry, f32); moved the
read to offset 60 per the documented layout.
Threshold-bootstrap defaults (will be ISV-derived in CRT.2 alongside
the rest of the controller anchors per pearl_controller_anchors_isv_driven):
degradation_factor = 2.0 (50% conviction decay vs entry → ≤0.5 term)
disagreement_factor = 32.0 (32 events of short/long disagreement)
pnl_decay_factor = 1.0 (pnl_adj_conv structurally ∈ [-1, +1])
Under these defaults disagreement_term is the only term that can fire
on its own — degradation_term caps at 0.5 (conv_ema/conviction_at_entry
≥ 0) and pnl_decay_term caps at 1.0 in the limit. The three terms still
add evidence in superposition; CRT.2 will retune factors against ISV
percentiles. Unit-test coverage validates the disagreement path
(composite_exit_signal_fires_on_disagreement).
Per pearl_no_host_branches_in_captured_graph — all composite logic
lives inside the decision kernel; no host roundtrip in the per-event
hot path. Per feedback_no_htod_htoh_only_mapped_pinned — no new
memcpy or stream.synchronize() in step_pnl_track or
dispatch_latent_market_orders.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The C1.3 commit 7c850a6c0 missed three test files that construct
UniformSimParams literals. cargo check reported E0063 missing-field
errors. Added delta_floor: 0.0 (band disabled, preserves pre-C1.3
behavior in tests) to:
- parallel_sim_correctness.rs
- decision_floor_coldstart.rs
- lob_sim_integrated_fuzz.rs
Per feedback_no_partial_refactor: the atomic refactor must include
every consumer in one logical change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per spec §4.0 + plan C1.3. Without this gate, every fractional change
in target_lots seeds an order via target-delta semantics; combined with
the continuous controller invocation (every event after A1) this
generates hyperactivity even with the multi-horizon §4.4 conviction
formula (C1.2). v2 Gate-1 failures confirmed: scalar EMA + amplification
clamp + per-event controller = 155k trades on a 2M-event smoke.
The band absorbs fractional target adjustments so most events produce
no order. When the signal genuinely shifts and target moves enough to
cross the band, the kernel seeds. Continuous evaluation, sparse action.
Greenfields atomic refactor — single config knob threaded through every
layer in one commit:
SweepBase.delta_floor: f32 (YAML, default 1.0 = 1 lot)
SimVariant.delta_floor: Option<f32> (per-variant override)
ResolvedSimVariant.delta_floor: f32
UniformSimParams.delta_floor: f32
BatchedSimConfig.delta_floor: Vec<f32>
LobSimCuda.delta_floor_d: CudaSlice<f32>
seed_inflight_limits_batched kernel param + skip-if-below-floor logic
New accessor: LobSimCuda::read_inflight_count(b) — counts active != 0
limit slots for backtest b. Not cfg(test); follows existing read_limit_slot
pattern.
Test: no_trade_band_blocks_micro_delta verifies that a second decision
with the same strong-bullish alpha (same target, effective = in-flight
lots) produces delta=0 and does not re-seed. max_lots=1 keeps the
arithmetic unambiguous.
Existing tests: all UniformSimParams struct literals updated with
delta_floor=0.0 (band disabled) so existing behaviour is preserved.
harness.rs from_uniform path uses delta_floor=1.0 (production default).
Hot-path discipline: the delta_floor_d upload happens once per run in
the existing config-upload block (alongside threshold, cost, max_hold_ns);
the kernel reads the device slot per-event without any host roundtrip. No
memcpy_htod / dtoh / dtov / synchronize introduced on the per-event path.
Per pearl_controller_anchors_isv_driven, this floor is currently a
config constant; CRT.2 (Phase 2) makes it ISV-derived from rolling
spread cost / signal volatility.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per spec §4.4 + plan C1.2. Replaces v2 A2 (1d889d2de) + A2.1
(fe2498769) scalar approach that failed Gate 1 catastrophically on
smokes vjmwc and lkrdf (155k trades, 9100% drawdown, Sharpe -15.7).
The structural fix: direction comes from per-horizon SIGNED sum, so
disagreeing horizons cancel. The scalar EMA on max(|p-0.5|) could only
smooth magnitude — it could not smooth direction jitter. Only this
multi-horizon weighted formula can.
Formula (spec §4.4):
weight_h = max(isv[h].net_edge, ε) / (isv[h].var + cost²)
weighted_h = magnitude_h × weight_h × direction_h
conviction_signed = sum(weighted_h) / sum(|weight_h|)
conviction_ema = Wiener-α EMA(|conviction_signed|) (reuses A2 slots)
target_lots = round(sign(conviction_signed) × conviction_ema × max_lots)
Pearl conformance:
- pearl_controller_anchors_isv_driven: weights from ISV state
- pearl_one_unbounded_signal_per_reward: net_edge/(var+cost²) is the
single unbounded factor; magnitude × direction is bounded in [-1,1]
- pearl_zscore_normalization_for_magnitude_asymmetric_signals: divide
by total_abs_weight for scale invariance
- pearl_trade_level_vol_for_stop_distance: cost² is variance bootstrap
- pearl_blend_formulas_must_have_permanent_floor: ε = cost · 0.01 on
net_edge — no cold-start regime switch
- pearl_audit_unboundedness_for_implicit_asymmetry: net_edge is
one-sided positive (losing-edge horizons drop out, not flip sign)
- pearl_wiener_alpha_floor_for_nonstationary: α floor at 0.4 (unchanged
EMA mechanism, repointed at the multi-horizon scalar)
DELETED:
- The `final_size *= (conv_ema/raw_max_conv)` aggregate rescale step
(the v2 bug that amplified weak signals)
- The per-horizon `signed_sizes[h]` Kelly + Sharpe-weight aggregation
loop in decision_policy_default (replaced by direct §4.4 formula)
- Three obsolete tests asserting on the old scalar path:
* conviction_ema_smooths_micro_oscillations
* conviction_ema_does_not_lag_reversals
* conviction_ema_rescale_never_amplifies_weak_signal
* cfg_high_kelly_floor helper (only used by deleted tests)
- cold_start_with_zero_floor_reproduces_old_bug (tested OLD kelly_floor
semantics that v3 doesn't expose)
- cold_start_persistent_bullish_now_closes (was passing under v2 via
target oscillation from integer rounding; v3 produces stable target
on stable signal — closes need price movement)
- alpha_noop_side_2_preserved (anchored on the v2 kelly_frac_floor
integer-truncation path that doesn't exist under v3)
Applied to BOTH decision_policy_default and decision_policy_program
(bytecode VM at OP_WRITE_ORDER). Per feedback_no_partial_refactor —
both consumers migrate atomically. The VM's per-horizon emit + aggregate
opcodes still execute but their stack `final_size` is unused at
OP_WRITE_ORDER (the §4.4 conviction-driven target replaces it). The
stack's attribution mask is still consumed for entry crediting.
The three conviction-EMA device slots (conviction_ema_d, _diff_var,
_sample_var) STAY in LobSimCuda. The Wiener-α EMA mechanism is reused
on the multi-horizon conviction scalar; the device-helper signature
changed from taking alpha_probs[] to taking the precomputed scalar
|conviction_signed|.
Kernel ABI change: decision_policy_default no longer takes
target_annual_vol_units / annualisation_factor / kelly_frac_floor /
sharpe_weight_floor (the §4.4 formula doesn't use them). Removed from
both the CUDA signature and the launch builder in sim/mod.rs. The
bytecode VM (decision_policy_program) still consumes those params for
its OP_EMIT_PER_HORIZON_SIZE / OP_AGG_WEIGHTED_SHARPE opcodes — their
device slots remain allocated.
Test: multi_horizon_conviction_cancels_on_disagreement verifies the
structural fix — bullish [0.7,0.3,0.7,0.3,0.5] horizons cancel to
no-trade output (side=2/3, size=0, conv_ema≈0).
Existing tests rebased: cfgs that used cost_per_lot_per_side=0.0 now
use 1.0 (the §4.4 formula's eps_edge floor requires cost > 0). Each
affected test file documents the rebase inline. Tests that were
anchored on observed v2-kernel values (per
pearl_tests_must_prove_not_lock_observations) are deleted; tests with
genuine invariants (boundedness, direction-sanity, stop-controller
behavior) are preserved.
Hot-path discipline: no memcpy_htod/dtoh/dtov/synchronize introduced.
Only kernel-internal computation; one launch arg removed from the
default decision kernel.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per spec §7 (v3 architecture reset, promoted from v2 Phase B into CRT.1).
Single-cache-line layout. Every reader and writer of open_trade_state
migrates in this commit per feedback_no_partial_refactor.
New 64-byte layout (existing 24-byte fields preserved at original offsets):
Offset Size Field
0 8 entry_ts_ns (u64)
8 4 entry_px_x100 (i32)
12 4 entry_size (i32)
16 4 realized_at_open (f32)
20 4 conviction_at_entry (f32) ← NEW
24 4 conviction_ema (f32) ← NEW
28 16 conviction_per_horizon_ema [4 × f32] ← NEW
44 4 pnl_adjusted_conviction_ema (f32) ← NEW
48 4 peak_unrealized_pnl (f32) ← NEW
52 4 degradation_consecutive_events (u32) ← NEW
56 4 disagreement_consecutive_events (u32) ← NEW
60 1 horizon_idx_dominant_at_entry (u8) ← NEW
61 3 pad
The 24-byte prefix is unchanged so downstream readers (stop_check_isv in
decision_policy.cu, max_hold event-rate check in resting_orders.cu) need
only update the stride constant. The new fields at offsets 20..63 are
populated by subsequent CRT.1 tasks (C1.2 multi-horizon conviction, C1.4
composite exit signal).
Open branch in pnl_track.cu zeros the new fields implicitly because
alloc_zeros on the device slot zero-initialises everything; subsequent
writes only touch the 0-23 byte range (existing fields). Close branch
reset-loop iterates OPEN_TRADE_STATE_BYTES which now zeros all 64.
Files changed:
crates/ml-backtesting/src/lob/mod.rs — pub const OPEN_TRADE_STATE_BYTES: usize = 64
crates/ml-backtesting/cuda/pnl_track.cu — #define OPEN_TRADE_STATE_BYTES 64
crates/ml-backtesting/cuda/resting_orders.cu — comment + stride literal 24→64
crates/ml-backtesting/tests/stop_controller.rs — open_trade_state_64_byte_layout test
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Gate 1 catastrophic failure (smoke vjmwc, commit 3d8f12deb):
- 62× hyperactivity (157,470 trades vs 2,511 baseline)
- 9,347% max-drawdown ($327M loss on $3.5M base)
- Sharpe -15.63 (2.4× worse than baseline)
Root cause: A2's formula
final_size *= (conv_ema / raw_max_conv)
amplified weak signals because EMA tracks the MEAN of raw_max_conv, NOT a
running max as the author's comment claimed. When raw_max_conv < conv_ema
(normal during quiet periods between strong signals), the multiplier was
> 1, blowing up small-signal positions:
raw=0.05, ema=0.3 → 6× amplification
raw=0.01, ema=0.3 → 30× amplification
Fix: clamp scale ∈ [0, 1] in BOTH decision_policy_default and
decision_policy_program (OP_WRITE_ORDER). Only damp when raw is stronger
than EMA (scale < 1); never amplify when raw is weaker (scale capped at 1).
Math:
scale = conv_ema / raw_max_conv // can be 0..∞
scale_clamped = min(scale, 1.0) // can only be 0..1
final_size *= scale_clamped
Test: conviction_ema_rescale_never_amplifies_weak_signal validates
target_lots stays ≤ 1 when alpha=0.51 (raw_conv=0.02) after EMA warmup
at alpha=0.7 (ema~=0.4). Without the clamp the broken multiplier is 20×.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
After A1 (commit 045850e8f) the controller fires every event. Without
smoothing, target lots would oscillate as alpha probabilities jitter
event-to-event, generating hyperactive spread-bleed from back-and-forth
trades.
Ships per spec §4.2 and §3.7: Wiener-α adaptive EMA on max-conviction-
across-horizons. Formula:
α_raw = diff_var / (diff_var + sample_var + ε)
α_active = max(α_raw, 0.4) per pearl_wiener_alpha_floor_for_nonstationary
ema = α_active × new + (1 − α_active) × prev
First-observation bootstrap: prev_ema == 0 → replace directly per
pearl_first_observation_bootstrap.
Three new device slots (per-backtest):
- conviction_ema_d: smoothed conviction (used for final-aggregate rescale)
- conviction_diff_var_ema_d: second-order EMA, drives adaptive α
- conviction_sample_var_ema_d: second-order EMA, drives adaptive α
Architectural choice: rescale at the final aggregate (final_size *= conv_ema /
raw_max_conv), not at per-horizon sig_mag. Per-horizon sizing keeps raw
|alpha-0.5|*2 so horizons with no signal (alpha=0.5) contribute zero —
the spec §4.2 "smoothed conviction" is a unified gain over the aggregate,
not a substitute for per-horizon signal strength. At bootstrap the scale
is exactly 1.0 (raw_max_conv == conv_ema) so the very first decision is
bit-identical to the pre-A2 kernel. Direction recovery (sign of
alpha-0.5) remains per-horizon — genuine reversals respond at event rate.
Threaded through both decision_policy_default and decision_policy_program
kernels' signatures and launches in sim/mod.rs.
Full multi-horizon conviction *aggregation* (spec §4.4) remains Phase B;
A2 ships ONLY the smoothing operator on the existing scalar conviction.
Pure on-device: no memcpy_htod / dtoh / dtov / synchronize in hot path.
Single test-only DtoH accessor (read_conviction_ema) for unit-test
inspection of the smoothed value.
Tests:
- conviction_ema_smooths_micro_oscillations — sentinel→bootstrap→bounded
EMA across 10 alternating high/low all-bullish alpha drives; direction
stable.
- conviction_ema_does_not_lag_reversals — sign flip in alpha → target
side flips next event.
- All 22 stop_controller + 5 decision_floor_coldstart + 3 threshold_and_cost
+ parallel_sim + ring3_replay + trainer_parity + 6 fuzz tests pass.
- 2 lob_sim_fixtures tests (fix_decision_alpha_buy_close, fix_decision_program_h4_only)
were already failing on baseline 045850e8f pre-A2; not a regression from
this commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per spec 2026-05-20-continuous-reasoning-trader-design.md §3.3 and §8:
decision_stride is REMOVED, not deprecated. No backwards-compat shim,
no fallback. Every consumer migrates in this commit per
feedback_no_partial_refactor.
Removed from:
- bin/fxt-backtest: RunArgs CLI flag, SweepBase field, SweepCell
override field, default_decision_stride() function, all three
BacktestHarnessConfig and RunArgs construction sites
- crates/ml-backtesting/src/harness.rs: BacktestHarnessConfig field,
MultiHorizonLoaderConfig decision_stride initializer, `let stride`
local, `if event_count % stride == 0` gate around
step_decision_with_latency; forward_step_into + step_decision now
share a single window-full guard (merged into one `if` block)
- crates/ml-alpha/src/data/loader.rs: MultiHorizonLoaderConfig field,
next_sequence stride logic simplified to stride=1 (consecutive
snapshots only)
- crates/ml-alpha/src/trainer/perception.rs: PerceptionTrainerConfig
field and Default impl; all four dt_s locals replaced with 1.0_f32
(training K-loop, graph-capture K-loop, forward_step_into CfC step,
eval K-loop)
- crates/ml-alpha/examples/alpha_train.rs: CLI flag, trainer_cfg and
both loader configs
- crates/ml/examples/alpha_baseline.rs: CLI flag, train + eval stride
gates replaced with unconditional read_all()
- config/ml/*.yaml: decision_stride: lines removed from
sweep_smoke, sweep_threshold_tuning, sweep_deployability,
sweep_decision_stride_example (file repurposed as generic example)
- tests: forward_step_golden, perception_overfit (×7 structs including
the stride=4 smoke repurposed as a second convergence check),
multi_horizon_loader (stride=4 spacing test repurposed as
ts_ns monotonicity check), ring3_replay, trainer_parity
Harness loop now invokes BOTH forward_step_into AND
step_decision_with_latency on every event whenever the snapshot window
is full. forward_step_into advances SSM state and writes alpha_probs_d;
step_decision_with_latency reads alpha_probs_d immediately after —
no CPU roundtrip, no stride gate.
n_decisions ≈ events_processed - seq_len + 1 after this commit
(vs ~9999 at stride=200 in the S2 baseline).
cargo check --workspace: clean
cargo test -p ml-backtesting --lib: 33 passed
cargo test -p ml-alpha --lib: 33 passed (6 ignored)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
S2.1 instrumentation (smoke cqpph @ 95a77c4ac) revealed the chain
worked end-to-end: max_hold check fired 1661 times, force-flat target
was written and seen by seed_inflight. Yet 86.5% of trades exceeded
the 60s cap with mean hold = 432s. Root cause: decision_policy's
stop_check_isv only runs every decision_stride events. At
decision_stride=200 on ES Q1 (2M events / ~91 days = ~3.9s sim-time
per event), decisions are ~13 min apart in sim-time, so the cap is
enforced 13 min late on average.
Fix: move the max_hold check to resting_orders_step (event-rate, runs
every iteration), same pattern as the session-gap force-close at
resting_orders.cu:282. Thread max_hold_ns_per_b and open_trade_state
into resting_orders_step. SL/trail stay in stop_check_isv because
they depend on ISV controller state at decision-rate.
Remove the dead decision-rate max_hold code and its 6 diagnostic
counter params from stop_check_isv per single-source-of-truth and
no-hiding rules. Keep mh_kernel_calls and mh_force_flat_seen_by_seed
counters as ongoing generic diagnostics. Update harness.rs log line
and stop_controller tests to exercise the event-rate path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
S2 cluster smoke ppcfk (29f7e923c) showed mean trade hold = 432s with
max_hold_ns=60s — 886/1024 trades exceed the cap that should hard-flat
them. Upload chain and pnl_track entry_ts write look structurally
correct on inspection. Need per-step counters to localize WHICH step
of the chain fails.
Adds 7 counters in stop_check_isv (kernel_calls, seen_zero, entry_ts_zero,
ts_underflow, elapsed_below_cap, would_fire, force_flat_written) plus 1
counter in seed_inflight_limits_batched (seed_saw_force_flat). Logged at
smoke end as a third nan_counters-like line.
Diagnostic only — no behavioral change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v1 instrumentation (S1.20) eliminated 3 of 6 hypotheses: apply_fill_to_pos
arithmetic is fully clean (avg_px=0 realised=0 realized_pnl=0). But
pnl_track open branch still saw vwap_entry=0 (194 times) or > 21M (216
times) at the open transition.
v2 adds per-vwap-write-site counters so we can pinpoint which apply_fill
branch produced the bad vwap, plus captures the actual last-bad-vwap
value and the path id (1..6 = open-flat, scale-in zero/huge, flip-beyond
zero/huge, session-gap-saw-stale).
The 7th counter (vwap_session_gap_was_bad) fires when the session-gap
force-close path sees pos.vwap_entry already in a corrupt state pre-gap.
Logged at end of smoke as `nan_counters_v2: zero_flat=N huge_flat=N
zero_scale=N huge_scale=N zero_flip=N huge_flip=N session_gap_was_bad=N
| b0_last_bad_vwap=X b0_last_bad_path=N`.
19/19 stop_controller CUDA tests pass; 5/5 decision_floor_coldstart pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Audit (fxt-data-audit on ES.FUT_2024-Q1) revealed real source-data
outliers that the hardcoded < 1e8 threshold didn't catch:
- bid_min = -$4.85 (negative bid)
- bid_p1 = $64.15 (1% of bids in sub-$100 range, far below ES)
- ask_max = $53,012 (10x above any plausible ES price)
The < 1e8 threshold = $100M was useless: never triggered on real ES.
Fix: parameterize the range. min_reasonable_px / max_reasonable_px as
per-backtest fields in UniformSimParams, ResolvedSimVariant, and
BatchedSimConfig (defaults 0.0 / f32::INFINITY = no-check, preserves
existing test fixtures). Sweep_smoke.yaml sets 1000/20000 for ES
futures — catches all observed outliers without rejecting any plausible
price. BacktestHarness calls upload_price_range() once after creating
the sim so the bounds are active before the first apply_snapshot.
CUDA kernel book_update_apply_snapshot gains two new args
(min_reasonable_px[n_backtests], max_reasonable_px[n_backtests]) that
replace the hardcoded > 0.0f && < 1.0e8f checks at both the top-of-book
gate and the per-level sanitization pass.
Test price_range_rejection_skips_snapshot validates: sub-$1000
snapshot, super-$20000 snapshot, and negative bid all skip with
snapshots_skipped counter increment; valid ES snapshot passes through.
17/17 stop_controller tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three orthogonal fixes for residual issues in smoke stx9p (97 zero
sentinels, 85 i32::MAX sentinels, 840/1024 over-60s holds):
Fix 1 — zero-sentinel residue (px > 0):
- Per-level sanitization in apply_snapshot_kernel only checked sz > 0.
- A book level with px=0, sz>0 passed → walk_* computed total_cost=0
→ avg_px=0 → vwap_entry=0. Trade record reported entry_px=0.
- Add px > 0.0f to bid_ok/ask_ok conditions.
Fix 2 — i32::MAX upper bound (px < 1e8):
- Post-Bug-D (1e9 nanoprice scaling) some boundary events carried prices
larger than any plausible instrument. Saturating cast to i32 produced
the 21474836 sentinel even when isfinite() passed.
- Add px < 1.0e8f upper bound to per-level AND top-of-book validation.
Fix 3 — session-gap force-close:
- max_hold check fires at decision_stride frequency, but during weekend
halts no events advance current_ts → max_hold never fires until next
session. Result: 49h holds in stx9p (840/1024 over 60s threshold).
- Detect ts gap > 1 hour in resting_orders_step. If position is open,
zero position_lots directly. pnl_track_step's existing close branch
emits the TradeRecord on the next call. No synthetic P&L added —
records show realised_pnl from whatever was accumulated before the gap
(honest: cannot fill across a halt).
- New per-backtest last_event_ts_d slot tracks the previous event ts.
- Test session_gap_force_closes_open_positions: 2-hour ts jump after
open verifies force-flat fires and exactly 1 TradeRecord is emitted.
All 16 stop_controller tests pass. All 5 decision_floor_coldstart pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two root-cause fixes surfaced by cluster smoke v74v4 (sweep_smoke-a2dfc6d99):
Bug D — DBN parser price scaling:
- BidAskPair::price_to_f64/price_from_f64 used /1e12 / *1e12 from test-data
calibration. DBN production uses 1e-9 nanoprice (the DBN standard). ES at
5500 raw 5_500_000_000_000 → 5.5 instead of 5500. Smoke trade records
showed entry_px=5.24 instead of expected ~5240 ES index points (1000×
too small). Fix: 1e12 → 1e9 in both functions. Round-trip symmetric;
tests updated.
Bug C-b — corrupt top-of-book sentinel:
- Per-level sanitization (Task 15) zeros each unhealthy MBP-10 level
individually. At session-boundary events with all 10 levels invalid,
the book becomes uniformly zero. apply_fill_to_pos then reads
bid_px[0]=0 / ask_px[0]=0 → vwap_entry=0 → trade record entry_px=0
(zero sentinel in v74v4 CSV, 162/1024 trades in n59t4).
- Fix: pre-validate top-of-book in apply_snapshot_kernel. If
bid_px[0]/ask_px[0]/bid_sz[0]/ask_sz[0] are non-finite or
bid_px[0]<=0/ask_px[0]<=0/bid_sz[0]<=0/ask_sz[0]<=0, atomically skip
the entire snapshot (book/prev_mid/atr_mid_ema unchanged). Add
per-backtest snapshots_skipped_d counter for observability.
Test corrupt_top_of_book_skips_snapshot_and_increments_counter validates
NaN top-price + zero top-size cases both increment the counter without
mutating state, and that a subsequent valid snapshot updates normally.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two orthogonal bugs that compounded to produce the exit_px=±21474836
sentinel in cluster smokes (315 trades baseline, 500 trades pearl):
Bug A — NaN/Inf book propagation:
- apply_snapshot_kernel passes through NaN/Inf prices from MBP-10
predecoded data (session boundaries, gap-fills).
- walk_ask_for_buy / walk_bid_for_sell accumulate cost += take * NaN.
- apply_fill_to_pos: avg_px = NaN; realized_pnl += NaN → permanent NaN.
- Subsequent close: (int)(NaN-derived * 5000) → i32::MAX sentinel.
- Fix: zero-out non-finite levels in apply_snapshot_kernel; add
isfinite() guards in walk helpers as defense-in-depth; ATR update
guards against non-finite mid.
Bug B — pnl_track close branch doesn't reset scratch:
- Scratch reset (full 24-byte memset to 0) was already present in the
current code; no source change required for Bug B.
Tests:
- book_nan_inf_prices_dont_corrupt_realized_pnl: NaN at ask[3] + Inf
at bid[5] survives the fill pipeline without making realized_pnl
non-finite.
- pnl_track_resets_scratch_on_close: open+close+5 flat events emits
exactly 1 record; second open+close emits exactly 1 more (=2 total).
14/14 stop_controller tests pass; all other ml-backtesting test suites
unaffected.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three follow-ups from cluster smoke gp74n (trade_vol pearl validation):
1. Max-hold force-close: max_hold_ns added as per-backtest config
(default 0 = disabled). Fires force-flat (3, 0) when current_ts -
entry_ts >= max_hold_ns, BEFORE SL/trail check. Tested via
max_hold_forces_close. Sweep YAML sets 60s cap to bound the long
tail observed in gp74n (263985s pathological hold).
2. exit_px defensive sanity check: 500/1024 gp74n trades reported
exit_px = ±i32::MAX/100 (float→int saturation sentinel from likely
NaN segment_realized). Defensive fix in pnl_track_step: if exit_px
is non-finite or diverges from entry_px by >10%, fall back to
entry_px with zero realised_pnl. Root cause to be traced separately.
3. Spec §9.2 criteria revision: mean_hold<30s replaced with p95<600s
+ max<=max_hold_ns. The 30s threshold reflected the pre-pearl
sub-cost churning bug, not a real feature criterion. p95 catches
long-tail pathology while letting alpha-driven exit timing breathe.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the Task 12 `2.0f * cost` floor (hardcoded multiplier) with
trade_vol = sqrt(realised_return_var) bootstrapped from cost². Per
pearl_trade_level_vol_for_stop_distance.md: microstructure ATR is the
wrong time scale for trade-level stop decisions; per-horizon
realised_return_var is the right one, with cost² as a structural cold-
start sentinel.
cost now appears exactly once — inside the sqrt as a bootstrap sentinel,
never as a distance multiplier. The 2.0f literal is eliminated;
controller is fully ISV-driven.
var_avg accumulates realised_return_var in the same single-pass horizon
loop as ema_loss/ema_win. Cold-start (var_avg=0): trade_vol = cost.
Post-bootstrap: sqrt(var_avg) dominates.
Test retargeted: cost_floor_prevents_sub_cost_stops →
trade_vol_floor_prevents_sub_cost_stops, with boundaries straddling
trade_vol=cost=0.125 instead of the prior 2*cost=0.25 (no-fire Δ=0.08,
fire Δ=0.20).
Spec §5, §10, §11, §12 amended.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cluster smoke w7b4p showed 44864 closed trades in 1M events at 200ms
latency — physically impossible without sub-cost churning. Root cause:
sl_distance = max(pnl_ema_loss, atr) ≈ 0.05 at cold-start, but round-trip
cost = 2 × 0.125 = 0.25. Every stop-out guaranteed loss > 5× distance;
EMA converges sub-cost.
Amendment: triple-max sl_distance = max(pnl_ema_loss, atr, 2*cost);
trail_distance = max(pnl_ema_win, atr, 2*cost). cost is an ISV-discipline
strategy anchor (already per-backtest from P4 — no new state slot, no
tuned constants). Added cost_per_lot_per_side_per_b param to both decision
kernel signatures and threaded cost_per_lot_per_side_d through both launch
sites in step_decision_with_latency.
Test: cost_floor_prevents_sub_cost_stops validates unrealized in (ATR, 0.25)
does NOT fire SL; unrealized > 0.25 DOES fire. Spec amended (§10, §12).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- StopRules struct + sl_tp_rules field + all literals deleted; the ISV
stop controller (Tasks 2-9) replaces this dead data path.
- use_cold_start_stopgap propagation deleted across harness,
batched_config, fxt-backtest, sweep_smoke.yaml, and 5 test files.
- Q1 stopgap branch in harness.rs deleted; replaced with the original
simple strategy-upload loop.
- decision_floor_coldstart test retargeted: cold_start_persistent_
bullish_with_default_stops_never_closes -> ..._now_closes, asserts
trades > 0 AND |pos| <= max_lots (99 trades, pos=1 on local GPU).
- CBSW spec + plan prefixed with SUPERSEDED headers.
Per feedback_single_source_of_truth_no_duplicates +
feedback_no_partial_refactor: contract change migrates every consumer
in one commit. No legacy wrappers; no version suffixes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two-instruction addition to the existing close-emission branch
(prev != 0 && now == 0). Without the reset, the next entry inherits
a stale HWM that could arm the trail spuriously on event 0 of the
new trade.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
market_targets[b] now interpreted as target absolute position (side=0
long, side=1 short, side=2 no-op preserved, side=3 force-flat).
seed_inflight_limits_batched computes order_lots = target_signed -
effective_position where effective_position sums pos.position_lots
plus all unfilled signed slot sizes (active in {1, 2}). Fixes both
the additive accumulation bug AND the worse latency-bypass variant
(naive delta would have made things 200x worse under 200ms latency).
3 new tests added (all passing):
- position_target_not_additive: latency=0, 10 repeated target events stay <= max_lots=5
- position_target_not_additive_with_latency: 200ms latency, 500 events, in-flight summation prevents runaway
- alpha_noop_side_2_preserved: side=2 events leave filled position unchanged
All 9 stop_controller tests pass; parallel_sim and fuzz regression clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Both decision kernels now call the same __device__ helper at the top
of per-backtest dispatch (after the program_lens early-return). Single
source of truth for stop logic across default and bytecode-VM paths.
Parity test ensures they produce identical market_target output for
matched scenarios.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Boundary test that fails on bit-pick-first or bit-pick-max regressions
of the open_horizon_masks averaging in stop_check_isv. Per-horizon
pnl_ema_loss values (2, 4, 3, 3, 3) yield mean=3.0 across the 0x1F
mask; snapshots placed relative to vwap_entry for ask-spread safety.
Tests Δ_entry=2.5 (no-fire) and Δ_entry=4.5 (fire).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
trail_distance = max(pnl_ema_win, atr_mid_ema). HWM ratchets up via
fmaxf each event while position open; trail fires when HWM clears
the arming threshold AND unrealized drops by trail_distance from
HWM. Same force-flat (3, 0) write as SL.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Skeleton helper called from decision_policy_default at top of
per-backtest dispatch. Returns 0 (no-op) on flat positions; trigger
logic for open positions added in Tasks 4-6. Single source of truth
for stop logic across decision_policy_{default,program}.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per-event Wiener-α=0.4 EMA on |Δmid| with first-observation bootstrap.
Floor source for the SL/trail-distance controller (spec §6). Thread 0
of the broadcast-snapshot kernel handles the update per backtest;
threads 1..9 still handle the 10 level writes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The threshold-tuning smoke at 81decf40f produced n_trades=0 despite
74.6% of decisions having max_conv ≥ 0.30 — the linear-weighted-mean
aggregator in decision_policy_default is structurally dilution-bound
at cold-start (per spec §1).
Q1 stopgap: when sim_variants[i].use_cold_start_stopgap = true, the
harness uploads a max-confidence Strategy bytecode program for that
backtest, routing decisions through decision_policy_program with
OP_AGG_MAX_CONFIDENCE. Existing kernel; zero CUDA changes.
Field additions (atomically across BatchedSimConfig + UniformSimParams
+ ResolvedSimVariant + SweepBase.SimVariant) — every UniformSimParams
literal migrated to include use_cold_start_stopgap: false (default).
The sweep YAML's sim_variants entry sets it to true only for the
validation run; production deployability uses Q2's kernel fix instead.
Sweep YAML (config/ml/sweep_smoke.yaml) flipped to use_cold_start_stopgap=true
at threshold=0.0, cost=0.125 — same anchor as the threshold-tuning
smoke that produced n_trades=0, for direct comparison.
This is a VALIDATION step. Cluster smoke at this commit MUST produce
n_trades > 100 + finite metrics. Q2's kernel CBSW immediately follows
and deletes this entire stopgap atomically (field, harness branch,
YAML setting, every literal).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the two sweep axes that the spec's deployability grid needs but
were missing from the kernels:
Threshold gate (decision_policy.cu, both kernels):
- New per-backtest `threshold_per_b` array kernel arg.
- Pre-Kelly prelude: if max_h |alpha[h] - 0.5| * 2 < threshold[b],
emit noop and return. Kept deterministic from alpha alone so the
threshold pre-registration step (p60-p95 absolute calibration on a
validation window, future P6) reflects exactly what gets gated in
deployment.
Per-fill cost integration (resting_orders.cu / apply_fill_to_pos):
- apply_fill_to_pos signature grows three args: b, cost_per_lot_per_side_per_b,
total_fees_per_b. Single insertion point at line 90.
- After the close-leg realized_pnl math runs (so the gross unwind P&L
is preserved), deduct fill_cost = filled_lots * cost_per_lot_per_side[b]
from pos.realized_pnl AND accumulate into total_fees_per_b[b].
- Net-of-cost semantics: isv_kelly_update_on_close reads realized_pnl
delta which is now net of cost — Kelly state learns from realistic
return distribution.
- All 3 apply_fill_to_pos call sites in step_resting_orders updated.
order_match.cu's submit_market_immediate path is dead code in the
post-P1 flow (everything routes through seed_inflight_limits_batched
→ step_resting_orders → apply_fill_to_pos) so not touched here.
BatchedSimConfig + UniformSimParams + BacktestHarnessConfig gain
threshold + cost_per_lot_per_side fields. All UniformSimParams
constructors in tests and main.rs updated with defaults (0.0, 0.0 =
gate disabled, frictionless).
Regression:
- threshold_gate_skips_low_conviction (p=0.51 + threshold=0.10 → noop)
- threshold_gate_allows_high_conviction (p=0.8 + threshold=0.10 → buy 1+)
- threshold_zero_is_passthrough (sanity)
- All P1+P2+P3 tests continue to pass via the new ABI.
cost_deducted_at_each_fill + kelly_state_sees_net_return end-to-end
tests deferred — they require a full submit_market → fill → close
sequence, which the production smoke exercises.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Migrates target_annual_vol_units, annualisation_factor, max_lots,
latency_ns, kelly_frac_floor, sharpe_weight_floor from scalar-broadcast
kernel args to per-backtest device arrays via BatchedSimConfig. Atomic
contract change per feedback_no_partial_refactor — kernel + sim + harness
+ all 3 existing test files migrate in this commit.
- crates/ml-backtesting/src/sim.rs → sim/mod.rs (directory module)
- crates/ml-backtesting/src/sim/batched_config.rs (NEW): BatchedSimConfig
+ UniformSimParams + validate(). from_uniform rebuilds the legacy
uniform-broadcast behaviour at n=1 (smoke/fixtures). from_grid lands
in P6 for the 140-variant sweep packing.
- LobSimCuda gains 6 per-backtest device buffers (target_annual_vol_units_d,
annualisation_factor_d, max_lots_d, latency_ns_d, kelly_frac_floor_d,
sharpe_weight_floor_d). step_decision_with_latency uploads from
BatchedSimConfig each call; both decision kernel launches now pass
per-backtest array pointers.
- decision_policy_default + decision_policy_program: scalar args become
const float* / const int* per_b arrays; first lines of each kernel
index by `b` into the arrays. Behaviour preserved at n=1 uniform.
- dispatch_latent_market_orders: reads latency per-backtest from
&BatchedSimConfig (host loop stays for P1; P2 replaces with kernel).
- step_decision_with_latency now ALWAYS dispatches through the latency
path; when cfg.latency_ns[b]=0 the in-flight slot's arrival_ts equals
current_ts and gets promoted immediately on next snapshot. Eliminates
the if/else branch and consolidates the launch path.
- harness.rs: BacktestHarness gains a sim_config field, built via
BatchedSimConfig::from_uniform at new() from the harness cfg's scalar
fields. The run loop passes &self.sim_config to step_decision_with_latency.
Regression coverage:
- parallel_sim_correctness::parallel_sim_equivalence_with_uniform_config
— n=8 with uniform config produces 8 bit-identical market_targets
(proves per-backtest indexing reduces correctly).
- Existing decision_floor_coldstart tests (3) all pass through the new
ABI — proves cold-start floor + variance-cap gate behaviour preserved.
- parallel_sim_independence_per_backtest deferred to P2 (needs
read_first_inflight_arrival_ts helper that depends on LimitSlot layout).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three follow-ups to the cold-start floor fix:
1. Kernel: MIN_TRADES_FOR_VAR_CAP gate. After the first trade closed,
`isv_kelly_update_on_close` set `realised_return_var = ret²` — a
single-sample variance proxy that systematically collapses
`cap_units = target_vol / sqrt(var × ann_factor)` near zero for
any biased return. cap_lots → 0 → no further trades despite strong
alpha. Gate the variance-derived cap behind `n_trades_seen >= 10`;
below the threshold cap falls back to host-supplied `max_lots`,
same as the pre-first-trade path. Same gate applied to both
`decision_policy_default` and `decision_policy_program`.
Regression: `post_first_loss_state_does_not_lock_out_further_trades`
reproduces the exact pre-fix state from the smoke (n_trades_seen=1,
var=103.6) and asserts the kernel still fires a long with p=0.8.
2. Aggregate: add `nvidia.com/gpu: 1` resource request. Scaleway's
L40S device plugin mounts libcuda.so.1 into the container only on
GPU-requesting pods; the aggregate logic is CPU-only but the
binary's dynamic loader needs the driver libs. Cheapest correct
fix until a separate CPU-only aggregator binary exists.
3. Smoke YAML: `max_events: 0` (exhaust loader). 100k events is
minutes of ES.FUT, far shorter than the h6000 holding horizon the
model was trained on. Full quarter exercises sustained trading +
variance estimate ramp-up.
All three regression tests pass locally on RTX 3050 Ti.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Production smoke completed end-to-end but produced n_trades=0 across 99,969
decisions — `decision_policy_default` and `decision_policy_program` both
applied a sentinel-skip pattern: if `isv_kelly_d` had not been seeded
(pnl_ema_win == 0), each horizon's signed-size stayed zero, AND each
horizon's aggregation weight (= recent_sharpe) also stayed zero. The
cross-horizon w_sum was therefore 0, final_size was 0, every market_target
was noop. State only updates on trade close → no trade ever fires →
infinite cold-start.
Per pearl_blend_formulas_must_have_permanent_floor (`max(real, floor)`,
not blend) and pearl_kelly_cap_signal_driven_floors, replace the sentinel-
skip with a two-layer floor on each kernel:
1. Kelly fraction: `max(kelly_frac_floor, computed_kelly)` — when state
is sentinel, falls back to the floor directly. Cap_lots falls back
to `max_lots` when realised_return_var is sentinel.
2. Aggregation weight: `max(sharpe_weight_floor, recent_sharpe)` — lets
cross-horizon sum produce a non-zero size before recent_sharpe is
populated. Once a horizon shows positive sharpe it dominates.
Plumbed through `step_decision_with_latency` / `step_decision` as two
new f32 args (atomic contract change, every caller migrated). Defaults
0.20 / 0.10 chosen so a strong-conviction signal (sig_mag ≥ 0.5) fires
1 lot at cold-start under max_lots=5 while weaker signals stay flat
(see `default_kelly_frac_floor` comment for the arithmetic). Exposed
as CLI flags + sweep-grid base/cell overrides.
Regression test `decision_floor_coldstart` proves:
- default floors (0.20/0.10) fire a 1-lot buy with p_h=0.8 and zero state
- zero floors reproduce the original noop bug
Also moves `aggregate` step to the GPU pool because fxt-backtest is
dynamically linked against libcuda.so.1 (the ci-compile-cpu hosts
don't expose CUDA driver libs).
Verified locally on RTX 3050 Ti — workspace cargo check passes, both
regression tests pass, trunk save/load roundtrip still passes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per project_ml_alpha_starting_capital greenfield posture: there is no V1
to differentiate from (the V1 trunk forward was dead code, no V1
checkpoint files exist in the wild). The 'v2' prefix on every identifier
was historical baggage from the migration period.
Renames:
- CheckpointV2 -> Checkpoint (also drops the version: u32 field —
bincode either deserialises a current envelope or errors; no migration
path needed)
- CheckpointVersionProbe removed (was only for V1 rejection)
- LAYER_NORM_CUBIN_V2 / VARIABLE_SELECTION_CUBIN_V2 / ATTENTION_POOL_CUBIN_V2
-> LAYER_NORM_CUBIN / VARIABLE_SELECTION_CUBIN / ATTENTION_POOL_CUBIN
- _ln_module_v2 / _vsn_module_v2 / _attn_module_v2 -> drop _v2 suffix
- smoke_load_v2_checkpoint test -> smoke_load_checkpoint
- config/ml/sweep_v2_*.yaml -> config/ml/sweep_*.yaml
- migration-era 'V2 weight skeleton' / 'V2 fields' / etc. comments
cleaned to remove the v2 prefix
Pre-existing 'v2' references in ml-backtesting CUDA files
(decision_policy.cu, pnl_track.cu) are NOT touched — those refer to
future planned 'v2' refinements (Portfolio mode, multi-fill averaging)
from the C1-C19 commits and reflect aspirational features unrelated to
this session's trunk-grows work.
Verification: ml-alpha + ml-backtesting + fxt-backtest all build clean.
perception_forward_golden bit-exact (max_diff = 0.000000).
Adds #[ignore]d integration test that loads a CheckpointV2 file produced
by alpha_train and verifies CfcTrunk::load_checkpoint accepts it.
Activated via FOXHUNT_SMOKE_CKPT env var on a CUDA-capable host.
Also adds a compile-time witness test confirming Summary.max_drawdown_pct
field exists post-X16 (required by emit_deployability_verdict).
Note: full BacktestHarness end-to-end smoke (running one cell against
fixture MBP-10) requires the v2 forward path to live on the trunk
(X11 — deferred). This commit verifies the producer/consumer wire-up.
Per spec §3.4, §4.1 (X18).
Existing lob_sim_fuzz only exercised book_update + market orders.
This new test suite drives the FULL integrated pipeline under random
adversarial conditions:
apply_snapshot (random-walk book)
→ step_resting_orders (random signed trade-flow signal)
→ broadcast_alpha (random per-horizon probs every 4th event)
→ step_decision_with_latency (mixed latency=0 + latency=50ms cells)
→ submit_market_immediate (immediate path)
→ pnl_track_step + isv_kelly_update_on_close
Warm-starts every backtest with random-but-plausible Kelly state
(at least h4 set credibly profitable so decisions actually open
positions). Random target_annual_vol + annualisation_factor + max_lots
per decision to vary the Kelly cap.
Per-50-event invariants:
• book monotonicity (bid_px[k] ≤ bid_px[k-1], ask_px[k] ≥ ask_px[k-1])
• no-crossed-book (ask[0] > bid[0])
• Pos.realized_pnl + Pos.vwap_entry finite (no NaN/Inf leaks)
• Pos.position_lots in plausible range (|≤100|)
• All 5 per-horizon IsvKellyState fields finite
Three test sizes:
integrated_fuzz_n1_short — N=1, 200 events
integrated_fuzz_n8_medium — N=8, 500 events
integrated_fuzz_n64_long — N=64, 1000 events
All three pass on RTX 3050. The N=64 × 1000 case exercises 64,000
event-snapshots × 16,000 decisions × ~12,800 trade attempts without
any invariant violation or NaN propagation.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Closes the spec §8 Ring 3 deferral. Two integration tests against
FOXHUNT_TEST_DATA real MBP-10 day files:
buy_and_hold_full_day
Walk the first .dbn.zst from open to close; submit a market buy
of 1 lot at the first event, run the full step-loop
(apply_snapshot + step_resting_orders) over every subsequent
event, then close with a market sell at the last event. Assert
realized P&L matches (close_mid - open_mid) within ±2 ticks per
spec slippage budget. Proves the book-walking + position
accounting + step-loop orchestration are wire-level correct
against real data, not just hand-crafted JSON fixtures.
walk_book_one_full_day
Same fixture, no trades — just iterates every event through
apply_snapshot + step_resting_orders and asserts the book
invariants (bid/ask monotonicity, no-crossed-book) hold at every
10_000-event checkpoint across the full day. Stress test of
the kernel against real market microstructure (gaps, locked
markets, regime shifts).
Both tests skip gracefully when FOXHUNT_TEST_DATA is absent OR the
predecoded sidecars are empty (the current 40-byte placeholder
fixtures in test_data/futures-baseline/ES.FUT/*.predecoded.bin will
trigger the skip path; populating those with a real Databento decode
makes both tests fire end-to-end). Ring 3 is operational, not
CI-mandatory.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The hardcoded WeightedByRealizedSharpe path from C7 is now joined by
a stack-based bytecode interpreter that consumes Strategy::flatten()
output, unlocking RegimeSwitch / Portfolio / non-default Ensemble /
single-Leaf compositions specified via policy-grid YAML.
cuda/decision_policy.cu — new kernel `decision_policy_program`:
Stack-based VM with parallel (value, attribution_mask) stacks.
Opcodes mirror src/policy/mod.rs::OpCode exactly:
NoOp / PushScalar / EvalRegime / BranchIfRegime
EmitPerHorizonSize (computes sized intent from alpha[h] +
IsvKellyState[h] using same Kelly + ISV-cap formula as the
hardcoded default)
AggMean / AggWeightedSharpe / AggMaxConfidence (pop n values,
push aggregated, OR attribution masks)
ApplyConflict (v1 no-op, reserved for Portfolio)
WriteOrder (terminal — converts top-of-stack to market_target +
open_horizon_masks attribution if currently flat)
AggWeightedSharpe recovers the source horizon from a single-bit
attribution mask to look up recent_sharpe; multi-bit masks (nested
aggregators that collapsed horizons) fall back to uniform weight.
decision_policy_default extended with a program_lens param: skips any
backtest whose plen > 0 (the program kernel handled it). The two
kernels run sequentially in step_decision_with_latency with mutual
exclusivity on each backtest slot.
LobSimCuda gains:
upload_program(b, &Program) — uploads a Strategy::flatten() output
to backtest b's slot in program_table_d, updates program_lens_d.
set_regime(b, regime_id) — writes regimes_d for OP_EVAL_REGIME /
OP_BRANCH_IF_REGIME consumption.
BacktestHarnessConfig.strategies (Vec<Strategy>) — empty means every
cell uses the hardcoded default; non-empty len must equal n_parallel
and each strategy is flattened + uploaded at construction.
bin/fxt-backtest --policy-grid <yaml> path now actually plumbs through
to the kernel (was parsed-but-ignored in C9). Empty grid keeps the
default behaviour.
New fixture decision_program_h4_only: uploads Strategy::Leaf(h4_only)
flattened to (EmitPerHorizonSize, WriteOrder) — 2 instructions, 16
bytes — and verifies the bytecode kernel produces equivalent end-state
to the hardcoded default (3 lot buy at vwap=5500). Proves the VM
dispatcher works end-to-end.
12/12 GPU fixtures green. 33 lib unit tests + 3 Ring 2 fuzz tests
still green.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The in-flight machinery from C11 is now reachable from the harness +
CLI. New step_decision_with_latency on LobSimCuda:
latency_ns == 0 → existing immediate-match path (submit_market_immediate
kernel fills against current book).
latency_ns > 0 → host reads market_targets, converts each non-noop
into seed_limit_order(active=2, price=very-aggressive,
arrival_ts_ns = current + latency_ns). The
resting_orders kernel promotes + fills at arrival
time, so the order sees whatever book exists then —
not the book at decision time.
Aggressive-price heuristic: buy at 1e9 / sell at 0 — the marketability
check (price ≥ best ask for buy, ≤ best bid for sell) unconditionally
crosses at arrival; the kernel then walks book levels for the actual
fill price. This models "market order with latency" correctly because
slippage emerges from the book-walking at arrival, not from a limit
price boundary.
BacktestHarness gains a latency_ns field; harness::run() always calls
step_decision_with_latency now. Also calls sim.step_resting_orders
per event (with trade_signed_vol=0.0 — the Mbp10RawInput trade-flow
hookup is deferred to a trades-feed integration commit) so in-flight
orders get a chance to promote on every snapshot.
bin/fxt-backtest no longer ignores --latency-ns; the flag value
propagates through BacktestHarnessConfig.latency_ns into the kernel
path. Default stays at 100_000_000 (IBKR + Scaleway baseline per
spec §4). Setting --latency-ns 0 selects the legacy immediate path.
Adds latency_in_flight_miss GPU fixture: limit at 5510 submitted
active=2 at T=1s with arrival_ts=T+100ms. step_resting at T+50ms
sees no promotion (still in-flight). Book moves +5 against buyer
during the 100ms window. step_resting at T+150ms promotes the
limit and fills it at the WORSE post-move book (vwap=5505 vs the
original 5500 it would have hit without latency). Slot ends active=0.
11/11 GPU fixtures green on RTX 3050. 33 lib tests still green.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
