chore/gitea-replace-gitlab
99 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
jgrusewski
|
cfaa420bd4 |
fix(ml-alpha): Phase 0 — per-step authoritative USD pnl in diag + fix mislabeled fields
The diag's `trading.*_pnl_*_usd` fields were systematically mislabeled, making
the eval verdict signal untrustworthy. Three quantities shared the `_usd`
suffix but were not in USD:
- `eval_summary.total_pnl_usd` (USD ground truth, $50/pt ES applied)
- `trading.realized_pnl_cum_usd` (cumulative SHAPED reward at close events;
pts × lots × Phase-5 shaping; NOT USD despite the suffix)
- `trading.pnl_cum_usd` (mathematically nonsense — reads rewards_d after
apply_reward_scale AND rl_popart_normalize whiten it in-place; dividing
by current_scale un-does only the first transform)
At baseline mid-smoke (b=128, 2000+500, seed=42): eval_summary.total_pnl_usd
= -$4,457,625 while diag.trading.realized_pnl_cum_usd = +$469k and
diag.trading.pnl_cum_usd = +$12.9k. 346x ratio, sign-opposite. Same
mechanism as `pearl_grwwh_eval_catastrophic_collapse` ($234M cluster
discrepancy). Every Pearson(reward, Δpnl_usd) computation against these
fields was shaped-vs-shaped tautology, not pnl alignment.
This commit:
DELETE `trading.pnl_cum_usd` (mathematically nonsense post-popart).
RENAME `trading.realized_pnl_cum_usd` -> `trading.shaped_reward_close_event_cum`.
Truthful name; the underlying values are post-Phase-5 shaped reward
summed at close events, useful for gradient-signal diagnostics but never
to be confused with USD pnl.
ADD `trading.realized_pnl_usd_delta` and `trading.realized_pnl_usd_cum` in
diag.jsonl / eval_diag.jsonl. Source: new per-batch float buffer
pnl_step_close_usd_d on LobSimCuda, zeroed via raw_memset_d8_zero before
each pnl_track_step launch, written by pnl_track.cu's close branch as
realised_pnl_usd_fp / 100.0 (same arithmetic as eval_summary's
TradeRecord aggregator, applying ES $50/pt). Single-writer per block —
no atomicAdd per feedback_no_atomicadd.
Direct readback via read_slice_d_into<f32> from sim.pnl_step_close_usd_d()
after step_with_lobsim_gpu returns (main stream already synchronized via
self.stream.synchronize() at pnl_track_step exit). NOT routed through
diag_staging double-buffer — a prior implementation tried this and
broke determinism (cross-stream race between main-stream
raw_memset_d8_zero + pnl_track_step write and diag-stream
raw_memcpy_dtod_async read). Direct same-stream read avoids the race
entirely and stays mega-graph compatible (the readback runs AFTER
per-step pipeline finishes, outside any captured CUDA graph).
Cumulative tracked Rust-side in alpha_rl_train.rs; reset at train->eval
boundary so realized_pnl_usd_cum tracks the eval window only.
scripts/tier1_5_verdict.py upgrade:
- signal_reward_alignment reads trading.realized_pnl_usd_cum
- signal_eval_pnl falls back to realized_pnl_usd_cum
- signal_consistency upgraded from WARN-at-5% to KILL-at-$50 with a new
consistency_kill_usd threshold key; cross-source disagreement is now
a hard kill not a warning
tests/eval_diag_emission.rs: bumped EXPECTED_LEAVES 711 -> 712 (-1 deleted,
-0 renamed, +2 added); added invariant that cum == running_sum(delta) and
regression check that legacy pnl_cum_usd / realized_pnl_cum_usd fields are
absent from the schema; allowed n_eval_steps + 1 row count for the
drain-row pattern.
Falsification gate (load-bearing, must hold every smoke from this commit
forward): `diag.last_eval_row.trading.realized_pnl_usd_cum` matches
`eval_summary.total_pnl_usd` within $50.
Validation (mid-smoke b=128, 2000 train + 500 eval, seed=42, RTX 3050):
- cargo build --release --example alpha_rl_train -p ml-alpha: exit 0
- cargo test -p ml-alpha --test eval_diag_emission: PASS (712 leaves)
- local-mid-smoke.sh: exit 0, drain row at step 2500
- cross-source consistency: eval_summary=-$4,457,625.00 vs
diag.realized_pnl_usd_cum=-$4,457,625.18 -> delta=$0.18 (exactly fp/100
f32 rounding noise; well within $50 tolerance)
- ./scripts/determinism-check.sh --quick: PASS — all checksums.* leaves
match across all 200 rows (rel-tol 1e-5, abs-tol 1e-7)
- Pre-commit hook on staged diff: PASS
Memory pearls created in this session document the diagnostic chain:
- pearl_diag_pnl_fields_are_shaped_reward_not_usd (the mislabeling root)
- pearl_reward_signal_anti_aligned_with_pnl ADDENDUM 2026-06-02b
(correction to the 2026-06-02 ADDENDUM — the "+0.93 Pearson at HEAD"
finding was a measurement artifact because both sides were in shaped
pts x lots units, not USD)
- pearl_advantage_kernel_is_done_gated_td_not_gae (related signal-density
gap — Phase 1.5 follow-up)
- pearl_phase5_term_4_is_almost_potential (Ng-Harada-Russell analysis of
Phase 5 shaping terms — Phase 1 follow-up)
Unlocks: TRUE Pearson(rewards.sum, Δpnl_usd) can now be measured per-step
at HEAD. The eval-collapse investigation (next phases: Ng-Harada-Russell
shaping cleanup + GAE upgrade) is now falsifiable with bit-deterministic
verdicts grounded in authoritative USD pnl.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
87a8259c6e |
fix(build): ml-backtesting honors CUDA_COMPUTE_CAP for sm_90 on H100
alpha-rl-mbg2n on H100 failed at LobSimCuda::new with `CUDA_ERROR_NO_BINARY_FOR_GPU` loading book_update.cubin. Root cause: ml-backtesting/build.rs read only `FOXHUNT_CUDA_ARCH` (set by lob-backtest-sweep-template) but NOT `CUDA_COMPUTE_CAP` (set by alpha-rl-template from `nvidia-smi --query-gpu=compute_cap` inside the compile pod). On H100 it silently fell through to default sm_86; the sm_86 cubin contains no PTX → no JIT path on sm_90 device. The docstring claimed "Mirrors crates/ml-alpha/build.rs" — it didn't (ml-alpha reads CUDA_COMPUTE_CAP). Completing the mirror now: detect_arch returns sm_<NN> honoring (in order): 1. CUDA_COMPUTE_CAP numeric env (alpha-rl-template) 2. FOXHUNT_CUDA_ARCH sm_-prefixed env (lob-backtest-sweep-template) 3. nvidia-smi --query-gpu=compute_cap at build time 4. Default sm_86 (RTX 3050 Ti local dev) Both production argo templates now produce sm_90 cubins on H100 and sm_89 on L40S without further changes. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
fa0858f0b1 |
feat(lobsim): per-backtest trade-count read + 4× TRADE_LOG_CAP (E.1-E.3)
Adds two new public methods to LobSimCuda + bumps TRADE_LOG_CAP to prevent eval-phase ring-buffer wrap at cluster scale. - read_per_backtest_trade_counts() -> Vec<u32>: cumulative trade counters per backtest (length n_backtests). Replaces the broken pattern in alpha_rl_train.rs where head_before_eval = aggregate across batch was compared against all_records = single-account ring. - read_trade_records_all() -> Vec<Vec<TradeRecord>>: all backtests' rings in one call. Mapped-pinned staging per feedback_no_htod_htoh_only_mapped_pinned: allocate MappedRecordBuffer<u8> for the payload + MappedRecordBuffer<u32> for heads, DtoD copy from device buffers into mapped-pinned dev_ptrs, sync, read host_ptrs. - TRADE_LOG_CAP 1024 → 4096: cluster v11 (alpha-rl-8ll7j) showed ~342 eval trades/account mean with peaks toward 1000. 4096 gives 4× headroom; memory cost b=1024 × cap × 40 B = 167 MB (was 41 MB), comfortable on L40S 48GB / H100 80GB. Both methods synchronize after DtoD so host reads see the data. E.4 (alpha_rl_train.rs aggregation block replacement) lands separately after Phase A cluster validates to avoid alpha_rl_train.rs conflict. See spec docs/superpowers/specs/2026-05-31-eval-summary-trade-aggregation-design.md and plan docs/superpowers/plans/2026-05-31-eval-summary-trade-aggregation.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
c67b58d0d0 |
perf(lobsim): convert step_fill + step_pnl_track to raw_launch
Replace 24 cudarc .arg() calls (8+16) with RawArgs + raw_launch. Eliminates ~48 cuStreamWaitEvent + cuEventRecord per step from cudarc's event tracking. Last hot-path launch_builder in the GPU RL training pipeline. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
6b89dbfcb8 |
perf(rl): eliminate 2 of 3 per-step lobsim syncs — 21ms → 10ms/step
Remove unnecessary stream.synchronize() from: 1. step_fill_from_market_targets (lobsim): stream ordering handles the dependency — step_pnl_track launches on the same stream 2. step_pnl_track (lobsim): downstream kernels read pos_d via stream ordering, no host read needed per step Defer pos_fraction readback by one step (same pattern as loss readback) — eliminates the third 7ms sync. pos_fraction is a dataset-level statistic that barely changes step to step. nsys: 3.0 → 1.0 slow syncs/step. Sync time: 21.8 → 9.65 ms/step. The remaining sync is the training graph event wait (actual compute). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
6f645df11f |
feat(rl): wire lobsim book data from GPU SoA — proper rewards in GPU loader path
Add apply_snapshot_from_device to RlLobBackend: DtoD copy from perception SoA buffers into lobsim book arrays + book_update kernel. No host roundtrip, no sync (stream-ordered). step_with_lobsim_gpu now calls apply_snapshot_from_device with batch 0's last-snapshot (K-1) book data. Produces non-zero dones, pnl, wr. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
a583bb508c |
feat(rl): pyramiding semantics — add/partial-flat/anti-martingale sizing
Implements the full pyramid trade-management suite:
P7.a: actions_to_market_targets gates pyramid adds on ISV-driven
threshold (slot 506); rl_unit_state_update allocates sequential
unit slots on position growth, deactivates oldest on shrink.
P7.b: HalfFlat (a9/a10) closes oldest unit's lots when pyramid>1;
trail-stop routes breaches through HalfFlat + close_unit_index
override instead of nuclear full-flat.
P7.c: Anti-martingale sizing on opening actions via signed outcome EMA
(slot 508) — size_eff = base × clamp(1 + κ × ema, MIN, MAX).
Diag: pyramid { units_count, add_count, outcome_ema } in step JSONL.
ISV slots: 506 threshold, 507 add_count, 508 outcome_ema,
509 κ, 510 MIN, 511 MAX. RL_SLOTS_END → 512.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
7df7c81d37 |
refactor(rl): FRD horizons + range_σ are ISV-driven, not literals
Closes the literal/const drift gap that F.5 introduced. Per
feedback_isv_for_adaptive_bounds + feedback_single_source_of_truth_no_duplicates:
adaptive bounds belong in ISV (or in a single canonical const that
ISV references), never duplicated as literals across modules.
Single canonical source: `crate::rl::common::FRD_HORIZON_TICKS` +
`FRD_BUCKET_RANGE_SIGMA` (already declared in F.5).
Producer-side fixes:
* Trainer ISV bootstrap (integrated.rs): the seed values for slots
500-503 now dereference the canonical consts instead of hardcoded
60.0/300.0/1800.0/3.0 literals. Future tuning of the consts
automatically propagates to both ISV seeds and loader-side
labels — no manual sync required, no drift possible.
* compute_frd_labels (loader.rs): takes `horizon_ticks` and
`range_sigma` as parameters instead of reading consts directly.
Caller (the file-load closure) sources them from the new
MultiHorizonLoaderConfig fields.
Consumer-side fixes — 8 MultiHorizonLoaderConfig literal sites now
provide the two new fields, all defaulting to the canonical consts:
* crates/ml-alpha/src/data/loader.rs (2 internal test-fixture sites)
* crates/ml-alpha/tests/multi_horizon_loader.rs (2 sites)
* crates/ml-alpha/examples/alpha_train.rs (2 sites)
* crates/ml-alpha/examples/alpha_rl_train.rs (2 sites)
* crates/ml-backtesting/src/harness.rs (1 site)
* crates/ml-backtesting/tests/{trainer_parity,ring3_replay}.rs (2 sites)
The "optimal by default" property is preserved: every caller that
doesn't explicitly override gets the spec-recommended 60/300/1800
ticks + ±3σ. Callers that need to retune set the config fields, and
the trainer's ISV slots provide a runtime knob for the same numerics.
Verification (RTX 3050 Ti):
* cargo check -p ml-alpha -p ml-backtesting --examples --tests → clean
* cargo test --lib (6/6 unit tests for FRD label gen + loss_balance) → pass
* frd_head 10/10 + integrated_trainer_smoke 1/1 + trade_mgmt 5/5 → pass
* audit-rust-consts → 0 flags
The two new MultiHorizonLoaderConfig fields are required (no Default
impl) — callers MUST opt in to the FRD label-generation contract by
naming the fields. This is the same discipline applied across other
config consumers; making them Option<...> would silently default to
"no FRD labels" and break F.4's expected supervised signal.
|
||
|
jgrusewski
|
cc4c47f471 |
audit(rust-consts): catch literal-vs-const drift + cleanup BOOK_LEVELS=10
Audit script (audit-rust-consts.sh) scans Rust src/examples for numeric
literals mirroring structural kernel-side consts (N_ACTIONS, Q_N_ATOMS,
HIDDEN_DIM, MAX_UNITS, BOOK_LEVELS). Closes the layer-3 gap noted in
feedback_use_consts_not_literals_for_structural_dims:
Layer 1: kernel `#define` allowlist → audit-isv
Layer 2: Rust `pub const` canonical → exists (e.g. N_ACTIONS in rl/common.rs)
Layer 3: Rust literals mirroring (2) → audit-rust-consts (this commit)
Honors `// audit-ignore: <SYMBOL>` per-line markers and skips `[u8; N]`
byte-buffer patterns (high false-positive class — almost always I/O
scratch, not structural dims).
Cleanup driven by first run (19 real flags, no grandfathering):
* New canonical: `BOOK_LEVELS` in `ml-alpha/src/cfc/snap_features.rs`
(10 book levels = same place as `Mbp10RawInput` struct)
* `ml-backtesting/src/lob/mod.rs`: redefine as `pub use` re-export from
ml-alpha (single source of truth; ml-backtesting depends on ml-alpha
via `Mbp10RawInput` already)
* 19 sites switched literal `10` → `BOOK_LEVELS`:
- snap_features.rs:44-47 (struct fields)
- data/loader.rs:872-876, 960 (Mbp10Snapshot → Mbp10RawInput convert)
- data/aggregation.rs:161 (level-wise aggregation loop)
- trainer/perception.rs:2750-2756, 6272-6278, 6686-6690, 7247-7253
(snapshot → batch staging loops)
- tests/lob_sim_fuzz.rs:21, lob_sim_integrated_fuzz.rs:22 (duplicate
const → use ml_backtesting::lob::BOOK_LEVELS)
* 5 sites marked `// audit-ignore: BOOK_LEVELS — <reason>`:
- harness.rs:572,574,594 (conviction-bucket histograms, 10 ≠ depth)
- multi_horizon_labels.rs:489,557,564 (10-element test price vecs)
Re-run after fixes: 0 suspect literals flagged. PASS.
|
||
|
jgrusewski
|
20c835713b |
fix(rl): wire TrailTighten/TrailLoosen + SP20 P1+P5 foundation
scripts/audit-wiring.sh dogfood pass flagged a7 (TrailTighten) and
a8 (TrailLoosen) as actions with no consumer anywhere in the
codebase (canonical pearl_dead_trail_stop_actions_a7_a8). Fix
bundles SP20 P1 (per-unit trade state buffers) and P5 (trail-stop
kernels) since they're the same architectural work.
Three new kernels:
rl_unit_state_update.cu — per-batch trade state machine. Runs
AFTER fill+extract_realized_pnl_delta.
Detects open/close/reverse position
transitions and populates unit slot 0
with entry_price, entry_step, lots,
initial_r, trail_distance. Slots 1-3
allocated for SP20 P7 pyramid expansion
but unused this commit.
rl_trail_mutate.cu — handles a7/a8 actions. Mutates ALL
active units' trail_distance bounded
by ISV [MIN, MAX] with symmetric
reciprocal adjust rate per SP20 §4.12:
a7: trail = max(MIN, trail × rate)
a8: trail = min(MAX, trail / rate)
rl_trail_stop_check.cu — per-batch per-unit breach check. Reads
shared lobsim best book (bid/ask),
computes mid, compares to each active
unit's (entry ± trail). On breach,
OVERRIDE actions[b] to FlatFromLong
(a3) or FlatFromShort (a4). Force-close
routes through existing flat plumbing
per pearl_stop_checks_run_at_deadline_cadence.
SP20 v3 §3 P5 calls for routing close
via partial-flat (a9/a10) so only the
at-risk unit closes — that needs P4
(N_ACTIONS=11). For now, ANY unit's
breach closes ENTIRE position via full
FlatFromLong/Short.
Per-batch per-unit buffers (8 new in trainer):
unit_entry_price_d [B × 4] f32
unit_entry_step_d [B × 4] i32
unit_lots_d [B × 4] i32
unit_initial_r_d [B × 4] f32
unit_trail_distance_d[B × 4] f32
unit_active_d [B × 4] u8
pyramid_units_count_d[B] i32
unit_prev_pos_lots_d [B] i32 (state-machine tracker, separate
from extract_realized_pnl_delta's
prev_position_lots_d for clean
kernel composability)
4 new ISV slots (494-497):
RL_TRAIL_MIN_INDEX — trail distance floor (seed 0.001)
RL_TRAIL_MAX_INDEX — trail distance ceiling (seed 100.0)
RL_TRAIL_K_INIT_INDEX — initial trail multiplier (seed 2.0, Turtle 2N)
RL_TRAIL_ADJUST_RATE_INDEX — tighten ratio (seed 0.9; symmetric reciprocal for loosen)
RL_SLOTS_END: 494 → 498.
LobSim exposes bid_px_d() + ask_px_d() public accessors. RlLobBackend
trait extended with the two accessors; the LobSimCuda impl wires
through.
Override stack ordering per SP20 §2.3:
1. rl_pi_action_kernel (sample)
2. rl_trail_mutate (a7/a8 → mutate, before stop check)
3. rl_trail_stop_check (per-unit breach → override action)
4. actions_to_market_targets (execute, including overridden flat)
5. step_fill_from_market_targets
6. extract_realized_pnl_delta
7. rl_unit_state_update (detect post-fill transitions)
Audit infrastructure refined as part of dogfooding:
* audit-isv allowlist extended for BOOK_LEVELS (structural book
depth) and ACTION_* prefix (enum-mirror constants — these are
structural API contracts matching src/rl/common.rs::Action positions)
* audit-wiring action-handler regex now matches BOTH literal
`action == <idx>` and `action == ACTION_<UPPER_SNAKE>` patterns,
and treats != as a handler too (a guard against the action is
valid wiring)
Both `audit-isv.sh` and `audit-wiring.sh` PASS cleanly with the
full manifest. audit-diag scheduled for first SP20 phase that adds
diag fields (this commit deliberately keeps diag exposure minimal
— full per-unit + trail diag blocks come with SP20 P13).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
7e7c8b5d90 |
feat(rl): R6 — replace LobEnv with kernel-driven GPU-pure env step
Replaces the flawed Phase F+G LobEnv trait + LobSimEnvAdapter +
MockLobEnv fixture with a GPU-pure env-interaction path.
DELETED:
- LobEnv trait (host-method-per-batch surface)
- MockLobEnv fixture (the toy-bandit pattern that hid the production defects)
ADDED:
- RlLobBackend trait (src/rl/reward.rs): narrow device-oriented
surface (apply_snapshot, pos_and_market_targets_mut, pos_d,
pos_bytes, step_fill_from_market_targets, n_backtests). Single
purpose: break the ml-alpha ↔ ml-backtesting dep cycle. No
mocking layer.
- 3 new GPU-pure kernels (cuda/):
* extract_realized_pnl_delta.cu — reads pos.realized_pnl +
position_lots from device Pos array, writes per-batch
(reward delta, done flag), updates trainer's prev_* buffers.
* apply_reward_scale.cu — element-wise rewards *= ISV[406].
* actions_to_market_targets.cu — 9-action grid → market_targets
{side, size} on device, reading current position_lots for
conditional Flat-from-Long/Short.
- LobSimCuda impls RlLobBackend (ml-backtesting/src/sim/mod.rs)
plus a new step_fill_from_market_targets entry that runs
submit_market_immediate + step_pnl_track without the host-side
targets-vec build. pos_and_market_targets_mut returns disjoint
field borrows (&pos_d, &mut market_targets_d).
- IntegratedTrainer: 3 cubin includes, 3 module/function fields,
launch_apply_reward_scale launcher, prev_realized_pnl_d /
prev_position_lots_d / rewards_d / dones_d buffers,
step_with_lobsim signature switched to RlLobBackend, body's
Step 5 rewritten as kernel-driven (no per-batch host loop, no
individual submit_action calls).
R6 PARTIAL — work R7 lifts:
- Thompson + log_pi stay host (R7 uses R4 kernels)
- mean_abs_pnl EMA stays host (R7 uses R3 ema_update_on_done)
- Advantage/return stays host (R7 uses R3 compute_advantage_return)
- 4 final DtoH copies for step_synthetic's host-slice signature
(R7 lifts step_after_encoder_forward to device-buffer args)
The "DtoH the device rewards/dones at end" comment in
step_with_lobsim documents the boundary. After R7, hot path has
zero host loops other than the now-unused Thompson host loop
(which R7 retires in favour of rl_action_kernel from R4).
Tests:
- integrated_trainer_smoke.rs: rewritten — real LobSimCuda with
synthetic book, one step_with_lobsim call, asserts finite losses
+ λs sum to 1. Smoke gate, not a convergence test.
- dqn_toy.rs, ppo_toy.rs: retired (empty stubs documenting
rationale). Files preserved so rename history survives. The
MockLobEnv toy-bandit pattern intrinsically couldn't catch the
production defects #1, #2, #5 that motivated this rebuild.
ml-backtesting added as ml-alpha dev-dep (cycle-safe: production
direction is ml-backtesting → ml-alpha; dev edge only loads when
building ml-alpha's own tests/examples).
Build cache-bust v29. cargo check + cargo build for all R-phase
tests green (pre-existing heads_bit_equiv index-OOB persists).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
da2ad438ca |
feat(rl): R2 — fee model + loader pair API
Ports two scope-complete fixes from the ml-alpha-phase-f-g-flawed reference branch: A7 fee model: - order_match.cu::submit_market_immediate gains 2 new kernel args (cost_per_lot_per_side, total_fees_per_b) mirroring the per-fill fee deduction in resting_orders.cu::apply_fill_to_pos:213-219. Fee deducts from pos.realized_pnl on EVERY fill (open, scale-in, counter); total_fees_per_b accumulates for telemetry. Single-writer-per-block plain += is safe — line 79 has `threadIdx.x != 0 → return` (feedback_no_atomicadd). - LobSimCuda::submit_market launch updated to pass the 2 new args. - LobSimCuda::upload_cost_per_lot_per_side host API lets callers configure ES-realistic fees (≈$1.25/contract/side). Default alloc_zeros = $0; production decision-policy path is unchanged (uploads its own cost via step_decision_with_latency). A8 loader pair API: - MultiHorizonLoader::next_sequence_pair returns (LabeledSequence, LabeledSequence) at adjacent anchors in the same source file. anchor_t sampled from [min_anchor, max_anchor−1) so anchor+1 also fits the upper-bound. Counts as ONE yielded sequence against n_max_sequences. - next_sequence_random and next_sequence_pair share a new private helper build_sequence_at(lf, anchor) -> LabeledSequence that contains the multi-resolution windowing logic. Single source of truth for the build (feedback_single_source_of_truth_no_duplicates). - next_sequence's caller-facing contract (random anchor, one sequence per call) is unchanged — alpha_train.rs supervised pipeline keeps working as-is. No new local tests this phase per the rebuild plan (R2): the fee deduction with default cost=0 is a no-op for existing callers, and G7 in R6 covers the with-fees path via a rebuilt reward_calibration test driving LobSimCuda directly (no LobEnv adapter). cargo check -p ml-alpha -p ml-backtesting + cargo build --tests on ml-backtesting both green; baseline test suite unaffected. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
f68e0a1d0d |
revert(loader): multi-resolution default '1:32' (single-scale) after htpp6 falsification
The Phase 1 multi-resolution layout (10 raw + 10 agg@30 + 12 agg@100) regressed ALL horizons in alpha-perception-htpp6 (2026-05-22): - auc_h100: 0.681 -> 0.512 (-0.169) - auc_h300: 0.617 -> 0.506 (-0.111) - auc_h1000: 0.576 -> 0.526 (-0.050) Hypothesis falsified. Root cause: Mamba2+CfC SSM encoder already used all 32 raw ticks effectively via state-recurrence; replacing 22 raw ticks with arithmetic-mean aggregates destroyed within-window microstructure variance (the actual h100 signal) AND broke temporal continuity that the recurrence relies on. Δt Fourier encoder couldn't compensate. Architectural pearl: SSM/RNN/CfC + multi-resolution input is incompatible without separate-encoder-per-scale or explicit scale tokens. Transformer-style positional encoding tolerates scale-mixing; recurrent state updates assume consecutive positions. Reverts default to '1:32'. Adds explicit single_scale_32() constructor for callers (harness, tests). Keeps default_three_scale() in code with deprecation note for future sub-variant experiments. Production defaults across alpha_train CLI, Argo template, dispatcher script, ml-backtesting harness now match the proven baseline. |
||
|
jgrusewski
|
99982cc8fb |
chore(ml-backtesting): migrate to MultiResolutionConfig
Backtest harness + tests use default_three_scale config (1:10,30:10, 100:12 = 1510 ticks of context). Replaces the previous seq_len=32 single-scale path which is now deleted. |
||
|
jgrusewski
|
78a9e08358 |
feat(loader): InstrumentFilter::FrontMonth for cross-quarter ES.FUT data
Replaces Option<u32> instrument_id_filter with InstrumentFilter enum {All,
Id(u32), FrontMonth}. FrontMonth runs a two-pass detect over the DBN
stream: pass 1 counts instrument_ids and collects SymbolMapping records,
picks the dominant id, validates it resolves to an ES contract via regex
ES[FGHJKMNQUVXZ]\d{1,2}; pass 2 streams the filtered records.
Motivated by alpha-perception-k54wd: a single-id filter on parent-symbol
ES.FUT data caught Q1 2024 (kept=73M) but kept=0 for Q2-Q9 because ES
front-month rolls quarterly (ESH4 -> ESM4 -> ESU4 -> ESZ4 ...). FrontMonth
self-tunes across the rolls without needing a per-file id table.
Sidecar keys distinguish modes: mbp10 / mbp10_instr<id> / mbp10_front_month.
CLI flag renamed --instrument-id -> --instrument-mode {all,id=N,front-month}
with matching parameter rename in argo-alpha-perception.sh + template.
|
||
|
jgrusewski
|
783297e002 | feat(loader): instrument_id filter + outdated test fix + sp18 fingerprint | ||
|
jgrusewski
|
859d0c738b |
feat(aux-labels): wire D-style outcome labels into MultiHorizonLoader
Extends MultiHorizonLoaderConfig with `outcome_label_cost` (default DEFAULT_OUTCOME_LABEL_COST_ES = 0.5 = 2 ticks for ES round-trip). LabeledSequence + LoadedFile gain outcome_long + outcome_short arrays of [Vec<f32>; N_HORIZONS] populated by generate_outcome_labels_d during LoadedFile construction (same !inference_only branch as BCE labels). CLI exposure: alpha_train.rs gains --outcome-label-cost flag with the ES default. All 6 MultiHorizonLoaderConfig consumers updated atomically per feedback_no_partial_refactor: alpha_train (train + val loaders), in-file test fixtures (×2), multi_horizon_loader.rs (×2), harness.rs, trainer_parity.rs, ring3_replay.rs. cargo check --workspace clean; ml-alpha lib tests 41 passed. B3+ tasks not yet touched: outcome labels are computed and propagated to LabeledSequence but no consumer reads them yet (aux trunk Tasks B3-B5 will wire that). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
0c8b4b5608 |
refactor(per-horizon): N_HORIZONS 5→3 — ml-backtesting full propagation
Source migration: - crates/ml-backtesting/src/lob/mod.rs:5 + policy/mod.rs:18: local const N_HORIZONS 5→3 via re-export from ml_alpha::heads::N_HORIZONS (single source of truth across crates) - crates/ml-backtesting/src/sim/mod.rs:1751,1773,1784: [IsvKellyStateHost; 5] array literals → ; N_HORIZONS] - crates/ml-backtesting/cuda/lob_state.cuh:9: #define N_HORIZONS 5→3 (this triggers cubin rebuild of decision_policy + 4 other ml-backtesting kernels that #include this header) Test migration (8 test files + 2 JSON fixtures): - threshold_and_cost.rs, decision_floor_coldstart.rs, parallel_sim_ correctness.rs, stop_controller.rs (37+10+1 broadcast_alpha calls), lob_sim_integrated_fuzz.rs, lob_sim_fixtures.rs: hardcoded [f32; 5] alpha-probs and [IsvKellyStateHost; 5] arrays → N_HORIZONS-sized via std::array::from_fn or [v; N_HORIZONS] literals - trainer_parity.rs:34 + ring3_replay.rs:47: horizons literal [30,100,300,1000,6000] → ml_alpha::heads::HORIZONS - fixtures/decision_alpha_buy_close.json + decision_program_h4_only.json: 5-element warm_start_isv_kelly / alpha_probs / expected_isv_kelly_after trimmed to 3 elements; active horizon relocated to N_HORIZONS-1 Library lib-test rewrite (per pearl_tests_must_prove_not_lock_observations): - crates/ml-backtesting/src/policy/mod.rs:197-233: lib tests default_strategy_has_5_horizon_leaves + ..._flattens_to_5_emits... renamed to N_HORIZONS-parametric form (observed-value 5 and 7 were bug-locks). cargo check -p ml-backtesting --all-targets: clean. cargo test -p ml-backtesting --lib: 33 passed. cargo test -p ml-backtesting --tests (non-CUDA, non-fixture-data): 2 passed, 53 ignored (CUDA-gated or FOXHUNT_TEST_DATA-gated). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
1e8188c947 |
refactor(per-horizon): N_HORIZONS 5→3, HORIZONS=[10,100,1000] — library + data layer
First commit of the horizon-rebase refactor (full plan at docs/superpowers/plans/2026-05-22-horizon-rebase-n3.md). Motivated by empirical evidence from this session's investigations: - C1: ES MBP-10 input ACF dies by L=300; K=1000/6000 are below noise floor - D3: binarized labels ρ(30,6000)=0.06; K=6000 carries no signal - Heads_w1 mask experiment: h6000 AUC collapsed 0.73→0.54, confirming long-horizon prediction needs cross-channel integration that 5-horizon bucket structure couldn't provide New horizon set [10, 100, 1000] matches C1's empirical autocorrelation elbows (~50ms, ~3.5s, ~45s wall-clock at 74µs median Δt). Scope of this commit: - crates/ml-alpha/src/heads.rs: N_HORIZONS 5→3, HORIZONS=[10,100,1000] - crates/ml-alpha/src/cfc/bucket_routing.rs: MAX_BUCKET_DIM 28→96, BUCKET_DIM_K=[43,43,42], BUCKET_CHANNEL_OFFSET=[0,43,86,128] - crates/ml-alpha/src/data/loader.rs: 5 hardcoded [T;5] types migrated - crates/ml-backtesting/src/sim/mod.rs: broadcast_alpha signature, local N_HORIZONS re-exports ml_alpha::heads::N_HORIZONS - crates/ml-backtesting/src/harness.rs: local N_HORIZONS re-export, per-horizon log labels, MultiHorizonLoaderConfig.horizons Tests + examples + CUDA kernels still need migration (subsequent commits per plan tasks 2-8). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
0384f76743 |
fix(decision-policy): use signed-conviction EMA, not magnitude-only EMA
Anti-calibration evidence (smoke ckpts across architecture variants
showed higher reported conviction → MORE negative PnL, -15.6 → -145.2)
traced to a magnitude/direction mismatch in the sizing formula:
conv_ema = EMA(|conviction_signed|) # magnitude smoothing
target_lots = sign(conviction_signed) # INSTANTANEOUS sign
× conv_ema × max_lots # × SMOOTHED magnitude
When per-horizon directions disagree event-to-event, the magnitude EMA
stays high (|x| EMA can't cancel) but the sign flips on noise. Result:
big trades in random direction whenever the 5 horizons disagree.
Replace with a single signed-conviction EMA:
conv_signed_ema = EMA(conviction_signed)
target_lots = sign(conv_signed_ema)
× |conv_signed_ema| × max_lots
Now BOTH sign AND magnitude come from the same smoothed signal. When
directions disagree, signed EMA → 0 collapses size to zero naturally.
Atomic migration across decision_policy.cu (2 kernels), pnl_track.cu
(open-branch reads the SIGNED buffer and takes fabsf for the magnitude-
semantics open_trade_state offsets that composite_exit_check forms ratios
from), and src/sim/mod.rs (renamed device buffers + accessor + 4 launch
sites). No legacy aliases per feedback_no_legacy_aliases; no parallel
buffers per feedback_single_source_of_truth_no_duplicates; α floor 0.4
preserved per pearl_wiener_alpha_floor_for_nonstationary; first-
observation bootstrap preserved per pearl_first_observation_bootstrap.
ml-backtesting lib: 33 passed.
GPU oracle: signed_conviction_ema_collapses_on_sign_flips passes —
observed steady-state shows |signed_ema| ≈ 0.205 (post-fix) producing
|target_lots| = 2 every tail event, vs the pre-fix bug's predicted
|target_lots| = 7 every tail event. Existing
multi_horizon_conviction_cancels_on_disagreement still passes (zero-
cancellation regime is even cleaner under signed EMA).
ml-alpha lib: 33 passed (no regression).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
ff9207e7bb |
feat(per-horizon-cfc): plumb kernel-step-trace into fxt-backtest inference path
Mirrors the alpha_train CLI flag (--kernel-step-trace <path>) into fxt-backtest. The PerceptionTrainerConfig already accepts the path; this commit just exposes the CLI surface and propagates through: fxt-backtest run --kernel-step-trace <path> -> BacktestHarnessConfig.kernel_step_trace_path -> PerceptionTrainerConfig.kernel_step_trace_path Sweep YAML gains a `kernel_step_trace: Option<PathBuf>` base field. Argo lob-backtest-sweep-template gains a `kernel-step-trace` workflow parameter (default disabled; when set, --features kernel-step-trace is passed to cargo build AND --kernel-step-trace to fxt-backtest). Gated behind the `kernel-step-trace` Cargo feature (matching alpha_train). When disabled (default), zero overhead. Enables per-step JSONL diagnostic emission from inference kernels -- needed for Smoke 2 deep-dive after sampling histograms (in parallel investigation) localize the per-horizon dynamics bottleneck. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
d2d34b6d4e |
diag(per-horizon-cfc): per-horizon alpha_probs sampling for Smoke 2 investigation
Adds --per-horizon-logit-sampling-stride CLI flag to fxt-backtest run/sweep
+ harness instrumentation that DtoHs alpha_probs every N events and
accumulates per-horizon mean/stddev/min/max + 10-bin histogram.
Emitted at end of CRT.diag block as:
per_horizon_logit_diag h{N}: count=... mean=... std=... min=... max=... hist=[...]
Goal: distinguish whether Smoke 2's uniform mean_run_len 1.04x across
horizons is caused by uniform per-horizon LOGITS (-> Task 11 scope
incomplete; heads_w1/w2/gate/main need block-diagonal too) or by
DIFFERENTIATED logits with collapsed downstream (-> different fix).
Default stride=None (disabled, zero overhead). With stride=1000 on
2M events, ~2000 DtoD stops x ~50us = ~100ms total overhead per run.
Per feedback_no_htod_htoh_only_mapped_pinned: uses mapped-pinned buffer
(single 5xf32 allocation reused on every sampling tick).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
f13d6c6dcf |
refactor(per-horizon-cfc): atomically remove MTER scaffolding
Per spec §2.5 and feedback_no_partial_refactor. Removes: - LoaderMode enum (single-variant after Sequential removal → deleted entirely) - next_sequence_sequential + sequential_file_idx + sequential_anchor_idx + last_call_was_file_boundary + is_file_boundary - last_seen_file_boundary, train_graph_boundary_state fields - notify_file_boundary method + boundary-state graph recapture logic - need_attn_pool_bootstrap match — always true now (random mode always bootstraps) - --loader-mode CLI flag + notify_file_boundary() call - loader-mode Argo template param Additional consumers migrated atomically (beyond the 4 files listed in the plan): ml-alpha/tests/perception_overfit.rs, ml-alpha/tests/multi_horizon_loader.rs, ml-backtesting/src/harness.rs, ml-backtesting/tests/trainer_parity.rs, ml-backtesting/tests/ring3_replay.rs — all referenced LoaderMode or the removed config fields. Workspace builds clean at this commit (pre-existing cudarc-cupti example and ml-crate test errors are unrelated to MTER removal — they fail at HEAD too). New per-horizon CfC arch lands in subsequent tasks of the same plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
7dd953aa2b |
feat(intervention-b): add LoaderMode::Sequential + attn_pool gate
Loader can now serve sequences in temporal order (Sequential mode); when active, the trainer skips attn_pool reset at non-file-boundary sequence boundaries (next commit will use this for stateful CfC + MTER). Random mode preserved as default; ZERO behavioral change for existing runs. Phased commit 1/8 of intervention B per docs/superpowers/specs/2026-05-21-crt-train-intervention-b-multi-timescale-readout.md |
||
|
jgrusewski
|
0e17fd4f2d |
diag(crt-2): per-horizon alpha-input EMA test — hypothesis investigation
Driven by ffr59 (commit
|
||
|
jgrusewski
|
b44a97ff94 |
diag(crt): empirical measurement battery for signal × controller × cost analysis
Pure-instrumentation commit. Zero behavior change. The Gate CRT.1 smoke p9cnk (commit |
||
|
jgrusewski
|
1e656948b3 |
arch(crt-1): composite exit_signal safety circuit-breaker (C1.4)
Per spec §4.3 + plan C1.4. Safety backup for the primary exit path
(target → 0 from collapsed multi-horizon conviction, C1.2). Catches
the edge case where conviction stays bounded above zero but the trade
is deep in unrealized loss, or short/long horizons disagree for many
events while threshold-gate logic still permits sizing.
Three terms, max-of-three composite:
degradation_term = (1 − conv_ema_now / conviction_at_entry) / 2.0
disagreement_term = disagree_consecutive_events / 32.0
pnl_decay_term = max(−pnl_adjusted_conv_ema, 0) / 1.0
exit_signal = max(deg, dis, pnl_decay)
Fires force-flat (market_targets = (3, 0)) when exit_signal >= 1.0.
State-update flow:
- pnl_track open branch: write conviction_at_entry (offset 20),
conviction_ema (offset 24, initialised to entry value),
pnl_adjusted_conviction_ema (offset 44, = 0),
peak_unrealized_pnl (offset 48, = 0). Takes new conviction_ema_per_b
read-only arg to snapshot at open.
- decision kernel intra-event (when pos open): update offsets
24/44/48/56 (disagreement counter) via update_open_trade_trajectory.
- composite_exit_check device helper: reads offsets 20/24/44/56,
writes (3, 0) on fire. Sibling of stop_check_isv with same return
semantics; called after trajectory update so all four reads see
the freshly-written values.
Fields not read in CRT.1 (offsets 28-43 per-horizon EMA, 52 degradation
counter, 60 horizon_idx_dominant) are NOT written by the open branch —
alloc_zeros covers init. Per feedback_no_hiding: don't populate unused
fields. Also fixes a latent C1.1 bug: pnl_track close branch read
horizon_idx from offset 20 (now conviction_at_entry, f32); moved the
read to offset 60 per the documented layout.
Threshold-bootstrap defaults (will be ISV-derived in CRT.2 alongside
the rest of the controller anchors per pearl_controller_anchors_isv_driven):
degradation_factor = 2.0 (50% conviction decay vs entry → ≤0.5 term)
disagreement_factor = 32.0 (32 events of short/long disagreement)
pnl_decay_factor = 1.0 (pnl_adj_conv structurally ∈ [-1, +1])
Under these defaults disagreement_term is the only term that can fire
on its own — degradation_term caps at 0.5 (conv_ema/conviction_at_entry
≥ 0) and pnl_decay_term caps at 1.0 in the limit. The three terms still
add evidence in superposition; CRT.2 will retune factors against ISV
percentiles. Unit-test coverage validates the disagreement path
(composite_exit_signal_fires_on_disagreement).
Per pearl_no_host_branches_in_captured_graph — all composite logic
lives inside the decision kernel; no host roundtrip in the per-event
hot path. Per feedback_no_htod_htoh_only_mapped_pinned — no new
memcpy or stream.synchronize() in step_pnl_track or
dispatch_latent_market_orders.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
84793ea46e |
fix(crt-1): add delta_floor: 0.0 to 3 missing test UniformSimParams initializers
The C1.3 commit
|
||
|
jgrusewski
|
7c850a6c02 |
arch(crt-1): no-trade band in seed_inflight — sparse action at signal cadence (C1.3)
Per spec §4.0 + plan C1.3. Without this gate, every fractional change in target_lots seeds an order via target-delta semantics; combined with the continuous controller invocation (every event after A1) this generates hyperactivity even with the multi-horizon §4.4 conviction formula (C1.2). v2 Gate-1 failures confirmed: scalar EMA + amplification clamp + per-event controller = 155k trades on a 2M-event smoke. The band absorbs fractional target adjustments so most events produce no order. When the signal genuinely shifts and target moves enough to cross the band, the kernel seeds. Continuous evaluation, sparse action. Greenfields atomic refactor — single config knob threaded through every layer in one commit: SweepBase.delta_floor: f32 (YAML, default 1.0 = 1 lot) SimVariant.delta_floor: Option<f32> (per-variant override) ResolvedSimVariant.delta_floor: f32 UniformSimParams.delta_floor: f32 BatchedSimConfig.delta_floor: Vec<f32> LobSimCuda.delta_floor_d: CudaSlice<f32> seed_inflight_limits_batched kernel param + skip-if-below-floor logic New accessor: LobSimCuda::read_inflight_count(b) — counts active != 0 limit slots for backtest b. Not cfg(test); follows existing read_limit_slot pattern. Test: no_trade_band_blocks_micro_delta verifies that a second decision with the same strong-bullish alpha (same target, effective = in-flight lots) produces delta=0 and does not re-seed. max_lots=1 keeps the arithmetic unambiguous. Existing tests: all UniformSimParams struct literals updated with delta_floor=0.0 (band disabled) so existing behaviour is preserved. harness.rs from_uniform path uses delta_floor=1.0 (production default). Hot-path discipline: the delta_floor_d upload happens once per run in the existing config-upload block (alongside threshold, cost, max_hold_ns); the kernel reads the device slot per-event without any host roundtrip. No memcpy_htod / dtoh / dtov / synchronize introduced on the per-event path. Per pearl_controller_anchors_isv_driven, this floor is currently a config constant; CRT.2 (Phase 2) makes it ISV-derived from rolling spread cost / signal volatility. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
632296021c |
arch(crt-1): multi-horizon ISV-weighted conviction replaces scalar EMA + rescale (C1.2)
Per spec §4.4 + plan C1.2. Replaces v2 A2 ( |
||
|
jgrusewski
|
e27fc90b9b |
arch(crt-1): open_trade_state 24→64 byte expansion — atomic refactor (C1.1)
Per spec §7 (v3 architecture reset, promoted from v2 Phase B into CRT.1).
Single-cache-line layout. Every reader and writer of open_trade_state
migrates in this commit per feedback_no_partial_refactor.
New 64-byte layout (existing 24-byte fields preserved at original offsets):
Offset Size Field
0 8 entry_ts_ns (u64)
8 4 entry_px_x100 (i32)
12 4 entry_size (i32)
16 4 realized_at_open (f32)
20 4 conviction_at_entry (f32) ← NEW
24 4 conviction_ema (f32) ← NEW
28 16 conviction_per_horizon_ema [4 × f32] ← NEW
44 4 pnl_adjusted_conviction_ema (f32) ← NEW
48 4 peak_unrealized_pnl (f32) ← NEW
52 4 degradation_consecutive_events (u32) ← NEW
56 4 disagreement_consecutive_events (u32) ← NEW
60 1 horizon_idx_dominant_at_entry (u8) ← NEW
61 3 pad
The 24-byte prefix is unchanged so downstream readers (stop_check_isv in
decision_policy.cu, max_hold event-rate check in resting_orders.cu) need
only update the stride constant. The new fields at offsets 20..63 are
populated by subsequent CRT.1 tasks (C1.2 multi-horizon conviction, C1.4
composite exit signal).
Open branch in pnl_track.cu zeros the new fields implicitly because
alloc_zeros on the device slot zero-initialises everything; subsequent
writes only touch the 0-23 byte range (existing fields). Close branch
reset-loop iterates OPEN_TRADE_STATE_BYTES which now zeros all 64.
Files changed:
crates/ml-backtesting/src/lob/mod.rs — pub const OPEN_TRADE_STATE_BYTES: usize = 64
crates/ml-backtesting/cuda/pnl_track.cu — #define OPEN_TRADE_STATE_BYTES 64
crates/ml-backtesting/cuda/resting_orders.cu — comment + stride literal 24→64
crates/ml-backtesting/tests/stop_controller.rs — open_trade_state_64_byte_layout test
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
fe24987690 |
fix(crt-a2.1): clamp conviction-EMA rescale to [0, 1] — never amplify weak signals
Gate 1 catastrophic failure (smoke vjmwc, commit
|
||
|
jgrusewski
|
1d889d2de9 |
arch(crt-a): Wiener-α conviction-EMA smoothing in decision_policy
After A1 (commit |
||
|
jgrusewski
|
045850e8f3 |
arch(crt-a): delete decision_stride field — greenfields atomic refactor
Per spec 2026-05-20-continuous-reasoning-trader-design.md §3.3 and §8:
decision_stride is REMOVED, not deprecated. No backwards-compat shim,
no fallback. Every consumer migrates in this commit per
feedback_no_partial_refactor.
Removed from:
- bin/fxt-backtest: RunArgs CLI flag, SweepBase field, SweepCell
override field, default_decision_stride() function, all three
BacktestHarnessConfig and RunArgs construction sites
- crates/ml-backtesting/src/harness.rs: BacktestHarnessConfig field,
MultiHorizonLoaderConfig decision_stride initializer, `let stride`
local, `if event_count % stride == 0` gate around
step_decision_with_latency; forward_step_into + step_decision now
share a single window-full guard (merged into one `if` block)
- crates/ml-alpha/src/data/loader.rs: MultiHorizonLoaderConfig field,
next_sequence stride logic simplified to stride=1 (consecutive
snapshots only)
- crates/ml-alpha/src/trainer/perception.rs: PerceptionTrainerConfig
field and Default impl; all four dt_s locals replaced with 1.0_f32
(training K-loop, graph-capture K-loop, forward_step_into CfC step,
eval K-loop)
- crates/ml-alpha/examples/alpha_train.rs: CLI flag, trainer_cfg and
both loader configs
- crates/ml/examples/alpha_baseline.rs: CLI flag, train + eval stride
gates replaced with unconditional read_all()
- config/ml/*.yaml: decision_stride: lines removed from
sweep_smoke, sweep_threshold_tuning, sweep_deployability,
sweep_decision_stride_example (file repurposed as generic example)
- tests: forward_step_golden, perception_overfit (×7 structs including
the stride=4 smoke repurposed as a second convergence check),
multi_horizon_loader (stride=4 spacing test repurposed as
ts_ns monotonicity check), ring3_replay, trainer_parity
Harness loop now invokes BOTH forward_step_into AND
step_decision_with_latency on every event whenever the snapshot window
is full. forward_step_into advances SSM state and writes alpha_probs_d;
step_decision_with_latency reads alpha_probs_d immediately after —
no CPU roundtrip, no stride gate.
n_decisions ≈ events_processed - seq_len + 1 after this commit
(vs ~9999 at stride=200 in the S2 baseline).
cargo check --workspace: clean
cargo test -p ml-backtesting --lib: 33 passed
cargo test -p ml-alpha --lib: 33 passed (6 ignored)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
92f8b10ed2 |
fix(crt-a): forward_step_into eliminates GPU↔CPU roundtrip + bit-identical seed
Corrective commit on top of
|
||
|
jgrusewski
|
a0e81fbdfc |
arch(crt-a): forward_step incremental SSM state — enables event-rate trunk forward
Per A0 investigation memo (commit
|
||
|
jgrusewski
|
8828e8ab13 |
fix(ml-backtesting): realize PnL on event-rate max_hold force-close
S2.2 moved max_hold to event-rate at step_resting_orders (cap is now
enforced — mean hold 432s → 58.43s, max 173893s → 96s). But it mirrored
the session-gap pattern `pos.position_lots = 0;` which intentionally
skips PnL ("cannot fill across halt"). Max_hold differs: the market is
live so the close SHOULD realize PnL via the current top-of-book.
Smoke t9msj (
|
||
|
jgrusewski
|
b92bd72c72 |
fix(ml-backtesting): move max_hold force-close from decision-rate to event-rate — actual cap enforcement
S2.1 instrumentation (smoke cqpph @
|
||
|
jgrusewski
|
95a77c4ac6 |
diag(ml-backtesting): max_hold enforcement counters localize why 86.5% of trades exceed the 60s cap
S2 cluster smoke ppcfk (
|
||
|
jgrusewski
|
29f7e923c6 |
fix(ml-backtesting): skip queue-decay fill for sentinel-priced market-like orders — ACTUAL root cause of vwap=0/huge sentinels
S1.20-S1.22 chased a wrong hypothesis (walk_* deep-level filter) for 3 rounds. The actual bug: resting_orders.cu:344 fills at cost = take * s.price in the queue-decay maturation arm. Market-like orders seed at line 644 with sentinel s.price = 1e9 (buy) / 0 (sell), counting on the marketability check at lines 372-... to fire first via walk_*. But queue_position initializes to 0 (no book level matches the sentinel), so on the first same-side trade volume the queue-decay arm fires and injects the sentinel into cost: buy cost = take * 1e9 -> avg_px = 1e9 -> huge_flat=216, sell cost = take * 0 -> avg_px = 0 -> zero_flat=194. Same pattern explains flip=9+6. Fix: detect s.price outside [min_reasonable_px, max_reasonable_px] in the queue-decay arm; absorb queue_position depletion without filling. Marketability check fires in the same iteration via walk_* with proper book prices. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
76ec68c1c9 |
fix(ml-backtesting): per-level price-range validation in walk_ask_for_buy/walk_bid_for_sell — eliminates sized-but-bad-priced sentinel propagation
v2 NaN instrumentation (S1.21) localized the bug to apply_fill_to_pos's open-from-flat branch writing pos.vwap_entry = avg_px where avg_px was 0 (194 cases) or > 21M finite (216 cases). The arithmetic was clean — root cause is walk_ask_for_buy/walk_bid_for_sell consuming size from deep levels (k=1..9) that have lvl_sz > 0 but lvl_px = 0 (or huge sentinel). MBP-10 fills empty depth slots beyond available levels with these sentinels. Existing per-level filter checked lvl_sz <= 0, !isfinite(lvl_sz), !isfinite(lvl_px) — but allowed lvl_px = 0 and lvl_px > max range. Fix: thread per-backtest min_reasonable_px / max_reasonable_px (already uploaded for the top-of-book skip in book_update_apply_snapshot via S1.19) through to walk_*, and reject any level whose price falls outside [min_px, max_px]. Same fix shape as Bug C-b (top-of-book skip), now applied at all 10 depths. Changes: resting_orders.cu (walk_* signatures + kernel param + 2 call sites), order_match.cu (walk_* signatures + kernel param + 1 call site), sim/mod.rs (submit_market + step_resting_orders launch args). 19 CUDA tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
fc41440dc9 |
diag(ml-backtesting): finer NaN instrumentation — per-vwap-write-site + last-bad-vwap capture
v1 instrumentation (S1.20) eliminated 3 of 6 hypotheses: apply_fill_to_pos arithmetic is fully clean (avg_px=0 realised=0 realized_pnl=0). But pnl_track open branch still saw vwap_entry=0 (194 times) or > 21M (216 times) at the open transition. v2 adds per-vwap-write-site counters so we can pinpoint which apply_fill branch produced the bad vwap, plus captures the actual last-bad-vwap value and the path id (1..6 = open-flat, scale-in zero/huge, flip-beyond zero/huge, session-gap-saw-stale). The 7th counter (vwap_session_gap_was_bad) fires when the session-gap force-close path sees pos.vwap_entry already in a corrupt state pre-gap. Logged at end of smoke as `nan_counters_v2: zero_flat=N huge_flat=N zero_scale=N huge_scale=N zero_flip=N huge_flip=N session_gap_was_bad=N | b0_last_bad_vwap=X b0_last_bad_path=N`. 19/19 stop_controller CUDA tests pass; 5/5 decision_floor_coldstart pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
025554afcd |
diag(ml-backtesting): kernel-side NaN instrumentation for residual sentinel root-cause
Cluster smoke 88dbp (post-Fix-S1.19 parameterized price range) still produces 97 zero + 85 i32::MAX sentinel entry_px values despite source data being fully filtered. Sentinels originate in kernel arithmetic paths post-sanitization, not from book input. Adds 6 per-backtest u32 counters incremented at NaN-producing sites: - nan_avg_px: avg_px = total_cost/filled_lots -> NaN/Inf - nan_realised: (avg_px - vwap_entry) * dir * unwind -> NaN/Inf - nan_realized_pnl: pos.realized_pnl becomes non-finite after += or -= - zero_vwap_at_open: pnl_track open branch saw vwap_entry == 0 - saturated_vwap_at_open: pnl_track open saw |vwap_entry| > 21M or NaN/Inf - defensive_exit_clamp: pnl_track close defensive clamp fired Each counter printed at end of smoke via nan_counters: log line. apply_fill_to_pos (resting_orders.cu) gains 3 counter args; NaN-producing paths return early WITHOUT propagating into pos state. pnl_track_step gains 3 counter args. All call sites threaded through sim/mod.rs launches. NanCounters struct + read_nan_counters() accessor added to LobSimCuda. Unit test nan_counters_initialize_to_zero confirms zero-init and accessor compile. 18/18 stop_controller, 5/5 decision_floor_coldstart pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
c03cf9aa38 |
fix(ml-backtesting): parameterized price-range sanitization replaces hardcoded 1e8
Audit (fxt-data-audit on ES.FUT_2024-Q1) revealed real source-data outliers that the hardcoded < 1e8 threshold didn't catch: - bid_min = -$4.85 (negative bid) - bid_p1 = $64.15 (1% of bids in sub-$100 range, far below ES) - ask_max = $53,012 (10x above any plausible ES price) The < 1e8 threshold = $100M was useless: never triggered on real ES. Fix: parameterize the range. min_reasonable_px / max_reasonable_px as per-backtest fields in UniformSimParams, ResolvedSimVariant, and BatchedSimConfig (defaults 0.0 / f32::INFINITY = no-check, preserves existing test fixtures). Sweep_smoke.yaml sets 1000/20000 for ES futures — catches all observed outliers without rejecting any plausible price. BacktestHarness calls upload_price_range() once after creating the sim so the bounds are active before the first apply_snapshot. CUDA kernel book_update_apply_snapshot gains two new args (min_reasonable_px[n_backtests], max_reasonable_px[n_backtests]) that replace the hardcoded > 0.0f && < 1.0e8f checks at both the top-of-book gate and the per-level sanitization pass. Test price_range_rejection_skips_snapshot validates: sub-$1000 snapshot, super-$20000 snapshot, and negative bid all skip with snapshots_skipped counter increment; valid ES snapshot passes through. 17/17 stop_controller tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
a85f38e97a |
fix(ml-backtesting): three residual sentinel + session-gap fixes
Three orthogonal fixes for residual issues in smoke stx9p (97 zero sentinels, 85 i32::MAX sentinels, 840/1024 over-60s holds): Fix 1 — zero-sentinel residue (px > 0): - Per-level sanitization in apply_snapshot_kernel only checked sz > 0. - A book level with px=0, sz>0 passed → walk_* computed total_cost=0 → avg_px=0 → vwap_entry=0. Trade record reported entry_px=0. - Add px > 0.0f to bid_ok/ask_ok conditions. Fix 2 — i32::MAX upper bound (px < 1e8): - Post-Bug-D (1e9 nanoprice scaling) some boundary events carried prices larger than any plausible instrument. Saturating cast to i32 produced the 21474836 sentinel even when isfinite() passed. - Add px < 1.0e8f upper bound to per-level AND top-of-book validation. Fix 3 — session-gap force-close: - max_hold check fires at decision_stride frequency, but during weekend halts no events advance current_ts → max_hold never fires until next session. Result: 49h holds in stx9p (840/1024 over 60s threshold). - Detect ts gap > 1 hour in resting_orders_step. If position is open, zero position_lots directly. pnl_track_step's existing close branch emits the TradeRecord on the next call. No synthetic P&L added — records show realised_pnl from whatever was accumulated before the gap (honest: cannot fill across a halt). - New per-backtest last_event_ts_d slot tracks the previous event ts. - Test session_gap_force_closes_open_positions: 2-hour ts jump after open verifies force-flat fires and exactly 1 TradeRecord is emitted. All 16 stop_controller tests pass. All 5 decision_floor_coldstart pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
5235b4515b |
fix(data,ml-backtesting): DBN nanoprice scaling + skip-on-corrupt-top-of-book
Two root-cause fixes surfaced by cluster smoke v74v4 (sweep_smoke-a2dfc6d99): Bug D — DBN parser price scaling: - BidAskPair::price_to_f64/price_from_f64 used /1e12 / *1e12 from test-data calibration. DBN production uses 1e-9 nanoprice (the DBN standard). ES at 5500 raw 5_500_000_000_000 → 5.5 instead of 5500. Smoke trade records showed entry_px=5.24 instead of expected ~5240 ES index points (1000× too small). Fix: 1e12 → 1e9 in both functions. Round-trip symmetric; tests updated. Bug C-b — corrupt top-of-book sentinel: - Per-level sanitization (Task 15) zeros each unhealthy MBP-10 level individually. At session-boundary events with all 10 levels invalid, the book becomes uniformly zero. apply_fill_to_pos then reads bid_px[0]=0 / ask_px[0]=0 → vwap_entry=0 → trade record entry_px=0 (zero sentinel in v74v4 CSV, 162/1024 trades in n59t4). - Fix: pre-validate top-of-book in apply_snapshot_kernel. If bid_px[0]/ask_px[0]/bid_sz[0]/ask_sz[0] are non-finite or bid_px[0]<=0/ask_px[0]<=0/bid_sz[0]<=0/ask_sz[0]<=0, atomically skip the entire snapshot (book/prev_mid/atr_mid_ema unchanged). Add per-backtest snapshots_skipped_d counter for observability. Test corrupt_top_of_book_skips_snapshot_and_increments_counter validates NaN top-price + zero top-size cases both increment the counter without mutating state, and that a subsequent valid snapshot updates normally. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
691dec144e |
fix(ml-backtesting): root-cause NaN/Inf book + duplicate close emissions
Two orthogonal bugs that compounded to produce the exit_px=±21474836 sentinel in cluster smokes (315 trades baseline, 500 trades pearl): Bug A — NaN/Inf book propagation: - apply_snapshot_kernel passes through NaN/Inf prices from MBP-10 predecoded data (session boundaries, gap-fills). - walk_ask_for_buy / walk_bid_for_sell accumulate cost += take * NaN. - apply_fill_to_pos: avg_px = NaN; realized_pnl += NaN → permanent NaN. - Subsequent close: (int)(NaN-derived * 5000) → i32::MAX sentinel. - Fix: zero-out non-finite levels in apply_snapshot_kernel; add isfinite() guards in walk helpers as defense-in-depth; ATR update guards against non-finite mid. Bug B — pnl_track close branch doesn't reset scratch: - Scratch reset (full 24-byte memset to 0) was already present in the current code; no source change required for Bug B. Tests: - book_nan_inf_prices_dont_corrupt_realized_pnl: NaN at ask[3] + Inf at bid[5] survives the fill pipeline without making realized_pnl non-finite. - pnl_track_resets_scratch_on_close: open+close+5 flat events emits exactly 1 record; second open+close emits exactly 1 more (=2 total). 14/14 stop_controller tests pass; all other ml-backtesting test suites unaffected. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
bf619a2e7b |
feat(ml-backtesting): max-hold + exit_px defensive + spec criteria revision
Three follow-ups from cluster smoke gp74n (trade_vol pearl validation): 1. Max-hold force-close: max_hold_ns added as per-backtest config (default 0 = disabled). Fires force-flat (3, 0) when current_ts - entry_ts >= max_hold_ns, BEFORE SL/trail check. Tested via max_hold_forces_close. Sweep YAML sets 60s cap to bound the long tail observed in gp74n (263985s pathological hold). 2. exit_px defensive sanity check: 500/1024 gp74n trades reported exit_px = ±i32::MAX/100 (float→int saturation sentinel from likely NaN segment_realized). Defensive fix in pnl_track_step: if exit_px is non-finite or diverges from entry_px by >10%, fall back to entry_px with zero realised_pnl. Root cause to be traced separately. 3. Spec §9.2 criteria revision: mean_hold<30s replaced with p95<600s + max<=max_hold_ns. The 30s threshold reflected the pre-pearl sub-cost churning bug, not a real feature criterion. p95 catches long-tail pathology while letting alpha-driven exit timing breathe. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
9b29f9fd0a |
feat(ml-backtesting): trade-vol floor replaces 2×cost literal in stop_check_isv
Replaces the Task 12 `2.0f * cost` floor (hardcoded multiplier) with trade_vol = sqrt(realised_return_var) bootstrapped from cost². Per pearl_trade_level_vol_for_stop_distance.md: microstructure ATR is the wrong time scale for trade-level stop decisions; per-horizon realised_return_var is the right one, with cost² as a structural cold- start sentinel. cost now appears exactly once — inside the sqrt as a bootstrap sentinel, never as a distance multiplier. The 2.0f literal is eliminated; controller is fully ISV-driven. var_avg accumulates realised_return_var in the same single-pass horizon loop as ema_loss/ema_win. Cold-start (var_avg=0): trade_vol = cost. Post-bootstrap: sqrt(var_avg) dominates. Test retargeted: cost_floor_prevents_sub_cost_stops → trade_vol_floor_prevents_sub_cost_stops, with boundaries straddling trade_vol=cost=0.125 instead of the prior 2*cost=0.25 (no-fire Δ=0.08, fire Δ=0.20). Spec §5, §10, §11, §12 amended. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
da6e887174 |
feat(ml-backtesting): cost-floor in stop_check_isv prevents sub-cost churning
Cluster smoke w7b4p showed 44864 closed trades in 1M events at 200ms latency — physically impossible without sub-cost churning. Root cause: sl_distance = max(pnl_ema_loss, atr) ≈ 0.05 at cold-start, but round-trip cost = 2 × 0.125 = 0.25. Every stop-out guaranteed loss > 5× distance; EMA converges sub-cost. Amendment: triple-max sl_distance = max(pnl_ema_loss, atr, 2*cost); trail_distance = max(pnl_ema_win, atr, 2*cost). cost is an ISV-discipline strategy anchor (already per-backtest from P4 — no new state slot, no tuned constants). Added cost_per_lot_per_side_per_b param to both decision kernel signatures and threaded cost_per_lot_per_side_d through both launch sites in step_decision_with_latency. Test: cost_floor_prevents_sub_cost_stops validates unrealized in (ATR, 0.25) does NOT fire SL; unrealized > 0.25 DOES fire. Spec amended (§10, §12). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |