d49003de6d1014653b0a26e11b754007f00a5611
2599 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
jgrusewski
|
d49003de6d |
feat(regime): vol_ref floor controller + cross-invocation disk persist
Closes the TrainingPersist loop for the regime defense per the KELLY_F_SMOOTH precedent (pearl_kelly_cap_signal_driven_floors). Three layers: (1) Per-step: alpha_regime_vol_update_kernel tracks min(vol_obs > 1e-12) in new slot 553 (REGIME_VOL_OBS_MIN_INDEX). Filtered against artifact- zero observations from stationary snapshots where curr == prev. (2) Per-cell: stacker_threshold_controller_update gains a fifth branch that reads vol_ref (slot 550) at cell-end, computes `target = floor_target_ratio × vol_ref`, slow-EMAs the floor anchor (slot 552) toward the target with rate `floor_update_rate` (0.1 per cell). Subfloor 1e-12 inside the kernel guards against the anchor collapsing to zero. (3) Per-invocation: alpha_baseline reads `config/ml/alpha_baseline_state.json` at startup and seeds slot 552 from the `regime_vol_ref_floor` field. At end of main(), the learned floor is written back via tmp+rename atomic write so concurrent walk-forward invocations see a consistent file. Matches the cross-fold-persistent shape of KELLY_F_SMOOTH. Block extended to 15 slots (539..=553). Smoke + kernel unit test pass -1/-1 for the new floor-controller indices (backward compat). Walk-forward CV verdict (Q1 fxcache, 3 sequential folds): iteration fold-A fold-B fold-C mean ± SD pre-defense (no regime) +91.52 -21.44 +46.74 +38.94 ± 56.88 hardcoded 1e-9 floor -19.77 +65.04 +6.45 +17.24 ± 43.42 learned floor (0.5 × cell_min) +74.78 -12.72 +8.80 +23.62 ± 45.59 learned floor (0.1 × vol_ref) -19.53 -26.51 +15.46 -10.20 ± 22.49 Controller infrastructure is structurally correct (loop closes, floor persists across invocations, kernel + disk + ISV all roundtrip). The TUNING is data-dependent — single-quarter CV doesn't have enough regime diversity to anchor the floor against. Multi-quarter fxcache validation is the next step (built cluster-side on the 9-quarter 2024-Q1..2026-Q1 ES futures dataset, downloaded as a single artifact for local CV). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
3c035ce1ae |
feat(regime): vol_ref bootstrap window + ISV-driven permanent floor
Two coupled fixes to the vol-EMA regime detector exposed by walk-forward CV after all features were promoted to always-on: (1) Bootstrap window for vol_ref (slot 551 = REGIME_VOL_REF_SAMPLES) - Replace the Pearl-A "first observation replaces directly" bootstrap with a running mean over the first N=100 vol_ema observations, then switch to β-tracking. For IID observations the running-mean estimator has variance σ²/N — a 100-sample mean is 10× less noisy than the single-shot replace. (2) Permanent floor on vol_ref (slot 552 = REGIME_VOL_REF_FLOOR) - The bootstrap alone exposed the asymmetric deadband-deadlock: if vol_ref converged to a tiny value during a calm initial stretch, vol_ref / vol_ema fired the moment any realistic vol resumed and Kelly stayed trapped at regime_scale_floor=0.25 forever. Floor lives in ISV slot (TrainingPersist) with a hardcoded 1e-12 sub-floor inside the kernel as numerical-underflow guard. - Host seeds slot 552 with 1e-9. Future controller kernel will refine this from observed cell-level vol minima with cross-fold persistence. Walk-forward CV on Q1 fxcache (3 folds, window=700K, train_frac=0.6): cost fold-A fold-B fold-C mean ± SD 0.00 -19.77 +65.04 (100%) +6.45 +17.24 ± 43.42 Fold B turnaround is the headline: -47.64 (bootstrap-only) → +65.04 (bootstrap + floor) confirms the floor is the load-bearing fix. Cross-fold std-dev compressed 24% at cost=0; mean dropped from +38.94 (pre-defense) to +17.24 (with defense). Classic mean/variance trade. Block extended to 14 slots (539..=552). Kernel sig: vol_ref_floor moved from f32 scalar to vol_ref_floor_index i32, so the anchor is named/addressable in ISV. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
34586dad68 |
refactor(alpha_baseline): rename, drop conditionals, strip dead paths
Rename binary alpha_compose_backtest → alpha_baseline and remove the boolean flags whose features are now mandatory: --c51 (always C51 distributional Q) --temporal (always Mamba2 temporal encoder) --isv-continual (controller always fires per eval episode) --regime-scale (vol-EMA regime defense always on) --pruned-actions (FALSIFIED 2026-05-15 per pearl_action_pruning_falsified) Every dependent code path was stripped, not just gated: - Linear-Q kernels (lq_fwd, lq_grad, munch_kernel) and their cubin loads are gone — C51 is the only Q-network. - Single-env push_kernel / h_store_kernel loads removed; the backtest has been batched-parallel-env since T14 and only the _batched variants are called here. (The smoke binary still uses single-env variants because one env per episode is its job.) - Dead transition buffers removed: states_dev, next_states_dev, actions_dev, rewards_dev, dones_dev, q_current_dev, q_next_dev, target_dev, single_state_dev, single_q_dev, probs_current_dev, probs_next_dev, m_dev, single_probs_dev, single-env state_pinned, action_pinned, window_tensor, h_enriched_buf_dev. - Dead constants and helpers: PRUNED_ACTIONS, N_WEIGHTS, N_BIASES, epsilon_greedy, epsilon_greedy_gated. End-to-end verification on the existing Q1 fxcache (rebuild was OOM locally; full multi-quarter validation is the next phase): cost=0.0000 best τ=0.250 Sharpe_ann=+36.83 win=0.984 trades/ep=83.3 cost=0.0625 best τ=0.250 Sharpe_ann=+38.53 win=0.996 trades/ep=83.2 cost=0.1250 best τ=0.250 Sharpe_ann=+38.37 win=0.994 trades/ep=83.3 cost=0.2500 best τ=0.250 Sharpe_ann=+34.24 win=0.990 trades/ep=85.6 cost=0.5000 best τ=0.250 Sharpe_ann=+31.83 win=0.946 trades/ep=84.8 Numbers track the prior T16-flag config (within stochastic noise), confirming the conditional-stripping was a pure simplification — no behavioral change, just a smaller, honester binary. Also updated: - scripts/alpha_pipeline.sh — A/B conditions collapse to fixed-cost vs cost-randomized training (the only opt-in left). - scripts/walk_forward_cv.sh — drop legacy flags, pass --window-k only. - crates/ml/src/env/loaders.rs — module doc-comment updated. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
d090685ca9 |
feat(phase-e-4-a-T16): vol-regime detection + cost-aware training
Adds two coupled interventions on the regime fragility exposed by
walk-forward CV (mean Sharpe +27 ± 56 at half-tick across 3 folds —
std-dev ≈ mean means the strategy is regime-dependent).
(1) Vol-EMA regime detector (new ISV slots 549/550)
- New alpha_regime_vol_update.cu kernel: per inference step, reads B
parallel-env mid prices, computes cross-env mean squared log-return,
and maintains ISV[549]=REGIME_VOL_EMA (Wiener-α with 0.4 floor +
Pearl A bootstrap) and ISV[550]=REGIME_VOL_REF (slow tracker β=0.005,
≈200-step horizon).
- Block-tree-reduce (no atomicAdd), guards against zero/non-finite mids.
(2) Pre-emptive Kelly attenuation (modified stacker controller)
- stacker_threshold_controller.cu takes 3 new args: regime_vol_ema_idx,
regime_vol_ref_idx, regime_scale_floor.
- Multiplies its reactive Sharpe-error Kelly output by
regime_scale = clamp(vol_ref / vol_ema, 0.25, 1.0)
- Disabled when indices = -1 (backward-compatible smoke + kernel test).
(3) Cost-aware training (--train-cost-hi)
- alpha_compose_backtest --train-cost-hi: when > --train-cost, each
training epoch samples cost ~ U[lo, hi] so the Q-network learns
cost-conservative behaviour across the realistic ES range.
(4) Wiring
- alpha_compose_backtest --regime-scale enables both per-step regime
kernel firing during eval AND the regime hookup in the per-episode
controller call. Mapped-pinned mids buffers, all compute device-side.
- ExecutionEnv exposes current_mid() so the host gather reads the
active snapshot mid per env without leaking the private cursor field.
Smoke + test sites pass -1/-1 for regime indices (backward compat).
Doc: docs/isv-slots.md ledger for slots 549/550.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
3aef276255 |
feat(phase-e-4-a): walk-forward CV via --data-start-offset
Adds a sliding-window walk-forward harness for the T10 backtest:
- New load_snapshots_from_fxcache_at(start_offset, ...) loader variant
reads bars [start_offset..start_offset+max_snapshots) from the fxcache.
Alpha-cache lookups use absolute bar indices, so the same
alpha_logits_cache.bin works across folds.
- New --data-start-offset CLI flag on alpha_compose_backtest.
- scripts/walk_forward_cv.sh runs 3 folds (window=700K, train_frac=0.6)
at offsets 0 / 600K / 1.2M, producing /tmp/cv_fold_{A,B,C}.json plus
an aggregated mean±stddev Sharpe table across folds.
Walk-forward result (alpha_logits_cache trained on bars 0..1.57M, so
fold C eval is fully past the stacker cut):
cost fold-A fold-B fold-C mean ± stddev
0.0000 +91.52 -21.44 +46.74 +38.94 ± 56.88
0.0625 +84.94 -27.97 +38.42 +31.79 ± 56.74
0.1250 +79.91 -31.22 +33.51 +27.40 ± 55.82
0.2500 +72.77 -45.41 +15.16 +14.17 ± 59.09
0.5000 +50.52 -59.82 -12.75 -7.35 ± 55.37
Fold B (mid-quarter, bars 600K..1.3M) is a disaster — win rate
collapses to 0-22% across all costs. Folds A and C succeed strongly.
Cross-fold SD ≈ mean, so the policy is regime-dependent and cannot
be reliably deployed without regime detection.
Mean Sharpe at half-tick (+27.40) is still ~7× the stateless
Phase 1d.4 baseline (-4.0), so the temporal encoder adds real value
on average — but the single-window +62 OOS celebrated earlier was
a cherry-picked favorable regime, not a deployment-ready result.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
3dc022b843 |
feat(phase-e-4-a): T10 Mamba2 backward chain + KC calibration
T10 wires the C51 → Mamba2 backward chain in both binaries:
- New alpha_train_window_store_batched_kernel captures per-step windows
for end-of-epoch re-forward (Mamba2 cache required for backward).
- Mamba2Block::backward_from_h_enriched lets the C51 grad-input feed
directly into Mamba2 backward, skipping the unused W_out projection.
- alpha_c51_grad_input → backward_from_h_enriched → Mamba2AdamW.step
closes the loop in alpha_dqn_h600_smoke and alpha_compose_backtest.
Smoke (--temporal --c51): all 4 KCs PASS. R_mean -6.3 → +4.2 vs
Phase E.3 close R_mean -4.7 (no-temporal). EARLY_Q_MOVEMENT
calibration (mamba2_snapshot + mamba2_weight_distance) lifts the
diagnostic from 0.0023 (head-only) to 0.0590 (head + encoder),
giving an honest learning signal when the encoder absorbs gradient.
Backtest (--c51 --temporal --window-k 16 --isv-continual):
cost=0.0000 best τ=0.250 Sharpe_ann=+34.56 (was +10.41 head-only,
-22.54 frozen-Mamba2)
cost=0.0625 best τ=0.250 Sharpe_ann=+33.22
cost=0.1250 best τ=0.250 Sharpe_ann=+30.85 (Phase 1d.4 baseline: -4.0)
cost=0.2500 best τ=0.250 Sharpe_ann=+27.73
cost=0.5000 best τ=0.250 Sharpe_ann=+15.68
Caveat: in-sample results; OOS gate next.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
0ab54dc8ef |
feat(phase-e-4-a): batched parallel-env TRAINING (greenfield)
T15: training rewrite mirroring T14 eval. N_par parallel envs lockstep H steps per epoch; ONE batched C51 update at B = N_par * H. Expected ~15× speedup vs sequential. - NEW kernel alpha_h_enriched_store_batched_kernel for batched h_enriched slot writes - Training section greenfielded: legacy sequential loop deleted - CLI flag --n-train-par (default 50) - Terminal next-state slot zeroed; done=1 at horizon masks Q_next contribution in Bellman projection — no terminal Mamba2 forward - docs/isv-slots.md updated per kernel-audit-doc hook Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
90c9d54454 |
feat(phase-e-4-a): batched parallel-env eval rewrite (greenfield)
The Phase E.4.A T14 backtest at B=1 with per-step stream.synchronize()
was running ~150μs/step × 9M steps = ~22 min — dominated by sync
overhead, not GPU compute. RTX 3050 Ti to L40S swap wouldn't help
(launch overhead is the bottleneck, not FLOPS).
Solution: batched parallel envs. N=cli.n_eval_episodes environments
run in LOCKSTEP per cell — ONE sync per step (instead of N syncs).
Expected ~30× speedup at N=500.
Changes:
1. ExecutionEnv snapshots → Arc<Vec<SnapshotRow>>
- new() wraps Vec into Arc internally (backward compat)
- new_arc() takes pre-existing Arc (for parallel envs)
- snapshots_arc() accessor for snapshot sharing
- 50MB × N memory duplication avoided
2. alpha_window_push_batched_kernel (NEW CUDA)
- Same chronological shift+insert semantics as single-env kernel
- Grid (state_dim_blocks, B, 1): one thread per (batch, feature)
- launcher: launch_alpha_window_push_batched
3. MappedI32 (per-binary) gains len param + read_all()
- smoke & backtest pass len=1 for existing single-int use
- backtest passes len=N for batched action readback
4. backtest binary eval loop GREENFIELDED
- Legacy sequential 'for ep in 0..N { for step in ... }' loop
body deleted entirely
- New: 'for step in 0..horizon' outer, lockstep over N envs
- Build N envs sharing snapshots_arc at cell start
- Per step: gather N states (CPU loop, <100μs for N=500) →
write to mapped-pinned [N, STATE_DIM] → push kernel B=N →
Mamba2 batched forward → C51 batched forward → Thompson
batched → ONE sync → read N actions → step N envs on CPU
- ISV-continual moved from per-episode to per-cell (single fire
with aggregate stats)
5. docs/isv-slots.md updated per kernel-audit hook
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
2feeeda8bb |
fix(phase-e-4-a): GPU kernel for h_enriched slot copy — eliminate per-step CPU roundtrip
The Phase E.4.A T8 wiring stored Mamba2's per-step cache.h_enriched
into h_enriched_buf_dev via a dtoh+htod sequence:
let h_host = stream.clone_dtoh(cache.h_enriched.cuda_data())?;
let mut buf_host = stream.clone_dtoh(&h_enriched_buf_dev)?; // <- whole buffer
for j in 0..hidden_dim { buf_host[slot_offset + j] = h_host[j]; }
stream.memcpy_htod(&buf_host, &mut h_enriched_buf_dev)?; // <- whole buffer
This violates feedback_cpu_is_read_only AND
feedback_no_htod_htoh_only_mapped_pinned. Worse, the buffer-wide
dtoh+htod every step is ~20K floats × 600 steps × 500 eps × 30 cells
= ~9M roundtrips totaling significant PCIe latency in the backtest.
Fix: new tiny CUDA kernel alpha_h_enriched_store_kernel in
alpha_window_push.cu (one thread per hidden-dim feature, writes
src[j] → buf[slot_offset + j]). Replaces the dtoh/htod sequence
in both smoke and backtest binaries.
Estimated speed-up at backtest scale: 3-6× on the temporal eval
path. Pure-GPU per-step inference restored — no synchronisation
points on the hot path.
docs/isv-slots.md updated per kernel-audit-doc hook.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
4d65ace625 |
feat(phase-e-4-a): mirror --temporal in backtest binary + T11 ISV-continual
Phase E.4.A Tasks 11+12: backtest binary gains the same
--temporal Mamba2 forward chain as the smoke binary, plus the
--isv-continual flag that fires the stacker-threshold controller
at the end of each eval episode (Pillar B).
Changes:
- Imports: ml_alpha::mamba2_block::{Mamba2Block, Mamba2BlockConfig},
ml_core::cuda_autograd::gpu_tensor::GpuTensor
- CLI flags: --temporal, --window-k, --mamba2-hidden-dim,
--mamba2-state-dim, --isv-continual
- Cubin loading: alpha_window_push + stacker_threshold_controller
- Q-net sizing: c51_input_dim = mamba2_hidden_dim when --c51 --temporal
- Buffers: window_tensor GpuTensor, h_enriched_buf_dev, isv_dev,
ctl_wiener_dev
- Training inference path: push + Mamba2 forward + h_enriched →
C51 forward (mirrors smoke binary)
- Training batched compute: terminal Mamba2 forward, h_enriched_buf
for current/next, c51_input_dim threading
- Eval inference path: push + Mamba2 forward + h_enriched → C51
forward + Thompson select (with scoped borrow guard)
- T11 ISV-continual: stacker-threshold controller fires at end of
each eval episode; ISV slot 543 (threshold), 545 (observed-rate),
546 (Kelly atten) update with realized rollout stats. Co-exists
with the τ-grid sweep (τ-grid still gates; ISV updates parallel
observation of "live deployment" behaviour).
- JSON output: new fields temporal, window_k, mamba2_hidden_dim,
mamba2_state_dim, isv_continual.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
75d67bd203 |
feat(phase-e-4-a): alpha_c51_grad_input kernel (T10 prerequisite)
Phase E.4.A Task 10 partial: adds the C51 gradient-w.r.t.-input
kernel needed to chain the C51 head's loss gradient back into the
Mamba2 temporal encoder.
Kernel signature: alpha_c51_grad_input_kernel reads probs[B, A, K],
m[B, K], actions[B], W[A*K, in_dim] and writes d_input[B, in_dim].
Math: dL/d_input[b,j] = Σ_k (p[b,a_taken,k] − m[b,k]) · W[a_taken*K+k, j]
(only the taken-action column of W contributes; restricted by the
sparse-over-actions C51 CE gradient structure).
Status: kernel + Rust launcher in place. Mamba2 backward NOT yet
wired in the smoke binary because ml_alpha::Mamba2Block::backward
takes d_logit [B, 1] (post-W_out scalar gradient), not
d_h_enriched [B, hidden_dim]. Two paths to complete:
1. Modify ml-alpha to expose backward_from_h_enriched(cache,
d_h_enriched) bypassing W_out.
2. Replicate the post-W_out backward sequence inline.
Both deferred pending backtest validation (T14): if frozen Mamba2
already lifts backtest Sharpe meaningfully, T10 becomes
optimisation rather than prerequisite. Smoke gate already passed
gate 1 (R_mean +3.9 vs C51-flat -1.1) with frozen weights.
docs/isv-slots.md updated.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
ed6f5588e1 |
feat(phase-e-4-a): wire Mamba2 forward in smoke --temporal path
Phase E.4.A Task 8: wire ml_alpha::Mamba2Block as the temporal
encoder before the C51 head when --temporal is set.
Architecture (--temporal):
state_pinned ──push─▶ window_tensor[1, K=16, in_dim=10]
│
▼ Mamba2Block::forward_train
h_enriched[1, hidden_dim=32]
│
▼ launch_alpha_c51_forward (input dim=32)
probs[1, 9, 51] ──▶ Thompson selector
Implementation:
- Mamba2Block constructed at startup with config (in_dim=10,
hidden_dim=32, state_dim=16, seq_len=K=16). Loaded from ml-alpha's
precompiled cubin.
- Per-step: window push (shift+insert), then forward_train returns
(logit, cache). We discard logit (ml-alpha's binary classifier head)
and use cache.h_enriched as the C51 input.
- Per-step h_enriched cached into h_enriched_buf_dev[(t)..t+hidden_dim].
- Batched training (end-of-episode): the C51 forward + grad use
h_enriched_buf_dev[0..ep_len*hidden] for the current state and
[hidden..(ep_len+1)*hidden] for next-state (1-step offset). Runs
one extra Mamba2 forward on the terminal window to populate slot
ep_len.
- C51 input dim (W shape) becomes mamba2_hidden_dim when --temporal,
STATE_DIM otherwise.
100-episode smoke verdict (vs C51-flat baseline):
R_mean ep 50: C51-flat -8.2 → --temporal +0.4 (+8.6)
R_mean ep 100: C51-flat +0.5 → --temporal +10.0 (+9.5)
rvr: +1.045 → +1.046 (unchanged)
Q_SPREAD: 23.9 → 12.6 (sharper distributions)
ACTION_ENTROPY: 1.42 → 1.49 (now PASSES 0.5×ln(9) threshold)
Note: Mamba2 weights are FROZEN at random Xavier init in this
commit — T10 (backward + AdamW step) lands next. The R_mean lift
above is from C51 learning over RANDOM temporal projections of the
window — random SSM acts as a feature-engineering reservoir.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
35dcb87709 |
refactor(phase-e-4-a): window push to shift+insert (chronological layout)
Switch alpha_window_push from circular-buffer-with-head_idx layout to shift+insert layout matching production mamba2_update_history. Slot 0 = oldest, slot K-1 = newest after each push, matching Mamba2Block's [B, K, in_dim] input contract directly (no reorder). Cost: O(K-1) shifts per state_dim feature per push. For K=16, state_dim=10: 10 threads × ~15 ops each = trivial. Kernel signature: drops head_idx, adds K. Test updated to verify chronological shift across 3 pushes into 4-slot buffer. docs/isv-slots.md updated per kernel-audit-doc hook. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
70df697328 |
feat(phase-e-4-a): wire sliding-window buffer in smoke (buffer-only)
Phase E.4.A Task 7: maintain a GPU-resident circular window buffer in the smoke binary's --temporal path. Per-step: 1. mapped-pinned state_pinned write (existing) 2. alpha_window_push_kernel writes state into window[head_idx] 3. head_idx = (head_idx + 1) % window_k 4. C51 forward proceeds against state_pinned (consumer of window wires in T8 — Mamba2 over the window) On episode reset: zero the buffer and reset head_idx so Mamba2 sees clean zero-context for the first window_k-1 steps. CLI: --temporal flag + --window-k (default 16, kernel max 32 per mamba2_alpha_kernel constraint). Validation: 100-episode smoke with --temporal produced bit-identical R_mean / rvr / kill-criteria values to the C51-flat baseline run — confirms buffer maintenance has zero side effect on the existing C51 path. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
6e86e5b435 |
feat(phase-e-4-a): alpha_window_push circular-buffer kernel + GPU test
Phase E.4.A Task 6: tiny CUDA kernel that pushes a state[state_dim] vector into slot `head_idx` of a circular window buffer [K, state_dim]. Host tracks head_idx and zeros buffer on episode reset. This is the GPU-side append primitive that the smoke binary's per-step inference will call (T7) before Mamba2 over the buffer (T8). GPU smoke (alpha_window_push_circular_writes_to_indexed_slot): writes state_a at slot 0, state_b at slot 2, verifies non-targeted slots stay zero. PASS. docs/isv-slots.md: documents kernel as slot-agnostic per kernel-audit-doc hook requirement. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
5d7d4fa3c6 |
feat(phase-e-4-a): add mbp10_dir param to fxcache loader (signature-only)
Phase E.4.A Task 4: extend load_snapshots_from_fxcache with `mbp10_dir: Option<&Path>`. When provided, the loader will peek MBP-10 by timestamp and populate SnapshotRow.bid_l[1..10]/ask_l[1..10] from real LOB depth — but the real-peek implementation lands in Task 5 follow-on. This commit: - introduces the parameter (callers pass None) - warns at runtime if mbp10_dir Some until T5 lands - enables downstream wiring of --use-real-depth + --mbp10-dir CLI flags in the smoke / backtest binaries T5 deferred: on ES futures the --real-spread experiment showed 76% of fxcache bars hit the 1-tick floor, so depth-from-MBP-10 likely won't move the needle for ES. Higher-leverage work (Mamba2 wiring) prioritised. T5 implementation reopens as a follow-on if E.4.A gates pass with synthesised depth. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
eb49e2a0f7 |
feat(alpha): Phase E.3 follow-up — C51 distributional Q + Thompson + L1-L10 depth + falsifications
C51 distributional Q-network with GPU Thompson selection borrowed
minimally from production (alpha_c51.cu: forward, project, grad,
expected_q, thompson_select kernels; ~260 lines). Uses Huber
negative-tail compression in projection per production
block_bellman_project_f. Action selection 100% GPU via mapped-pinned
i32 output + __threadfence_system + host volatile read (matches
gpu_training_guard MappedBuffer pattern).
Backtest result (2D sweep, 500 episodes per cell, 30 cells):
cost=0 C51 +10.41 vs linear-Q -15.72 (+26pt, BEATS Phase 1d.4
no-RL baseline +4.4 by 6pt)
cost=0.125 C51 -13.81 vs -29.17 (+15pt closes half-tick gap)
Win rate at cost=0 best τ: linear-Q 0.008 → C51 0.552.
Calibration hypothesis vindicated; documented in
memory/pearl_c51_thompson_closed_phase_e3_gap.md.
Also in this commit (Phase E.3 follow-up cleanup):
- --pruned-actions falsified (2.4× worse Sharpe). Documented in
memory/pearl_action_pruning_falsified.md.
- --real-spread falsified for ES futures (76% of bars at 1-tick floor).
- SnapshotRow bid_l/ask_l extended from [f32; 3] to [f32; 10].
L4-L10 synthesized in this commit; real MBP-10 peek lands in E.4.A T5.
- docs/isv-slots.md updated per kernel-audit-doc hook requirement.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
771936b768 |
feat(alpha): --train-threshold for backtest + Phase E.3 honest verdict
Phase E.3 follow-up. Adds --train-threshold to alpha_compose_backtest so
the Q-network can be trained against a FIXED gate (instead of just
applying the gate at eval). Default 0.39 = the equilibrium the smoke's
controller stabilized to at ep 200+ (alpha_dqn_h600_smoke gated run).
Smoke result (gated training, controller running):
ep 100: thresh=0.32 obs=0.226 R_mean=-5.5 atten=0.75
ep 200: thresh=0.38 obs=0.082 R_mean=-3.0 atten=0.50
ep 300: thresh=0.39 obs=0.081 R_mean=-3.2 atten=0.25
ep 1000: thresh=0.39 obs=0.039 R_mean=-4.7 atten=0.10
The controller CONVERGES cleanly to threshold ≈ 0.39 with observed
trade rate at/below the 0.08 target. rollout_R_mean drops from -19
(no-gate training) to -4.7 (gated training): 4× less loss per episode.
rvr stays at +1.045σ (unchanged). The closed-loop architecture works
end to end.
(Note: smoke verdict FAILs on ACTION_ENTROPY (0.68 < threshold 1.10).
This is the policy correctly Waiting 95%+ of the time — the kill
criterion was designed to catch "collapse to one bad action," but
collapse-to-Wait under a strong gate is the RIGHT behavior. Verdict
threshold is misaligned with the gated paradigm; not a regression.)
Backtest result with --train-threshold 0.39:
cost eval-gate only train+eval gated Δ
------ -------------- ---------------- ----
0.0000 -15.72 -17.06 -1.3
0.0625 -21.30 -22.91 -1.6
0.1250 -29.17 -31.26 -2.1
0.2500 -42.12 -36.68 +5.4
0.5000 -54.86 -53.83 +1.0
Training with the gate did NOT meaningfully improve absolute Sharpe.
The eval-best threshold remains 0.20-0.25 in BOTH runs (not 0.39).
The Q-network's primary contribution is the binary trade/don't-trade
decision; the action-choice (Buy direction + placement) is largely
determined by alpha sign — linear Q can't time entry better than the
threshold filter does on its own.
Honest analysis: the gap to Phase 1d.4 baseline (+4.4 at cost=0,
-4.0 at half-tick) is NOT architectural but ECONOMIC:
Env spread: bid/ask synthesized at ±0.125-tick around mid
→ round-trip spread cost = 0.25 per trade
At τ=0.20 with 168 trades/ep: 168 × 0.25 = 42 in spread costs
Mean reward = -5 → alpha extracts ~37 of value
All eaten by spread
Phase 1d.4 baseline likely trades much less (~20-50 trades/ep at best
operating point — pure threshold-only policy, no RL). Our policy
trades 3-8× more because the DQN's action choices add fine-grained
trade attempts beyond the threshold filter's wait/trade gate.
The control loop architecture (Phase E.1 + E.2 + E.3 gate consumption)
is VALIDATED — gate produces monotone Sharpe lift, +1.045σ rvr held,
trade-rate-self-correction converges cleanly. But beating Phase 1d.4's
absolute Sharpe requires:
1. MLP for the Q-network (more representation capacity for
entry-timing decisions within the alpha confidence band)
2. OR action-space constraints (collapse the 9-action space — drop
fine-grained L1/L2 placement, keep just {Wait, BuyMarket,
SellMarket, FlatMarket})
3. OR better fill economics (real LOB instead of fixed ±0.125-tick
synthesis)
These are Milestone E.3 follow-up work (Tasks 24-28 sweeps + future
architectural changes). The composition backtest validated what it
was designed to: the cost-edge frontier of the linear Q + Phase 1d.3
alpha + controller setup, and surfaced the next architectural
question (representation capacity vs action-space size vs fill
realism).
Branch: sp20-aux-h-fixed, pushed.
|
||
|
jgrusewski
|
a36ad53a57 |
feat(alpha): wire slot 543 consumption — 2D threshold × cost sweep
Phase E.3 Task 23 follow-up. Adds the confidence-threshold gate that
consumes the controller's ISV[543] output. Both binaries:
fn epsilon_greedy_gated(q, alpha_confidence, threshold, eps, rng) -> u8 {
if alpha_confidence < threshold { return 0; /* Wait */ }
epsilon_greedy(q, eps, rng)
}
State[1] is the env's alpha_confidence = |sigmoid(alpha_logit) - 0.5|
which is in [0, 0.5]; threshold is also clamped [0, 0.5], so direct
comparison is valid.
alpha_dqn_h600_smoke (closed-loop with controller):
Adds current_threshold: f32 cache, initialised to 0.0 (no gate),
refreshed via stream.clone_dtoh(&isv_dev) after each per-episode
controller invocation. Action selector reads current_threshold for
the NEXT episode's step decisions.
alpha_compose_backtest (2D sweep):
Adds --threshold-grid CLI flag (default [0.0, 0.05, 0.10, 0.15, 0.20,
0.25] — Phase 1d.4 pattern). Eval loop becomes 2D (threshold × cost).
Per-bin includes avg_n_trades for trade-rate visibility. End-of-run
prints BEST per-cost = max Sharpe_ann across τ.
Results (1000 train ep, 300 eval ep × 5 τ × 5 costs):
cost τ=0.00 best τ Sharpe lift trades/ep saved
------- ---------- --------- ----------- ---------------
0.0000 -41.78 -15.72 (τ=0.20) +26.1 477 → 168 (-65%)
0.0625 -71.46 -21.30 (τ=0.25) +50.2 476 → 138 (-71%)
0.1250 -86.78 -29.17 (τ=0.20) +57.6 482 → 167 (-65%)
0.2500 -108.57 -42.12 (τ=0.25) +66.5 480 → 132 (-73%)
0.5000 -146.76 -54.86 (τ=0.25) +91.9 478 → 136 (-72%)
Win rate at cost=0: 7.7% (no gate) → 20.3% (τ=0.20).
The gate architecture is VALIDATED: monotone improvement in win rate +
Sharpe + trade-rate reduction across all costs. The control loop
(controller → slot 543 → policy gate → observed rate feedback) is
sound. But the policy is STILL negative-Sharpe at every cost.
Phase 1d.4 baseline at half-tick: -4.0 (ours: -29.17). 25-pt gap.
Root cause of the remaining gap: the Q-network was TRAINED without
gate awareness. It learned Q-values for the over-trading regime. The
eval-only gate filters those decisions but can't fix miscalibrated
Q-values. Phase 1d.4 baseline beats us because its policy
(always-market-when-confident) is INHERENTLY gated by design — no
mismatched Q-values to fix.
Next iteration to close the 25-pt gap: train WITH gate on, so the
Q-network learns weights for the gated policy class. This means:
either (a) controller runs during training (smoke pattern) and the
threshold develops endogenously, or (b) fixed --train-threshold CLI
during training. Either way, the Q-network sees Wait-at-low-confidence
during the learning phase and adapts.
Files touched:
crates/ml/examples/alpha_dqn_h600_smoke.rs (gate + threshold cache)
crates/ml/examples/alpha_compose_backtest.rs (gate + 2D sweep)
config/ml/alpha_compose_backtest.json (2D verdict)
|
||
|
jgrusewski
|
2af8e02fd8 |
feat(alpha): Phase E.3 composition backtest — reveals slot 543 needs consumption
Phase E.3 Task 23. Trains the Phase E execution-policy DQN on the first
80% of fxcache snapshots, then evaluates the frozen policy (ε=0) on the
held-out 20% across a transaction-cost sweep. Compares absolute Sharpe
vs the Phase 1d.4 always-market-when-confident baseline.
Pipeline pieces:
- Shared loaders extracted into crates/ml/src/env/loaders.rs (used by
both alpha_dqn_h600_smoke and alpha_compose_backtest)
- alpha_compose_backtest.rs: train DQN on first n_train bars, then
frozen-eval n_eval episodes per cost level
- cost grid: [0.0, 0.0625, 0.125, 0.25, 0.5] (price units per
contract round-turn)
- Annualised Sharpe via per-episode Sharpe × sqrt(episodes/year)
where episodes/year ≈ 252 · 6.5h · 3600s / (horizon · 12s)
Run (horizon=600, 1000 train ep, 500 eval ep/cost, 1.5M snapshots):
cost n_ep mean_R std_R Sharpe/ep Sharpe_ann win_rate
0.0000 500 -11.09 8.03 -1.380 -39.50 0.090
0.0625 500 -20.68 9.88 -2.093 -59.89 0.012
0.1250 500 -29.38 9.49 -3.095 -88.58 0.000
0.2500 500 -48.23 11.90 -4.052 -115.98 0.000
0.5000 500 -84.59 17.43 -4.854 -138.92 0.000
Phase 1d.4 baseline for comparison: +4.4 ann. at cost=0, -4.0 at half-tick.
The Phase E policy LOSES MONEY across the whole cost grid — even at
frictionless cost=0. This is not a contradiction with the H=600 PASS
verdict (rvr=+1.04σ): the smoke's rvr is RELATIVE TO RANDOM, while
backtest Sharpe is ABSOLUTE. "Better than random by 1 std" is still
losing if random loses big.
The diagnostic that the E.2 controller already surfaced:
ISV[543] STACKER_THRESHOLD saturated at upper clamp (0.5) — policy
trades 85% of the time vs the 8% target. Over-trading pays spread on
every bar regardless of alpha confidence. Even with perfect alpha
(Phase 1d.3 AUC=0.673), trading 85% × spread cost > alpha edge.
The Phase 1d.4 baseline beats us at cost=0 because it WAITS unless
|stacker_logit| > threshold — the threshold gate filters bars with
weak alpha signal. The Phase E controller PRODUCES slot 543 but the
DQN's action selection doesn't CONSUME it.
This is exactly what the E.3 backtest is FOR: revealing that the
Phase E.1/E.2 producer-side architecture without consumer-side gating
is incomplete. The composition backtest validates the architecture's
weak link.
NEXT (E.3 task 24-28 or a side fix): wire slot 543 consumption into
the action selection. At each step:
if |ISV[543] − 0.5| > |stacker_logit − 0.5|:
action = Wait // confidence below threshold, sit out
else:
action = argmax(Q)
Or equivalently: action = if confidence_high(alpha_logit, ISV[543])
{ argmax(Q) over Buy/Sell actions } else { Wait }.
Once slot 543 is consumed, re-run alpha_compose_backtest and expect
Sharpe to move toward / past the Phase 1d.4 baseline.
Loader refactor: extracted load_fill_model_from_json, load_alpha_cache,
load_snapshots_from_fxcache from alpha_dqn_h600_smoke.rs into
crates/ml/src/env/loaders.rs. The smoke now calls the shared module
via ml::env::loaders::*. ~150 lines of duplicated code removed.
Build + run verified: smoke still builds clean. Backtest runs in ~30s
(train 8s + eval 20s + setup).
Branch: sp20-aux-h-fixed, pushed.
|
||
|
jgrusewski
|
91383507fc |
feat(alpha): wire stacker-threshold controller into smoke rollout-end
Phase E.2 Task 17. Loads stacker_threshold_controller.cubin at smoke
startup, initialises ISV[544] (TRADE_RATE_TARGET) to 0.08 (CLI flag
--trade-rate-target, never reset), allocates a 3-float Wiener state
buffer for slot 545's Pearl A+D state.
Invokes the controller at every episode end with:
rollout_trade_count = count of non-Wait actions in the episode
rollout_total_decisions = actions_host.len() (= ep_len)
rollout_realized_sharpe = ep_terminal_R / RANDOM_BASELINE_STD
(per-rollout analog of the rvr metric;
lets the Kelly-atten controller respond
to in-policy performance vs the baseline
noise floor)
CLI args added:
--trade-rate-target default 0.08 (8% per-step trade rate target)
--k-threshold default 0.01
--k-atten default 0.005
--target-sharpe default 0.5
--wiener-alpha-floor default 0.4
--ctl-alpha-meta default 0.1
Periodic log line extended:
ep ... | KC q/H/rvr/ΔQ ... | CTL thresh=... obs=... atten=...
Final JSON adds:
final_stacker_threshold
final_trade_rate_observed_ema
final_stacker_kelly_attenuation
trade_rate_target
Smoke run (H=600, 1000 episodes) verifies the controller is alive:
ISV[543] STACKER_THRESHOLD: 0.000 → 0.5000 (saturated at ceiling)
ISV[545] TRADE_RATE_OBSERVED_EMA: 0.000 → 0.712
ISV[546] STACKER_KELLY_ATTENUATION:0.000 → 0.100 (hit floor)
Verdict: PASS — rvr=+1.043σ (unchanged from Task 12b PASS, expected
since smoke doesn't yet CONSUME slots 543/546).
Tuning notes (calibration for production, not bugs):
• Threshold saturating at 0.5 → policy trades ~85% (target 8%, off by
10×). Either re-calibrate target_trade_rate from realistic backtest
behaviour, or raise the clamp ceiling. Current ε-greedy with low
threshold-consumption gate produces high trade rate.
• Kelly atten hit floor (0.1) because rollout_sharpe (~-0.004) is far
below target_sharpe=0.5. The target needs to match the rollout
metric's scale, OR the metric should be time-normalised. The
current ep_terminal_R / baseline_std proxy is meaningful but its
scale doesn't match a typical annualised Sharpe target.
These tuning items don't gate Milestone E.2 — the producer-side
controller is correctly driving the ISV slots; *consuming* those slots
(threshold gate on alpha signal, Kelly-cap multiplier) is Phase E.3
work (alpha + execution composition).
Phase E.2 Tasks 16 + 17 close-out: kernel + launcher + GPU smoke test
+ wired into smoke binary + initialisation + verified end-to-end. Tasks
19-22 (NoisyNet) are gated on Task 12 FAIL, which we passed — skipped.
Task 18 (alpha-trust ablation, ~9-18 hours compute) deferred to a
dedicated session if needed.
|
||
|
jgrusewski
|
85792ed28a |
feat(alpha): stacker-threshold + Kelly-attenuation controller kernel
Phase E.2 Task 16. Engagement-rate self-correcting controller per
pearl_engagement_rate_self_correction. Single-block, single-thread
kernel; runs once per rollout-end boundary.
ISV slots driven:
543 STACKER_THRESHOLD_INDEX clamp [0, 0.5] P-controller on rate
545 TRADE_RATE_OBSERVED_EMA_INDEX Pearl A+D floored Wiener-α
546 STACKER_KELLY_ATTENUATION_INX clamp [0.1, 1.0] P-controller on Sharpe
Reads ISV[544] TRADE_RATE_TARGET_INDEX (TrainingPersist anchor, set once
at training start).
Control law:
observed = trade_count / max(decisions, 1)
ISV[545] ← Pearl_A+D_floored(observed, prev, x_lag)
[α* floor = 0.4 per pearl_wiener_alpha_floor_for_nonstationary;
controller co-adapts with policy → need responsive EMA]
err_rate = ISV[545] - ISV[544]
ISV[543] ← clamp(0, 0.5, ISV[543] + k_threshold · err_rate)
err_sharpe = rollout_sharpe - target_sharpe
ISV[546] ← clamp(0.1, 1.0, prev_atten + k_atten · err_sharpe)
where prev_atten = 1.0 if ISV[546] == 0.0 (sentinel-start)
else ISV[546]
Wiener-α is INLINE (not via canonical apply_pearls_ad_kernel chain)
because the EMA is part of the control loop, not a separate diagnostic
slot. Lower latency, fewer kernels per step.
Floor at 0.1 on Kelly attenuation per
pearl_blend_formulas_must_have_permanent_floor — can't be 0, would
zero out position sizing permanently.
Pub launcher `launch_stacker_threshold_controller` in alpha_kernels.rs
with full safety asserts. Slot indices passed as i32 args (decouple
slot numbering from kernel).
GPU smoke test `stacker_threshold_controller_smoke_matches_hand_
computation` verifies 2-iteration sequence:
Iter 1 (Pearl A): observed=0.30 → ISV[545]=0.30; ISV[543]: 0.05 → 0.052
Iter 2 (Pearl D): observed=0.05 → ISV[545]=0.175 (α* hit floor 0.5)
ISV[543]: 0.052 → 0.05275
Both within 1e-5 tolerance. Anchor slot 544 unchanged.
`cargo test -p ml --lib alpha_kernels`: 6 pass (compile witness + 5 GPU
smokes including this one) on RTX 3050 Ti in 2.18s.
Audit doc docs/isv-slots.md updated per Invariant 7.
|
||
|
jgrusewski
|
5c0bcb1fdb |
fix(alpha): MBP-10 parser full-levels copy + fit_poisson L2 regularization
Two carried-over limitations from Phase E.0 / E.1 fixed and verified.
1. MBP-10 parser bug fix (`parse_mbp10_streaming` + `parse_mbp10_file`)
The DBN crate's `Mbp10Msg` carries the FULL post-update top-10 book
in `levels: [BidAskPair; 10]` per message — not just the single
update event's price/size. Previously the parser only called
`update_level(0, ...)` with the update event's fields, leaving
`current_snapshot.levels[1..10]` at default-empty. Downstream:
- OFI calculator reading L2-L5 got zeros → produced wrong OFI
features (the canonical Phase 1c/1d 81-dim feature stack has
multi-level OFI as features 0..5; with the bug these were
constant zero).
- microprice (`snapshot.levels[1]`) got zeros.
- FillModel L2/L3 fit observations got zeros, so L2/L3
coefficients were undefined (we worked around by replicating
L1 with attenuated intercept).
Fix: after `update_level(0, ...)`, copy fields from
`mbp10.levels[lvl]` into `current_snapshot.levels[lvl]` for `lvl
in 1..max_lvl`. Field-by-field copy preserves the existing scale
convention (raw 1e9 fixed-point i64). Applied to both streaming
and async file-parse code paths.
Comment "For simplicity, store all updates in level 0 / A full
implementation would maintain proper level ordering" removed.
2. fit_poisson L2 regularization
New `fit_poisson_l2(features, observed, max_iters, lr, l2_lambda)`
API (the old `fit_poisson` delegates with l2_lambda=0). L2 penalty
applies to slope coefficients β[1..5] but NOT to intercept β[0]
(penalizing the intercept biases toward p≈0.5 for all-zero-feature
samples, breaking the recovery test). Per-iteration update:
β[0] -= lr · grad[0] / n (intercept)
β[k] -= lr · (grad[k] / n + λ · β[k]) (slope, k ∈ 1..5)
Canonical motivation: on real 5.2M-trade ES.FUT data the
unregularized fitter converged to β_spread ≈ -40 (Task 5c commit
|
||
|
jgrusewski
|
cd5aa3402b |
feat(alpha): wire Phase 1d.3 stacker into smoke — H=600 VERDICT PASS
Phase E.1 Task 12b complete. The H=600 DQN smoke now consumes real
alpha_logit from the Phase 1d.3 stacker (Mamba2 + 7-input MLP stacker
trained for AUC=0.673 on test), and PASSES all four kill criteria:
Q_SPREAD_EMA = 10.92 ≥ 0.05 PASS
ACTION_ENTROPY_EMA = 1.97 ≥ 1.099 PASS
RETURN_VS_RANDOM_EMA = +1.043 ≥ 0.0 PASS ← jumped +3.62σ
EARLY_Q_MOVEMENT_EMA = 0.099 ≥ 0.01 PASS
Overall: PASS (H=6000 scale-up VIABLE)
Before/after comparison (same env, same DQN, only alpha_logit changed):
alpha_logit=0 alpha_logit=Phase1d.3
rollout_R_mean (final) -18,272 -18
RETURN_VS_RANDOM_EMA -2.58σ +1.04σ
Overall verdict FAIL PASS
The 1000× reduction in episode loss + the +3.62σ rvr swing definitively
proves the "first-best-action lock-in" hypothesis from the previous FAIL
analysis was a SYMPTOM, not the cause. The cause was alpha_logit=0
placeholder starving the policy of directional signal. With real Phase
1d.3 alpha, the linear Q-network learns to use it cleanly — no
NoisyNet, no MLP, no architectural change needed.
Integration pieces in this commit:
1. Cargo workspace registration: ml-alpha added as a workspace dep,
ml's manifest now depends on ml-alpha for FxCacheReader access.
(ml-alpha already depends only on ml-core, so no circular risk.)
2. alpha_dqn_h600_smoke.rs: two new CLI args
--fxcache-path <PATH> load snapshots from precomputed fxcache
(mid from raw_close, bid/ask synthesized
at fixed half-tick, 81-dim features extracted
for spread_bps / l1_imbalance / ofi / mid_drift)
--alpha-cache <PATH> load Phase 1d.3 stacker logit cache produced
by `alpha_train_stacker --alpha-cache-out`.
Each cache entry aligns to the corresponding
fxcache bar, populates SnapshotRow.alpha_logit
(and derives alpha_confidence = |sigmoid(z)-0.5|).
3. Snapshot source selection: in main(), --fxcache-path takes priority
when both paths are set; --alpha-cache requires --fxcache-path
(alignment guarantee). Original --mbp10-dir path unchanged for
non-cached runs.
4. Two new helper fns: load_alpha_cache (binary [u32 n] + [f32; n]
reader), load_snapshots_from_fxcache (FxCacheReader → Vec<SnapshotRow>
with synthesized bid/ask and alpha_logit/alpha_confidence from cache).
alpha_logits_cache.bin (7.6 MB, 1.97M f32 entries) is .gitignore'd —
regenerable from `cargo run -p ml-alpha --release --example
alpha_train_stacker -- --fxcache-path <FXC> --alpha-cache-out
config/ml/alpha_logits_cache.bin` (~2 min on RTX 3050 Ti).
Reproduction of this PASS verdict:
cargo run -p ml --release --example alpha_dqn_h600_smoke -- \
--fxcache-path /home/jgrusewski/Work/foxhunt/test_data/feature-cache/9297....fxcache \
--alpha-cache config/ml/alpha_logits_cache.bin \
--horizon 600 --n-episodes 1000
Total run time ~10s after fxcache load. Verdict + per-checkpoint KC
trajectory in config/ml/alpha_dqn_h600_smoke.json.
NEXT: Task 13 — scale to H=6000 (the production horizon). Per the plan,
PASS at H=600 unlocks H=6000.
|
||
|
jgrusewski
|
8958637c77 |
feat(alpha): stabilize alpha_dqn_h600_smoke — reward norm + target net + grad clip
Three stabilizers applied to the H=600 DQN smoke after initial run showed
unstable training (early_mvmt=2268× at lr=1e-6, NaN at lr=1e-4):
1. Reward normalization (--reward-scale, default 1000)
Rewards divided by scale BEFORE the Munchausen target. TD error
drops from ~1000 (raw reward magnitude at H=600) into O(1) target /
gradient / weight-update scale. Action selection + rollout-R
reporting use ORIGINAL rewards (so rvr math stays correct against
the Task 7c baseline).
2. Target network (--target-update-every, default 10 episodes)
Separate w_target_dev / b_target_dev buffers. Q_next(s') forward
uses target weights; SGD updates online only. Hard-update copies
online → target every K episodes. Breaks the V_soft(s') chase-its-
own-tail divergence of online-only Munchausen.
3. Gradient clipping (--grad-clip, default 1.0)
New `alpha_clip_inplace_kernel` in alpha_linear_q.cu (element-wise
clamp). Applied to dW and db after grad, before SGD. Safety net.
Diagnostic fix: weight_norm was direction-insensitive — orthogonal
rotations don't change ||W||_F, so early_mvmt read ≈0 even when training.
Switched to weight_distance_from_init = ||W_now − W_init||_F +
||b_now − b_init||_F (captures rotation). q_early = q_init + distance
so kernel's |q_early − q_init| / |q_init| ratio = distance / ||W_init||_F.
With lr bumped back up to 1e-4 (default for the stabilized config),
verified at horizon=100, n_episodes=200:
Q_SPREAD_EMA = 23.64 (≥ 0.05) PASS
ACTION_ENTROPY_EMA = 1.86 (≥ 1.0986) PASS
RETURN_VS_RANDOM_EMA = +0.586 (≥ 0.0) PASS
EARLY_Q_MOVEMENT_EMA = 0.0212 (≥ 0.01) PASS
Overall: PASS (H=6000 scale-up VIABLE)
early_mvmt grew monotonically (0.005 → 0.021) across the 200-episode
run — direction-sensitive diagnostic confirms genuine policy learning.
Audit doc docs/isv-slots.md updated per Invariant 7.
Next: H=600 / 1000-episode run on full data; if PASS holds, Task 13
(H=6000 scale-up) unlocks.
|
||
|
jgrusewski
|
fa30c2dd66 |
feat(alpha): alpha_dqn_h600_smoke — runnable Task 12 DQN smoke
Phase E.1 Task 12. Linear Q-network (W [9×10] + b [9], no hidden layer)
trained with ε-greedy + Munchausen target on the Phase E ExecutionEnv.
End-to-end runnable: load env, train, periodically launch
alpha_kill_criteria + apply_pearls_ad chain at episode boundaries, emit
PASS/FAIL verdict against the 4 kill criteria thresholds.
Pipeline per training step (all on GPU):
1. forward Q_current on s_batch via alpha_linear_q_forward
2. forward Q_next on s'_batch via alpha_linear_q_forward
3. alpha_munchausen_target → targets[batch]
4. alpha_linear_q_grad → dW, db (sparse over taken actions)
5. alpha_linear_q_sgd_step on W and b (separate launches)
6. every K episodes: kill_criteria + apply_pearls_ad chain → ISV[539..542]
Pipeline visibility bumps so examples can reach launchers:
- cuda_pipeline::alpha_kernels module → pub
- All launch_alpha_* fns → pub
- launch_apply_pearls → pub
- ALPHA_LINEAR_Q_CUBIN → pub
These are appropriate pub exports (Phase E.1 public API surface).
Initial micro-smoke (horizon=100, n_episodes=50, lr=1e-6):
Q_SPREAD_EMA = 3.12 (≥ 0.05) PASS
ACTION_ENTROPY_EMA = 2.12 (≥ 1.0986) PASS
RETURN_VS_RANDOM_EMA = +1.03 (≥ 0.0) PASS
EARLY_Q_MOVEMENT_EMA = 2268 (≥ 0.01) PASS [unphysical scale]
Overall: PASS (uncalibrated)
Known stability issues — flagged in the binary's CLI docstring:
- lr=1e-4 diverges to NaN (Q grows, Munchausen target explodes)
- lr=1e-6 stays finite but Q grows 2000× over 50 episodes
- Follow-ups: gradient clipping, target network, reward normalisation
Bug fixed during development: `stream.memcpy_htod(&host, &mut buf.clone())`
was uploading to a TEMPORARY clone (dropped immediately) — `kc_scalar_dev`
and `kc_action_counts_dev` never got their host data → entropy=0, early_mvmt=0,
rvr stuck at the alloc-zeros default. Fixed by removing `.clone()` and using
direct `&mut` refs.
Reads:
config/ml/alpha_fill_coeffs.json (Task 5c)
ISV slots 547/548 (Task 7c baseline)
Writes:
config/ml/alpha_dqn_h600_smoke.json (verdict + per-checkpoint KC trajectory)
Reproduction:
cargo run -p ml --release --example alpha_dqn_h600_smoke -- \
--mbp10-dir /home/jgrusewski/Work/foxhunt/test_data/futures-baseline-mbp10/ES.FUT \
--horizon 600 --n-episodes 1000
Audit doc docs/isv-slots.md updated per Invariant 7.
|
||
|
jgrusewski
|
36ab50814e |
feat(alpha): alpha_linear_q kernels + launchers for Task 12 DQN smoke
Phase E.1 Task 12a. Three new CUDA kernels for the H=600 DQN smoke
(Task 12 proper) that lands in a follow-up commit:
alpha_linear_q_forward_kernel Q = X · W^T + b
alpha_linear_q_grad_kernel dW, db sparse MSE-TD over taken actions
alpha_linear_q_sgd_step_kernel element-wise params -= lr · grad
Architecture: single linear layer, no hidden layer. The Phase E state
vector has meaningful direct features (alpha_logit, spread_bps, position,
ofi_sum_5, …) so linear Q can capture real relations like Q[Buy] ∝
alpha_logit. If linear can't pass the kill-criteria gate, no architecture
upgrade will save it — and the smoke proceeds with NoisyNet escalation
per the plan.
Sparse gradient: only the taken action contributes (standard DQN TD
loss). No atomicAdd needed — one thread per (i, j) loops over the batch
and adds only when actions[b] == i.
GPU contract:
- No host branches inside any kernel (graph-capture compatible)
- No atomicAdd (per feedback_no_atomicadd)
- All compute on GPU (forward, grad, weight update)
- Tiny launch overhead — fits per-step (batch=64 forward = 576 threads,
1 block; grad = 99 threads, 1 block)
Three pub(crate) Rust launchers in alpha_kernels.rs match the
launch_apply_pearls pattern. Cubin embedded via include_bytes!.
Smoke test `linear_q_forward_grad_sgd_round_trip_matches_hand_math`
exercises all three kernels end-to-end on a small (batch=2, state_dim=2,
n_actions=3) case with full hand-math:
Forward: Q = [[2.1, 3.2, 0.3], [4.1, 5.2, 0.3]] ✓
Grad: dW = [[-1.8, -2.7], [0.8, 1.0], [0, 0]]
db = [-0.9, 0.2, 0] ✓
SGD: W' = [[1.18, 0.27], [-0.08, 0.90], [0, 0]]
b' = [0.19, 0.18, 0.30] ✓
All within 1e-4 tolerance. `cargo test -p ml --lib alpha_kernels`:
5/5 pass on RTX 3050 Ti in 2.04s (compile witness + 4 GPU smokes).
Audit doc docs/isv-slots.md updated per Invariant 7.
|
||
|
jgrusewski
|
ebbd28437a |
test(alpha): chained pipeline smoke — Task 12 wiring validation
End-to-end test for the kernel COMPOSITION that the H=600 DQN smoke
(Task 12 proper) will use at each rollout boundary:
t+0 alpha_kill_criteria_compute_kernel → scratch[0..4]
t+1 apply_pearls_ad_kernel(n_slots=4) → ISV[539..543]
The two prior alpha_kernels smokes (munchausen + kill_criteria) validated
kernels in isolation. This smoke validates the COMPOSITION on the same
stream — failures here are different (stream-ordering, Pearls index base,
Wiener offset base, scratch visibility) and would silently break Task 12.
Two iterations with stationary synthetic inputs:
Iter 1 (Pearl A bootstrap)
prev_x_mean=0 AND x_lag=0 → ISV[539..542] populated with raw scratch
observations = [0.2041, 0.8980, 1.0670, 0.1] within 0.01 tolerance.
Anchor slots 547/548 remain at Task 7c values (-5185, 4953).
Iter 2 (Pearl D stationary)
dx_mean = dx_step = 0 → α* = 0 → ISV unchanged from iter 1 within
1e-4. Stationary signal stays at the bootstrap value.
Test would catch:
- Producer's scratch write not visible to applicator (stream-ordering)
- Wrong Pearls isv_idx_base / wiener_offset_base
- Pearl A sentinel detection broken (formula yields 0 at t=0)
- Wiener state corruption (iter 2 drifts from iter 1)
Reuses launch_apply_pearls from sp4_wiener_ema.rs (pub(crate)). Helper
fn run_chained_iter factors the producer→applicator sequence so the two
iterations are byte-identical apart from the Wiener state's evolution.
`cargo test -p ml --lib alpha_kernels`: 4 pass (compile witness + 2 prior
GPU smokes + this chained smoke) on RTX 3050 Ti in 1.89s total. Audit doc
docs/isv-slots.md updated per Invariant 7.
Remaining Task 12 work (full H=600 DQN trainer integration with this
pipeline at rollout boundaries) is queued for a dedicated session.
|
||
|
jgrusewski
|
697bb586d0 |
test(alpha): kill-criteria GPU smoke matching hand-computed observations
Adds the second of the two Phase E.1 kernel smoke tests in
alpha_kernels.rs (companion to the munchausen_target smoke from
|
||
|
jgrusewski
|
91d1a52b9c |
refactor(alpha): rename phase_e_* → alpha_* — system-scoped naming
The kill-criteria producer, Munchausen target kernel, Rust launchers,
fit/baseline binaries, and their output JSON artifacts are *durable
infrastructure* of the alpha trading system (live across Phase E/F/G/...),
not milestone-scoped to Phase E specifically. Aligns with the earlier
`phase_e_isv_slots.rs` → `alpha_isv_slots.rs` rename rationale.
What was renamed:
Code files:
crates/ml/src/cuda_pipeline/phase_e_kill_criteria.cu → alpha_kill_criteria.cu
crates/ml/src/cuda_pipeline/phase_e_munchausen_target.cu → alpha_munchausen_target.cu
crates/ml/src/cuda_pipeline/phase_e_kernels.rs → alpha_kernels.rs
crates/ml/examples/phase_e_fit_fill_model.rs → alpha_fit_fill_model.rs
crates/ml/examples/phase_e_random_baseline.rs → alpha_random_baseline.rs
Artifacts:
config/ml/phase_e_fill_coeffs.json → alpha_fill_coeffs.json
config/ml/phase_e_random_baseline.json → alpha_random_baseline.json
Kernel function names:
phase_e_kill_criteria_compute_kernel → alpha_kill_criteria_compute_kernel
phase_e_munchausen_target_kernel → alpha_munchausen_target_kernel
Rust launcher names:
launch_phase_e_kill_criteria → launch_alpha_kill_criteria
launch_phase_e_munchausen_target → launch_alpha_munchausen_target
Static cubin names:
PHASE_E_MUNCHAUSEN_TARGET_CUBIN → ALPHA_MUNCHAUSEN_TARGET_CUBIN
Historical milestone tags in doc-comments ("Phase E.1 Task N (2026-05-15)")
are RETAINED — they record WHEN the work landed and what plan it
implemented, which doesn't change with the system-scoped rename.
Plus: ADDS the alpha_munchausen_target GPU smoke test in alpha_kernels.rs.
End-to-end validates the launcher + kernel against hand-computed expected
values: batch=2 with one terminal sample; expected targets [29.8, 1.1];
got match within 0.05 tolerance on RTX 3050 Ti. PROVES the Task 9/10
kernels actually run on GPU.
All affected references updated in:
- build.rs (kernel compile list)
- mod.rs (module registration)
- state_reset_registry.rs (4 RegistryEntry descriptions for slots 539-542)
- alpha_isv_slots.rs (slot table comment)
- docs/isv-slots.md (audit-doc cross-references)
Verified:
cargo test -p ml --lib alpha_kernels: 2/2 pass (including GPU smoke)
cargo test -p ml --lib state_reset_registry: 10/10 pass
cargo build -p ml --release --example alpha_fit_fill_model --example alpha_random_baseline: clean
|
||
|
jgrusewski
|
feb2e8cc34 |
feat(alpha): phase_e_kernels — Rust launchers for Task 9 + Task 10 cubins
Phase E.1 Task 11. Two pub(crate) launchers expose the kill-criteria
producer (Task 9) and Munchausen target augmentation (Task 10) for use
by future trainer integration:
launch_phase_e_kill_criteria(stream, kernel, q_values_dev, action_counts_dev,
scalar_inputs_dev, isv_dev, batch, n_actions,
scratch_out_dev)
→ kicks the kill_criteria producer; caller chains apply_pearls_ad_kernel
(n_slots=4, isv_idx_base=ALPHA_ISV_BLOCK_LO=539) to smooth into slots
539..542 via Pearl A bootstrap + Pearl D Wiener-α.
launch_phase_e_munchausen_target(stream, kernel, q_next_dev, q_current_dev,
actions_dev, rewards_dev, dones_dev,
gamma, alpha_m, tau, log_clip_min,
target_out_dev, batch, n_actions)
→ one-thread-per-sample target augmentation; α_m/τ/log_clip_min are
scalar args so a downstream ISV-driven controller can tune them.
Plan deviation: Task 11 plan-spec said "replace hardcoded n_step=32,
gamma=0.999 literals" but grep across gpu_dqn_trainer.rs found ZERO such
literals — the trainer already reads gamma via read_isv_signal_at(
GAMMA_DIR_EFF_INDEX) and epsilon via read_isv_signal_at(AUX_TRUNK_EPS_
INDEX). The actual E.1 deliverable was Rust launchers for the new
cubins, which this commit lands.
Both launchers follow the launch_apply_pearls pattern in
sp4_wiener_ema.rs — pre-loaded CudaFunction as parameter, u64 device
pointers, debug_assert! guards.
Audit doc docs/isv-slots.md updated per Invariant 7.
Tested via `cargo test -p ml --lib phase_e_kernels` — 1 compile-witness
passes. Real GPU integration test in Task 12.
|
||
|
jgrusewski
|
b1ba41d403 |
feat(alpha): Munchausen DQN target term kernel (Vieillard et al. 2020)
Phase E.1 Task 10. Standalone target-augmentation kernel: m = α_m · max(τ · log π(a|s), log_clip_min) V_soft(s') = max(Q_next) + τ · log Σ exp((Q_next − max) / τ) target = r + m + γ · V_soft(s') (terminal: r + m) π(a|s) ∝ exp(Q_online(s, a) / τ) — softmax policy from the online net. Munchausen bonus is implicit KL regularisation between successive policies; soft-V replaces the hard max bootstrap with a τ-weighted softmax average. Both softmaxes are computed via log-sum-exp with the max-trick. This is essential at τ ≈ 0.03 where raw exp(Q/τ) would overflow f32 for any Q-spread > 25 nats. The kernel is one-thread-per-batch-sample, no atomicAdd, no host branches. α_m, τ, log_clip_min are kernel args (not hard-coded), so a Phase E.2+ controller can ISV-drive them. Typical Vieillard values: α_m=0.9, τ=0.03, log_clip_min=-1.0. Does NOT touch any ISV slot — pure target augmentation. Cubin: target/release/build/ml-*/out/phase_e_munchausen_target.cubin (12.8 KB). Launcher integration is Task 11 (consumes target_out where the C51/MSE loss kernels currently consume `r + γ · max_a' Q_target`). Audit doc docs/isv-slots.md updated per Invariant 7. |
||
|
jgrusewski
|
d493b729bf |
feat(alpha): phase_e_kill_criteria producer kernel (slots 539-542)
Phase E.0 Task 9. Single-block, single-thread CUDA producer that writes 4
raw scalar observations into a contiguous scratch_out[0..4] block:
scratch_out[0] → ISV[539] Q_SPREAD_EMA (kill: ≥ 0.05)
scratch_out[1] → ISV[540] ACTION_ENTROPY_EMA (kill: ≥ 0.5·ln(9) ≈ 1.10)
scratch_out[2] → ISV[541] RETURN_VS_RANDOM_EMA (kill: ≥ 0, i.e. ≥ random)
scratch_out[3] → ISV[542] EARLY_Q_MOVEMENT_EMA (kill: ≥ 0.01, learned)
Downstream `apply_pearls_ad_kernel` (n_slots=4) chained on the same stream
applies Pearl A first-observation bootstrap + Pearl D Wiener-α smoothing —
composes with the canonical val_sharpe_delta_compute_kernel pattern rather
than reimplementing Wiener math (per feedback_no_cpu_compute_strict — one
Wiener implementation in the codebase, the GPU one).
Inputs:
q_values[batch, n_actions] most-recent Q forward output (device)
action_counts[n_actions] empirical action histogram (device)
scalar_inputs[3] [rollout_R_mean, q_init_norm, q_early_norm]
via mapped-pinned (host writes between rollouts)
isv reads slots 547 + 548 (random baseline
mean/std — populated once by Task 7c,
TrainingPersist)
No atomicAdd (per feedback_no_atomicadd), no host branches
(per pearl_no_host_branches_in_captured_graph), no CPU compute
(per feedback_cpu_is_read_only). __threadfence_system() before exit for
the chained Pearls applicator's visibility guarantee.
Cubin produced: target/release/build/ml-*/out/phase_e_kill_criteria.cubin
(16.8 KB).
Launcher integration is Task 11 (separate commit so this task can stand
alone for cubin validation). Audit doc docs/isv-slots.md updated per
Invariant 7.
|
||
|
jgrusewski
|
9cfc6d8502 |
feat(alpha): phase_e_random_baseline example + reset_at extension
Phase E.0 Task 7b. Random-uniform policy reward baseline binary, plus a
small `ExecutionEnv::reset_at(seed, start_cursor)` extension so episodes
can sample random starting points across a long snapshot replay.
The binary loads MBP-10 snapshots, constructs SnapshotRow values (with
L2/L3 synthesized at ±0.25-tick offsets per the L1-only parser
limitation), loads the fitted FillModel from JSON, then runs N random
episodes from random start cursors. Reports mean / std / quintile
percentiles + kill threshold (mean + 2σ) for E.1 to exceed.
Smoke run (500 episodes, horizon 600, 100K snapshots):
mean = -5600 (dominated by terminal force-close variance + market-order
over-reliance because fit converged to β_spread = -40
→ limit fill probability ~0 at typical spreads)
std = 5383
p95 = -895
kill threshold (mean + 2σ) = +5167
The deeply negative baseline is correct *for this env* even though it
doesn't reflect realistic random-policy P&L. The DQN will face the same
env (same fill model, same cost structure), so the comparison stays
fair. Fitter regularisation (to prevent β_spread runaway) is a Phase E.1
follow-up.
Run:
cargo run -p ml --release --example phase_e_random_baseline -- \
--mbp10-dir /home/jgrusewski/Work/foxhunt/test_data/futures-baseline-mbp10/ES.FUT \
--fill-coeffs config/ml/phase_e_fill_coeffs.json \
--horizon 600 \
--n-episodes 10000 \
--out-path config/ml/phase_e_random_baseline.json
env.reset_at also called by reset() (1-line refactor); no behavior change.
|
||
|
jgrusewski
|
12151ccf6a |
feat(alpha): fitted FillModel coefficients from 500K ES.FUT snapshots
Phase E.0 Task 5c. Ran phase_e_fit_fill_model on the ES.FUT 2024-Q1 MBP-10 + trade tape (5.2M trades, 3.9M MBP-10 events, 500K snapshots accumulated at snapshot_interval=50 over a ~24-minute window). Total runtime ~80s. Empirical fill rates within 60s window: - bid_l1: 4.97% (matches L1 maker-side activity in trending market) - ask_l1: 71.34% (high — most 60s windows see an aggressive buy) Fitted L1 cloglog coefficients (all 5 features): BID L1: β_0=-0.213 β_spread=-2.064 β_imbal=-0.099 β_ofi=-0.006 β_logτ=-0.286 ASK L1: β_0=+0.016 β_spread=-40.336 β_imbal=+0.041 β_ofi=+0.652 β_logτ=-0.055 Sanity (sign checks all pass): - β_spread < 0 both sides (wider spread → fewer fills) ✓ - bid β_imbal < 0 (more bid stack → harder to get hit by sell) ✓ - ask β_ofi > 0 (buying pressure correlates with ask fills) ✓ - β_logτ < 0 both sides (quieter markets → slower execution) ✓ L1-only limitation: as documented in the binary header, the parser only populates levels[0]; L2/L3 in the JSON are L1 with β_0 -= ln(L+1) attenuation. Default --out-path bumped to config/ml/phase_e_fill_coeffs.json so future re-runs land in the same committed location. |
||
|
jgrusewski
|
3bdf74018d |
feat(alpha): phase_e_fit_fill_model example — cloglog FillModel calibration
Phase E.0 Task 5b. Calibrates FillModel coefficients from historical MBP-10
+ trade tape. Streams snapshots concurrently with time-sorted trades; per
snapshot, determines binary fill outcome ("would a posted L1 limit have
been hit within next --window-seconds?"), accumulates (FillFeatures, y),
calls fit_poisson (cloglog Bernoulli, see
|
||
|
jgrusewski
|
a1e3336b1f |
feat(alpha): ExecutionEnv — 10-dim state, 9-action gating, terminal-only reward
Phase E Task 6. Core environment for the execution-policy DQN.
Episode lifecycle: position-open signal → up to `horizon_snapshots` →
forced close. Within the episode the policy chooses 9 actions; illegal
actions degrade to Wait inside step() (soft masking) — the network's
Q-output is never masked, preserving clean C51 and Munchausen targets.
Reward is **terminal-only**: realized PnL minus fees, accumulated over
all fills and force-closed at terminal mid if still open. No dense
per-snapshot shaping per pearl_event_driven_reward_density_alignment
(the canonical SP11→SP12 lesson — dense shaping on event-driven
objectives creates exposure-positive bias).
Bugs in the plan's sketch, caught + fixed during implementation:
1. Synthetic test's bid/ask were CONSTANT while mid drifted — would
have made the "uptrend" test lose money. Fixed: drift bid/ask
together with mid.
2. L1/L2 closing path used Side::None for fill lookup → 0% fill rate
for closing limits. Fixed: closing-long → Side::Sell (post at ask),
closing-short → Side::Buy (post at bid).
3. `cursor`-based hash for fill PRNG was non-deterministic across
replay seeds. Replaced with self-contained SplitMix64 RNG state
(no `rand` dep added) — fill randomness is now bit-identical across
train/eval replays at the same seed.
4. `max_step_per_episode` and `mid_at_decision` fields stored but
never read — dead per feedback_no_stubs. Removed.
5. Test harness used n==horizon, which made the cursor-exhaustion
guard (`cursor+1 >= len`) fire before the planned FlatMarket call.
Fixed by setting n > horizon so the meaningful `ep.step >= horizon`
path terminates the episode.
6. `.unwrap()` in test bodies → `.expect("…")` per pre-commit policy.
7. Volume-weighted entry_price across multi-fill scaling (the plan
only handled the open-from-flat case).
5 unit tests:
- state_has_expected_dim_and_is_finite (STATE_DIM=10, no NaN)
- market_buy_then_market_sell_on_uptrend_profits (PnL math)
- illegal_action_degrades_to_wait (soft mask, no fee, no fill counted)
- terminal_force_close_pays_out_when_position_open (force-close at mid)
- rng_is_deterministic_across_resets (same seed → identical fills)
`cargo test -p ml --lib env`: 16 passed (4 action_space + 7 fill_model
+ 5 execution_env); full module compiles clean with no new warnings.
|
||
|
jgrusewski
|
d08ab461db |
feat(alpha): fit_poisson via cloglog likelihood + Serde derives on FillCoeffs
Phase E Task 5 (code portion; the 5.2M-trade fit run lands separately).
Diagnosis: the plan's draft used pure Poisson NLL with Bernoulli y∈{0,1},
which converges to λ = empirical rate ȳ. But the runtime fill_prob() uses
`p = 1 − exp(−λ)`, so a trained λ=0.4 → predicted p=0.33 → systematic
~30pp under-fill bias on every passive backtest order. Training and
inference must agree on what λ means.
Fix: switch the fitter to the **cloglog (complementary log-log) binary
likelihood**. Per-sample gradient:
∂L/∂β_k = (p − y) · (μ/p) · x_k
where μ = exp(β·x), p = 1 − exp(−μ)
The μ/p factor is the link derivative; p.max(1e-7) handles the μ→0 limit
in f32 (the analytical limit μ/p → 1 is achieved automatically because
both numerator and denominator vanish proportionally).
Recovery test verifies the fix: 1000 deterministic 40% fills, all-zero
features → fitter recovers β_0 ≈ ln(0.5108) ≈ -0.672 (the value at which
1 − exp(−exp(β_0)) = 0.4), within tolerance 0.05. Slope coefficients
stay near zero (features uninformative).
Also adds Serde derives on FillCoeffs / FillFeatures / FillModel for JSON
serialization (downstream when the calibration example lands).
Deferred: the calibration example (Steps 3-7 of the plan task) is held
back until paired with the actual 5M-trade fit run — the plan's example
has a placeholder loop that violates feedback_no_stubs, and the loader
lift from precompute_features.rs deserves a dedicated commit.
4 new tests (7 total in env::fill_model now):
- fit_recovers_baseline_when_features_uninformative
- fit_rejects_empty_and_mismatched_inputs
- coeffs_round_trip_through_json
`cargo test -p ml --lib env::fill_model`: 7 passed.
|
||
|
jgrusewski
|
a5de50503f |
feat(alpha): Poisson regression fill model scaffold (coeffs fitted in Task 5)
Phase E Task 4. Medium-tier fill simulator per the design memo.
For each (level ∈ {L1,L2,L3}, side ∈ {Bid,Ask}):
λ(features) = exp(β · [1, spread_bps, L1_imb, OFI_5, log(τ+1)])
Per-snapshot Bernoulli fill probability for a posted limit order:
p = 1 − exp(−λ)
Market orders fill immediately at the opposite-side L1 quote (no slippage
modeled at medium tier). Closing actions (Side::None) return λ=0 — they
are handled separately in the env.
Scaffolding only. Task 5 fits the 30 coefficients (6 distributions × 5
features) from the 5.2M-trade historical tape. The skeleton constructor
uses β=0 → λ=1 → p≈0.632, a stable sanity default for early smokes.
Numeric guard: `linear.exp().min(50.0)` caps λ to prevent f32 overflow
under outlier features before fitting lands. Fitted models should stay
well below this cap in practice.
4 unit tests:
- skeleton_has_uniform_fill_prob (β=0 → p≈0.632 within 0.01)
- fill_prob_bounded_under_outlier_features
- lambda_cap_prevents_f32_overflow (β=100 outlier path)
- side_none_returns_zero_rate
`cargo test -p ml --lib env::fill_model`: 4 passed.
|
||
|
jgrusewski
|
94f8f65571 |
feat(alpha): 9-action discrete execution action space + legality gating
Phase E Task 3. Introduces crates/ml/src/env/ for the execution-policy
environment. This commit lands the action space only; fill_model (Task 4)
and execution_env (Task 6) append to mod.rs in their respective commits
per feedback_wire_everything_up (no orphan declarations).
- 9-action discrete space: Wait + {Buy,Sell,Flat} × {Market,L1,L2 / L1}
- Soft action masking: illegal actions degrade to Wait in env::step
(not by masking Q-output) per design memo — preserves clean C51
categorical targets and Munchausen term (Task 10)
- `is_legal(position)` with position ∈ {-1, 0, +1} (sign only;
magnitude is decoupled into the Kelly layer in Task 11)
- repr(u8) discriminants round-trip with from_u8 for replay-buffer
storage by the existing GPU DQN trainer
4 unit tests:
- n_actions_matches_enum_cardinality
- legality_gating_by_position (covers flat/long/short × all 9 actions)
- decode_closing_flag_is_only_flat
- repr_u8_round_trip (replay-buffer contract)
`cargo test -p ml --lib env::action_space`: 4 passed.
|
||
|
jgrusewski
|
8bf8bdf874 |
feat(alpha): state_reset_registry entries + dispatch arms for slots 539..548
Per feedback_registry_entries_need_dispatch_arms — both must land in the same commit; the test `every_fold_and_soft_reset_entry_has_dispatch_arm` walks training_loop.rs::reset_named_state source and asserts coverage. - 10 RegistryEntry rows appended after SP22 block (full producer/consumer rationale per existing dense-description pattern) - 7 dispatch arms in reset_named_state (the 3 TrainingPersist anchors — slots 544/547/548 — are not dispatched per the existing convention) - All sentinels 0.0 per pearl_first_observation_bootstrap All 10 registry tests pass. Audit doc docs/isv-slots.md updated per Invariant-7. |
||
|
jgrusewski
|
aa5908aa53 |
feat(alpha): reserve ISV slot block 539..550 for alpha trading system
12 contiguous slots reserved for durable alpha-system infrastructure: - 539-542: diagnostics (Q-spread, action entropy, return-vs-random, early-Q-movement EMAs) — read by Phase E kill criteria - 543-546: stacker-threshold controller (threshold, target, observed-rate, Kelly attenuation) — engagement-rate self-correction - 547-548: random-uniform baseline (mean, std) — anchor for kill criterion 541 - 549-550: reserved spare (absorb growth without bumping ISV_TOTAL_DIM) Named `alpha_isv_slots` rather than `phase_e_isv_slots` because these slots are intended to outlive any individual Phase E/F/G milestone — the SP4..SP22 naming was iteration-scoped, but this block is system-scoped infrastructure. ISV_TOTAL_DIM bumped to 551. All indices validated in 4 unit tests (bounds, membership, uniqueness, total-dim coverage). Audit doc docs/isv-slots.md updated per Invariant-7. |
||
|
jgrusewski
|
db874b1841 |
feat(foxhuntq): Phase 1c snapshot-resolution alpha + leakage fix + variable-dim fxcache
Three things landing atomically because they're load-bearing for each other: 1. **Trend-scanning leakage fix** — trend_scanning.rs was emitting OLS slope+t-stat over a *forward* window [t, t+L]. With the Phase 1a label = sign(price[t+60] − price[t]), the forward feature window overlaps the label window, contaminating it. Purged walk-forward only sterilizes forward-looking *labels* that cross the train/val split, not forward-looking *features* that peek inside the same horizon the label measures. The leak inflated MLP accuracy from 0.49 (legacy 74-dim baseline) to 0.75 — vanished to 0.50 after switching to a trailing window. Bounded the perfect-fit t-stat sentinel from ±1e6 → ±20 (p<1e-30 is already meaningless); eliminated the 16k corruption-cap drops. 2. **Variable-dim alpha column** — fxcache schema now carries the alpha-feature width via metadata (`alpha_feature_dim`), not a compile-time constant. Same on-disk format hosts the 134-dim bar-level stack OR the 81-dim snapshot stack. Reader + auto-detect honor the metadata-declared dim; downstream MLP auto-sizes `in_dim`. Single schema, no forks. 3. **Snapshot pipeline (Phase 1c falsification)** — `snapshot_pipeline.rs`: 81-dim per-MBP10-snapshot extractor reusing 10 snapshot-native alpha blocks + 6 new snapshot-specific features (time-since-trade, time-since-snap, event-rate, spread-bps, L1-imbalance, microprice-mid drift). `precompute_features` gets `--row-unit snapshot` flag; emits one fxcache row per LOB update (1.97M rows from MBP-10 data vs 206K for bar mode). **Smoke verdict on real data** (ES.FUT, 1.97M snapshots, 384K val): - Bar-level honest alpha: accuracy=0.5005, AUC=0.5043 (no signal) - **Snapshot-level alpha**: accuracy=0.5241, AUC=0.6849 (real signal, 384K val) - GBM corroboration: accuracy=0.5401 (non-linear partitioning sees more) - Horizon decay: alpha peaks at K=20-50 snapshots (~5-25ms), gone by K=500 - Regime-conditional: spread-Q4 quintile hits 0.752 accuracy on 76k samples Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
19bb3bc3c8 |
fix(sp22-vnext): aux_outcome CF half mirror on-policy (NOT all -1)
Smoke train-k95mj epoch 0 + epoch 1 printed
`trade_outcome_ce=0.000e0` — K=3 CE EMA stuck at Pearl A sentinel
across all training steps.
Root cause chain:
1. insert_batch is called with bs = total = 4_096_000 (cf-mult-
expanded), replay cap = 300_000 (L40S GpuProfile). Tail-clip:
off = 3_796_000, slice(off..) takes the last 300K elements.
2. Prior fix (
|
||
|
jgrusewski
|
878cc9ba72 |
feat(sp22-vnext): F-3c follow-up — K=3 CE EMA in stdout HEALTH_DIAG aux line
The F-3c commit (
|
||
|
jgrusewski
|
256a5fa5ab |
fix(sp22-vnext): K=3 aux_outcome_labels CF-half buffer underrun
Phase B4b-2 introduced `exp_aux_to_label_per_sample` sized
`[alloc_episodes × alloc_timesteps]` (no cf-mult expansion — the
B4b-1 per-step kernel only writes the on-policy half). The batch
finalisation cloned into the emitted `aux_outcome_labels` with size
`total = base_total × 2` (cf-mult expanded), so
`dtod_clone_i32(src, total, ...)` invoked `src.slice(..total)` on a
half-sized source → `CudaSlice::try_slice` returned `None` → the
internal `unwrap()` panicked at cudarc safe/core.rs:1648.
Repro: workflow train-xzv56 panicked after rollout completed
(timestep=999) on fold 0; rollout itself ran clean, the OOM from
the prior commit is gone.
Fix: replace the single `dtod_clone_i32` with a 3-step build:
1. alloc_zeros::<i32>(total)
2. cuMemsetD32Async(ptr, 0xFFFFFFFFu32, total, stream) — fills
all `total` slots with i32 mask sentinel -1 (byte pattern
0xFFFFFFFF reinterprets as i32(-1))
3. memcpy_dtod the first `base_total` real labels from
exp_aux_to_label_per_sample into the on-policy half
CF half remains at -1. The K=3 sparse-CE loss masks `label == -1`
out of the mean and B_valid count, so CF samples contribute zero
gradient — exactly the semantic we want (CF actions have no
observed trade-close outcome to predict against).
Lib suite: 1016/0 green. Audit doc updated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
0acf77e656 |
config(dqn): strip H100-tuned VRAM overrides from dqn-production.toml
dqn-production.toml hard-coded three H100-tuned values that shadow
GpuProfile auto-detection: batch_size=16384, buffer_size=500K,
gpu_n_episodes=4096. After the L40S-default flip lands (prior commit
|
||
|
jgrusewski
|
bbd52c3aa7 |
feat(sp22-vnext): FoldReset registry entries for B5b/C collector buffers
Two collector-side device buffers used by the K=3 trade-outcome head
were missing FoldReset registry coverage:
- prev_aux_outcome_probs [alloc_episodes × 3]
TRUE stale-read risk. Producer writes end-of-step; consumer
(experience_state_gather) reads start-of-next-step into
state[121..124). Without FoldReset the new fold's step-0 state
gather would inject the previous fold's last-step softmax probs
into the first batch's state slots.
- exp_aux_to_input_buf [alloc_episodes × 262]
Cleanliness-only. Concat kernel overwrites all 262 columns every
step before the K=3 forward reads them, so no steady-state stale-
read risk. Registered for parity with the rest of the K=3
pipeline + to satisfy feedback_registry_entries_need_dispatch_
arms (the pin test asserts every registry entry has a matching
dispatch arm in reset_named_state).
Both fields promoted to pub(crate) on GpuExperienceCollector so
reset_named_state can reach them. Matching dispatch arms added with
the standard memset_zeros pattern (is_win_per_env / hold_baseline_
buffer style).
Tests: All 10 state_reset_registry tests pass, including the critical
every_fold_and_soft_reset_entry_has_dispatch_arm pin test that walks
the dispatch body and validates parity with registry entries. Full
lib suite 1015/1 (the failing test is the pre-existing
test_dqn_checkpoint_round_trip NoisyLinear flake — pred1/pred2 sign
mismatch surfacing ~30-50% of full-suite runs, documented in
project_sp22_h6_vnext_resume memory as unrelated to this work).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
4b40710b7c |
feat(sp22-vnext): Phase B5b-2 collector trade plan forward — resolves K=3 asymmetry
Phase B5b's K=3 input concat passed plan_params=NULL because the collector had no trade plan launch. Trainer-side K=3 forward trained on real plan_params while the collector queried at plan_params=0 — documented train/inference asymmetry on the plan-conditioning surface. Phase B5b-2 mirrors the (now-corrected) trainer `launch_trade_plan_forward` chain on the collector inside the rollout step: 1. SGEMM: hidden[N, AH] = h_s2_q[N, SH2] @ W_fc[AH, SH2]^T 2. bias+relu in-place on hidden 3. SGEMM: pre_out[N, 6] = hidden[N, AH] @ W_out[6, AH]^T 4. trade_plan_activate → exp_plan_params[N, 6] Weight resolution uses the same `aux_w_ptrs` array the K=3 forward already consumes (`f32_weight_ptrs_from_base`); indices 91-94 match the corrected trainer-side reads. The `trade_plan_activate` kernel is loaded from `EXPERIENCE_KERNELS_CUBIN` (same cubin the rest of `exp_module_extra` uses; the trainer loads it from there too). The K=3 concat now takes `exp_plan_params.raw_ptr()` instead of NULL — both sides see f(h_s2; W_plan_*_init), symmetry restored. Plan tensors at [91..94] still have no backward (no Adam updates), so the plan-head weights stay at Xavier cold-start forever. This commit delivers the symmetry the K=3 head requires, not a learned plan signal — adding a real plan-head backward is a follow-up project. New struct fields on GpuExperienceCollector: - exp_trade_plan_hidden_buf [alloc_episodes × adv_h] - exp_trade_plan_pre_out_buf [alloc_episodes × 6] - exp_plan_params [alloc_episodes × 6] - exp_trade_plan_activate_kernel: CudaFunction Lib test suite: 1016/0 green maintained. Audit doc updated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
fb9b62a1f9 |
fix(plan-head): trainer trade plan forward reads from wrong param indices
`launch_trade_plan_forward` read `w_fc/b_fc/w_out/b_out` from `padded_byte_offset(¶m_sizes, [82..86))` — i.e., the ISV-conditioning + recursive-confidence tensors (`b_isv_gate`, `w_isv_gamma`, `b_isv_gamma`, `w_conf_fc`). The actual plan tensors live at `[91..95)` per `compute_param_sizes` and the matching Xavier init block. No backward exists for the plan head, so the plan tensors at `[91..94]` sat at cold-start Xavier values forever while `plan_params` was being driven by whichever ISV/conf weights happened to occupy the wrong offsets. Every downstream consumer (`backtest_plan_kernel`, `plan_isv` slots in `experience_kernels`, `compute_plan_params` in `q_value_provider`, regime gating, and the SP22 H6 vNext B5b concat path) was therefore conditioning on noise correlated with ISV optimisation, not on a learned plan. Discovered while preparing Phase B5b-2 (collector trade plan launch). Two-line index fix + diagnostic comment + audit doc entry. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |