2feeeda8bb5dc9863aa3427d72399c861b79aee1
2591 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
jgrusewski
|
2feeeda8bb |
fix(phase-e-4-a): GPU kernel for h_enriched slot copy — eliminate per-step CPU roundtrip
The Phase E.4.A T8 wiring stored Mamba2's per-step cache.h_enriched
into h_enriched_buf_dev via a dtoh+htod sequence:
let h_host = stream.clone_dtoh(cache.h_enriched.cuda_data())?;
let mut buf_host = stream.clone_dtoh(&h_enriched_buf_dev)?; // <- whole buffer
for j in 0..hidden_dim { buf_host[slot_offset + j] = h_host[j]; }
stream.memcpy_htod(&buf_host, &mut h_enriched_buf_dev)?; // <- whole buffer
This violates feedback_cpu_is_read_only AND
feedback_no_htod_htoh_only_mapped_pinned. Worse, the buffer-wide
dtoh+htod every step is ~20K floats × 600 steps × 500 eps × 30 cells
= ~9M roundtrips totaling significant PCIe latency in the backtest.
Fix: new tiny CUDA kernel alpha_h_enriched_store_kernel in
alpha_window_push.cu (one thread per hidden-dim feature, writes
src[j] → buf[slot_offset + j]). Replaces the dtoh/htod sequence
in both smoke and backtest binaries.
Estimated speed-up at backtest scale: 3-6× on the temporal eval
path. Pure-GPU per-step inference restored — no synchronisation
points on the hot path.
docs/isv-slots.md updated per kernel-audit-doc hook.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
4d65ace625 |
feat(phase-e-4-a): mirror --temporal in backtest binary + T11 ISV-continual
Phase E.4.A Tasks 11+12: backtest binary gains the same
--temporal Mamba2 forward chain as the smoke binary, plus the
--isv-continual flag that fires the stacker-threshold controller
at the end of each eval episode (Pillar B).
Changes:
- Imports: ml_alpha::mamba2_block::{Mamba2Block, Mamba2BlockConfig},
ml_core::cuda_autograd::gpu_tensor::GpuTensor
- CLI flags: --temporal, --window-k, --mamba2-hidden-dim,
--mamba2-state-dim, --isv-continual
- Cubin loading: alpha_window_push + stacker_threshold_controller
- Q-net sizing: c51_input_dim = mamba2_hidden_dim when --c51 --temporal
- Buffers: window_tensor GpuTensor, h_enriched_buf_dev, isv_dev,
ctl_wiener_dev
- Training inference path: push + Mamba2 forward + h_enriched →
C51 forward (mirrors smoke binary)
- Training batched compute: terminal Mamba2 forward, h_enriched_buf
for current/next, c51_input_dim threading
- Eval inference path: push + Mamba2 forward + h_enriched → C51
forward + Thompson select (with scoped borrow guard)
- T11 ISV-continual: stacker-threshold controller fires at end of
each eval episode; ISV slot 543 (threshold), 545 (observed-rate),
546 (Kelly atten) update with realized rollout stats. Co-exists
with the τ-grid sweep (τ-grid still gates; ISV updates parallel
observation of "live deployment" behaviour).
- JSON output: new fields temporal, window_k, mamba2_hidden_dim,
mamba2_state_dim, isv_continual.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
75d67bd203 |
feat(phase-e-4-a): alpha_c51_grad_input kernel (T10 prerequisite)
Phase E.4.A Task 10 partial: adds the C51 gradient-w.r.t.-input
kernel needed to chain the C51 head's loss gradient back into the
Mamba2 temporal encoder.
Kernel signature: alpha_c51_grad_input_kernel reads probs[B, A, K],
m[B, K], actions[B], W[A*K, in_dim] and writes d_input[B, in_dim].
Math: dL/d_input[b,j] = Σ_k (p[b,a_taken,k] − m[b,k]) · W[a_taken*K+k, j]
(only the taken-action column of W contributes; restricted by the
sparse-over-actions C51 CE gradient structure).
Status: kernel + Rust launcher in place. Mamba2 backward NOT yet
wired in the smoke binary because ml_alpha::Mamba2Block::backward
takes d_logit [B, 1] (post-W_out scalar gradient), not
d_h_enriched [B, hidden_dim]. Two paths to complete:
1. Modify ml-alpha to expose backward_from_h_enriched(cache,
d_h_enriched) bypassing W_out.
2. Replicate the post-W_out backward sequence inline.
Both deferred pending backtest validation (T14): if frozen Mamba2
already lifts backtest Sharpe meaningfully, T10 becomes
optimisation rather than prerequisite. Smoke gate already passed
gate 1 (R_mean +3.9 vs C51-flat -1.1) with frozen weights.
docs/isv-slots.md updated.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
ed6f5588e1 |
feat(phase-e-4-a): wire Mamba2 forward in smoke --temporal path
Phase E.4.A Task 8: wire ml_alpha::Mamba2Block as the temporal
encoder before the C51 head when --temporal is set.
Architecture (--temporal):
state_pinned ──push─▶ window_tensor[1, K=16, in_dim=10]
│
▼ Mamba2Block::forward_train
h_enriched[1, hidden_dim=32]
│
▼ launch_alpha_c51_forward (input dim=32)
probs[1, 9, 51] ──▶ Thompson selector
Implementation:
- Mamba2Block constructed at startup with config (in_dim=10,
hidden_dim=32, state_dim=16, seq_len=K=16). Loaded from ml-alpha's
precompiled cubin.
- Per-step: window push (shift+insert), then forward_train returns
(logit, cache). We discard logit (ml-alpha's binary classifier head)
and use cache.h_enriched as the C51 input.
- Per-step h_enriched cached into h_enriched_buf_dev[(t)..t+hidden_dim].
- Batched training (end-of-episode): the C51 forward + grad use
h_enriched_buf_dev[0..ep_len*hidden] for the current state and
[hidden..(ep_len+1)*hidden] for next-state (1-step offset). Runs
one extra Mamba2 forward on the terminal window to populate slot
ep_len.
- C51 input dim (W shape) becomes mamba2_hidden_dim when --temporal,
STATE_DIM otherwise.
100-episode smoke verdict (vs C51-flat baseline):
R_mean ep 50: C51-flat -8.2 → --temporal +0.4 (+8.6)
R_mean ep 100: C51-flat +0.5 → --temporal +10.0 (+9.5)
rvr: +1.045 → +1.046 (unchanged)
Q_SPREAD: 23.9 → 12.6 (sharper distributions)
ACTION_ENTROPY: 1.42 → 1.49 (now PASSES 0.5×ln(9) threshold)
Note: Mamba2 weights are FROZEN at random Xavier init in this
commit — T10 (backward + AdamW step) lands next. The R_mean lift
above is from C51 learning over RANDOM temporal projections of the
window — random SSM acts as a feature-engineering reservoir.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
35dcb87709 |
refactor(phase-e-4-a): window push to shift+insert (chronological layout)
Switch alpha_window_push from circular-buffer-with-head_idx layout to shift+insert layout matching production mamba2_update_history. Slot 0 = oldest, slot K-1 = newest after each push, matching Mamba2Block's [B, K, in_dim] input contract directly (no reorder). Cost: O(K-1) shifts per state_dim feature per push. For K=16, state_dim=10: 10 threads × ~15 ops each = trivial. Kernel signature: drops head_idx, adds K. Test updated to verify chronological shift across 3 pushes into 4-slot buffer. docs/isv-slots.md updated per kernel-audit-doc hook. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
70df697328 |
feat(phase-e-4-a): wire sliding-window buffer in smoke (buffer-only)
Phase E.4.A Task 7: maintain a GPU-resident circular window buffer in the smoke binary's --temporal path. Per-step: 1. mapped-pinned state_pinned write (existing) 2. alpha_window_push_kernel writes state into window[head_idx] 3. head_idx = (head_idx + 1) % window_k 4. C51 forward proceeds against state_pinned (consumer of window wires in T8 — Mamba2 over the window) On episode reset: zero the buffer and reset head_idx so Mamba2 sees clean zero-context for the first window_k-1 steps. CLI: --temporal flag + --window-k (default 16, kernel max 32 per mamba2_alpha_kernel constraint). Validation: 100-episode smoke with --temporal produced bit-identical R_mean / rvr / kill-criteria values to the C51-flat baseline run — confirms buffer maintenance has zero side effect on the existing C51 path. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
6e86e5b435 |
feat(phase-e-4-a): alpha_window_push circular-buffer kernel + GPU test
Phase E.4.A Task 6: tiny CUDA kernel that pushes a state[state_dim] vector into slot `head_idx` of a circular window buffer [K, state_dim]. Host tracks head_idx and zeros buffer on episode reset. This is the GPU-side append primitive that the smoke binary's per-step inference will call (T7) before Mamba2 over the buffer (T8). GPU smoke (alpha_window_push_circular_writes_to_indexed_slot): writes state_a at slot 0, state_b at slot 2, verifies non-targeted slots stay zero. PASS. docs/isv-slots.md: documents kernel as slot-agnostic per kernel-audit-doc hook requirement. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
5d7d4fa3c6 |
feat(phase-e-4-a): add mbp10_dir param to fxcache loader (signature-only)
Phase E.4.A Task 4: extend load_snapshots_from_fxcache with `mbp10_dir: Option<&Path>`. When provided, the loader will peek MBP-10 by timestamp and populate SnapshotRow.bid_l[1..10]/ask_l[1..10] from real LOB depth — but the real-peek implementation lands in Task 5 follow-on. This commit: - introduces the parameter (callers pass None) - warns at runtime if mbp10_dir Some until T5 lands - enables downstream wiring of --use-real-depth + --mbp10-dir CLI flags in the smoke / backtest binaries T5 deferred: on ES futures the --real-spread experiment showed 76% of fxcache bars hit the 1-tick floor, so depth-from-MBP-10 likely won't move the needle for ES. Higher-leverage work (Mamba2 wiring) prioritised. T5 implementation reopens as a follow-on if E.4.A gates pass with synthesised depth. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
eb49e2a0f7 |
feat(alpha): Phase E.3 follow-up — C51 distributional Q + Thompson + L1-L10 depth + falsifications
C51 distributional Q-network with GPU Thompson selection borrowed
minimally from production (alpha_c51.cu: forward, project, grad,
expected_q, thompson_select kernels; ~260 lines). Uses Huber
negative-tail compression in projection per production
block_bellman_project_f. Action selection 100% GPU via mapped-pinned
i32 output + __threadfence_system + host volatile read (matches
gpu_training_guard MappedBuffer pattern).
Backtest result (2D sweep, 500 episodes per cell, 30 cells):
cost=0 C51 +10.41 vs linear-Q -15.72 (+26pt, BEATS Phase 1d.4
no-RL baseline +4.4 by 6pt)
cost=0.125 C51 -13.81 vs -29.17 (+15pt closes half-tick gap)
Win rate at cost=0 best τ: linear-Q 0.008 → C51 0.552.
Calibration hypothesis vindicated; documented in
memory/pearl_c51_thompson_closed_phase_e3_gap.md.
Also in this commit (Phase E.3 follow-up cleanup):
- --pruned-actions falsified (2.4× worse Sharpe). Documented in
memory/pearl_action_pruning_falsified.md.
- --real-spread falsified for ES futures (76% of bars at 1-tick floor).
- SnapshotRow bid_l/ask_l extended from [f32; 3] to [f32; 10].
L4-L10 synthesized in this commit; real MBP-10 peek lands in E.4.A T5.
- docs/isv-slots.md updated per kernel-audit-doc hook requirement.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
771936b768 |
feat(alpha): --train-threshold for backtest + Phase E.3 honest verdict
Phase E.3 follow-up. Adds --train-threshold to alpha_compose_backtest so
the Q-network can be trained against a FIXED gate (instead of just
applying the gate at eval). Default 0.39 = the equilibrium the smoke's
controller stabilized to at ep 200+ (alpha_dqn_h600_smoke gated run).
Smoke result (gated training, controller running):
ep 100: thresh=0.32 obs=0.226 R_mean=-5.5 atten=0.75
ep 200: thresh=0.38 obs=0.082 R_mean=-3.0 atten=0.50
ep 300: thresh=0.39 obs=0.081 R_mean=-3.2 atten=0.25
ep 1000: thresh=0.39 obs=0.039 R_mean=-4.7 atten=0.10
The controller CONVERGES cleanly to threshold ≈ 0.39 with observed
trade rate at/below the 0.08 target. rollout_R_mean drops from -19
(no-gate training) to -4.7 (gated training): 4× less loss per episode.
rvr stays at +1.045σ (unchanged). The closed-loop architecture works
end to end.
(Note: smoke verdict FAILs on ACTION_ENTROPY (0.68 < threshold 1.10).
This is the policy correctly Waiting 95%+ of the time — the kill
criterion was designed to catch "collapse to one bad action," but
collapse-to-Wait under a strong gate is the RIGHT behavior. Verdict
threshold is misaligned with the gated paradigm; not a regression.)
Backtest result with --train-threshold 0.39:
cost eval-gate only train+eval gated Δ
------ -------------- ---------------- ----
0.0000 -15.72 -17.06 -1.3
0.0625 -21.30 -22.91 -1.6
0.1250 -29.17 -31.26 -2.1
0.2500 -42.12 -36.68 +5.4
0.5000 -54.86 -53.83 +1.0
Training with the gate did NOT meaningfully improve absolute Sharpe.
The eval-best threshold remains 0.20-0.25 in BOTH runs (not 0.39).
The Q-network's primary contribution is the binary trade/don't-trade
decision; the action-choice (Buy direction + placement) is largely
determined by alpha sign — linear Q can't time entry better than the
threshold filter does on its own.
Honest analysis: the gap to Phase 1d.4 baseline (+4.4 at cost=0,
-4.0 at half-tick) is NOT architectural but ECONOMIC:
Env spread: bid/ask synthesized at ±0.125-tick around mid
→ round-trip spread cost = 0.25 per trade
At τ=0.20 with 168 trades/ep: 168 × 0.25 = 42 in spread costs
Mean reward = -5 → alpha extracts ~37 of value
All eaten by spread
Phase 1d.4 baseline likely trades much less (~20-50 trades/ep at best
operating point — pure threshold-only policy, no RL). Our policy
trades 3-8× more because the DQN's action choices add fine-grained
trade attempts beyond the threshold filter's wait/trade gate.
The control loop architecture (Phase E.1 + E.2 + E.3 gate consumption)
is VALIDATED — gate produces monotone Sharpe lift, +1.045σ rvr held,
trade-rate-self-correction converges cleanly. But beating Phase 1d.4's
absolute Sharpe requires:
1. MLP for the Q-network (more representation capacity for
entry-timing decisions within the alpha confidence band)
2. OR action-space constraints (collapse the 9-action space — drop
fine-grained L1/L2 placement, keep just {Wait, BuyMarket,
SellMarket, FlatMarket})
3. OR better fill economics (real LOB instead of fixed ±0.125-tick
synthesis)
These are Milestone E.3 follow-up work (Tasks 24-28 sweeps + future
architectural changes). The composition backtest validated what it
was designed to: the cost-edge frontier of the linear Q + Phase 1d.3
alpha + controller setup, and surfaced the next architectural
question (representation capacity vs action-space size vs fill
realism).
Branch: sp20-aux-h-fixed, pushed.
|
||
|
jgrusewski
|
a36ad53a57 |
feat(alpha): wire slot 543 consumption — 2D threshold × cost sweep
Phase E.3 Task 23 follow-up. Adds the confidence-threshold gate that
consumes the controller's ISV[543] output. Both binaries:
fn epsilon_greedy_gated(q, alpha_confidence, threshold, eps, rng) -> u8 {
if alpha_confidence < threshold { return 0; /* Wait */ }
epsilon_greedy(q, eps, rng)
}
State[1] is the env's alpha_confidence = |sigmoid(alpha_logit) - 0.5|
which is in [0, 0.5]; threshold is also clamped [0, 0.5], so direct
comparison is valid.
alpha_dqn_h600_smoke (closed-loop with controller):
Adds current_threshold: f32 cache, initialised to 0.0 (no gate),
refreshed via stream.clone_dtoh(&isv_dev) after each per-episode
controller invocation. Action selector reads current_threshold for
the NEXT episode's step decisions.
alpha_compose_backtest (2D sweep):
Adds --threshold-grid CLI flag (default [0.0, 0.05, 0.10, 0.15, 0.20,
0.25] — Phase 1d.4 pattern). Eval loop becomes 2D (threshold × cost).
Per-bin includes avg_n_trades for trade-rate visibility. End-of-run
prints BEST per-cost = max Sharpe_ann across τ.
Results (1000 train ep, 300 eval ep × 5 τ × 5 costs):
cost τ=0.00 best τ Sharpe lift trades/ep saved
------- ---------- --------- ----------- ---------------
0.0000 -41.78 -15.72 (τ=0.20) +26.1 477 → 168 (-65%)
0.0625 -71.46 -21.30 (τ=0.25) +50.2 476 → 138 (-71%)
0.1250 -86.78 -29.17 (τ=0.20) +57.6 482 → 167 (-65%)
0.2500 -108.57 -42.12 (τ=0.25) +66.5 480 → 132 (-73%)
0.5000 -146.76 -54.86 (τ=0.25) +91.9 478 → 136 (-72%)
Win rate at cost=0: 7.7% (no gate) → 20.3% (τ=0.20).
The gate architecture is VALIDATED: monotone improvement in win rate +
Sharpe + trade-rate reduction across all costs. The control loop
(controller → slot 543 → policy gate → observed rate feedback) is
sound. But the policy is STILL negative-Sharpe at every cost.
Phase 1d.4 baseline at half-tick: -4.0 (ours: -29.17). 25-pt gap.
Root cause of the remaining gap: the Q-network was TRAINED without
gate awareness. It learned Q-values for the over-trading regime. The
eval-only gate filters those decisions but can't fix miscalibrated
Q-values. Phase 1d.4 baseline beats us because its policy
(always-market-when-confident) is INHERENTLY gated by design — no
mismatched Q-values to fix.
Next iteration to close the 25-pt gap: train WITH gate on, so the
Q-network learns weights for the gated policy class. This means:
either (a) controller runs during training (smoke pattern) and the
threshold develops endogenously, or (b) fixed --train-threshold CLI
during training. Either way, the Q-network sees Wait-at-low-confidence
during the learning phase and adapts.
Files touched:
crates/ml/examples/alpha_dqn_h600_smoke.rs (gate + threshold cache)
crates/ml/examples/alpha_compose_backtest.rs (gate + 2D sweep)
config/ml/alpha_compose_backtest.json (2D verdict)
|
||
|
jgrusewski
|
2af8e02fd8 |
feat(alpha): Phase E.3 composition backtest — reveals slot 543 needs consumption
Phase E.3 Task 23. Trains the Phase E execution-policy DQN on the first
80% of fxcache snapshots, then evaluates the frozen policy (ε=0) on the
held-out 20% across a transaction-cost sweep. Compares absolute Sharpe
vs the Phase 1d.4 always-market-when-confident baseline.
Pipeline pieces:
- Shared loaders extracted into crates/ml/src/env/loaders.rs (used by
both alpha_dqn_h600_smoke and alpha_compose_backtest)
- alpha_compose_backtest.rs: train DQN on first n_train bars, then
frozen-eval n_eval episodes per cost level
- cost grid: [0.0, 0.0625, 0.125, 0.25, 0.5] (price units per
contract round-turn)
- Annualised Sharpe via per-episode Sharpe × sqrt(episodes/year)
where episodes/year ≈ 252 · 6.5h · 3600s / (horizon · 12s)
Run (horizon=600, 1000 train ep, 500 eval ep/cost, 1.5M snapshots):
cost n_ep mean_R std_R Sharpe/ep Sharpe_ann win_rate
0.0000 500 -11.09 8.03 -1.380 -39.50 0.090
0.0625 500 -20.68 9.88 -2.093 -59.89 0.012
0.1250 500 -29.38 9.49 -3.095 -88.58 0.000
0.2500 500 -48.23 11.90 -4.052 -115.98 0.000
0.5000 500 -84.59 17.43 -4.854 -138.92 0.000
Phase 1d.4 baseline for comparison: +4.4 ann. at cost=0, -4.0 at half-tick.
The Phase E policy LOSES MONEY across the whole cost grid — even at
frictionless cost=0. This is not a contradiction with the H=600 PASS
verdict (rvr=+1.04σ): the smoke's rvr is RELATIVE TO RANDOM, while
backtest Sharpe is ABSOLUTE. "Better than random by 1 std" is still
losing if random loses big.
The diagnostic that the E.2 controller already surfaced:
ISV[543] STACKER_THRESHOLD saturated at upper clamp (0.5) — policy
trades 85% of the time vs the 8% target. Over-trading pays spread on
every bar regardless of alpha confidence. Even with perfect alpha
(Phase 1d.3 AUC=0.673), trading 85% × spread cost > alpha edge.
The Phase 1d.4 baseline beats us at cost=0 because it WAITS unless
|stacker_logit| > threshold — the threshold gate filters bars with
weak alpha signal. The Phase E controller PRODUCES slot 543 but the
DQN's action selection doesn't CONSUME it.
This is exactly what the E.3 backtest is FOR: revealing that the
Phase E.1/E.2 producer-side architecture without consumer-side gating
is incomplete. The composition backtest validates the architecture's
weak link.
NEXT (E.3 task 24-28 or a side fix): wire slot 543 consumption into
the action selection. At each step:
if |ISV[543] − 0.5| > |stacker_logit − 0.5|:
action = Wait // confidence below threshold, sit out
else:
action = argmax(Q)
Or equivalently: action = if confidence_high(alpha_logit, ISV[543])
{ argmax(Q) over Buy/Sell actions } else { Wait }.
Once slot 543 is consumed, re-run alpha_compose_backtest and expect
Sharpe to move toward / past the Phase 1d.4 baseline.
Loader refactor: extracted load_fill_model_from_json, load_alpha_cache,
load_snapshots_from_fxcache from alpha_dqn_h600_smoke.rs into
crates/ml/src/env/loaders.rs. The smoke now calls the shared module
via ml::env::loaders::*. ~150 lines of duplicated code removed.
Build + run verified: smoke still builds clean. Backtest runs in ~30s
(train 8s + eval 20s + setup).
Branch: sp20-aux-h-fixed, pushed.
|
||
|
jgrusewski
|
91383507fc |
feat(alpha): wire stacker-threshold controller into smoke rollout-end
Phase E.2 Task 17. Loads stacker_threshold_controller.cubin at smoke
startup, initialises ISV[544] (TRADE_RATE_TARGET) to 0.08 (CLI flag
--trade-rate-target, never reset), allocates a 3-float Wiener state
buffer for slot 545's Pearl A+D state.
Invokes the controller at every episode end with:
rollout_trade_count = count of non-Wait actions in the episode
rollout_total_decisions = actions_host.len() (= ep_len)
rollout_realized_sharpe = ep_terminal_R / RANDOM_BASELINE_STD
(per-rollout analog of the rvr metric;
lets the Kelly-atten controller respond
to in-policy performance vs the baseline
noise floor)
CLI args added:
--trade-rate-target default 0.08 (8% per-step trade rate target)
--k-threshold default 0.01
--k-atten default 0.005
--target-sharpe default 0.5
--wiener-alpha-floor default 0.4
--ctl-alpha-meta default 0.1
Periodic log line extended:
ep ... | KC q/H/rvr/ΔQ ... | CTL thresh=... obs=... atten=...
Final JSON adds:
final_stacker_threshold
final_trade_rate_observed_ema
final_stacker_kelly_attenuation
trade_rate_target
Smoke run (H=600, 1000 episodes) verifies the controller is alive:
ISV[543] STACKER_THRESHOLD: 0.000 → 0.5000 (saturated at ceiling)
ISV[545] TRADE_RATE_OBSERVED_EMA: 0.000 → 0.712
ISV[546] STACKER_KELLY_ATTENUATION:0.000 → 0.100 (hit floor)
Verdict: PASS — rvr=+1.043σ (unchanged from Task 12b PASS, expected
since smoke doesn't yet CONSUME slots 543/546).
Tuning notes (calibration for production, not bugs):
• Threshold saturating at 0.5 → policy trades ~85% (target 8%, off by
10×). Either re-calibrate target_trade_rate from realistic backtest
behaviour, or raise the clamp ceiling. Current ε-greedy with low
threshold-consumption gate produces high trade rate.
• Kelly atten hit floor (0.1) because rollout_sharpe (~-0.004) is far
below target_sharpe=0.5. The target needs to match the rollout
metric's scale, OR the metric should be time-normalised. The
current ep_terminal_R / baseline_std proxy is meaningful but its
scale doesn't match a typical annualised Sharpe target.
These tuning items don't gate Milestone E.2 — the producer-side
controller is correctly driving the ISV slots; *consuming* those slots
(threshold gate on alpha signal, Kelly-cap multiplier) is Phase E.3
work (alpha + execution composition).
Phase E.2 Tasks 16 + 17 close-out: kernel + launcher + GPU smoke test
+ wired into smoke binary + initialisation + verified end-to-end. Tasks
19-22 (NoisyNet) are gated on Task 12 FAIL, which we passed — skipped.
Task 18 (alpha-trust ablation, ~9-18 hours compute) deferred to a
dedicated session if needed.
|
||
|
jgrusewski
|
85792ed28a |
feat(alpha): stacker-threshold + Kelly-attenuation controller kernel
Phase E.2 Task 16. Engagement-rate self-correcting controller per
pearl_engagement_rate_self_correction. Single-block, single-thread
kernel; runs once per rollout-end boundary.
ISV slots driven:
543 STACKER_THRESHOLD_INDEX clamp [0, 0.5] P-controller on rate
545 TRADE_RATE_OBSERVED_EMA_INDEX Pearl A+D floored Wiener-α
546 STACKER_KELLY_ATTENUATION_INX clamp [0.1, 1.0] P-controller on Sharpe
Reads ISV[544] TRADE_RATE_TARGET_INDEX (TrainingPersist anchor, set once
at training start).
Control law:
observed = trade_count / max(decisions, 1)
ISV[545] ← Pearl_A+D_floored(observed, prev, x_lag)
[α* floor = 0.4 per pearl_wiener_alpha_floor_for_nonstationary;
controller co-adapts with policy → need responsive EMA]
err_rate = ISV[545] - ISV[544]
ISV[543] ← clamp(0, 0.5, ISV[543] + k_threshold · err_rate)
err_sharpe = rollout_sharpe - target_sharpe
ISV[546] ← clamp(0.1, 1.0, prev_atten + k_atten · err_sharpe)
where prev_atten = 1.0 if ISV[546] == 0.0 (sentinel-start)
else ISV[546]
Wiener-α is INLINE (not via canonical apply_pearls_ad_kernel chain)
because the EMA is part of the control loop, not a separate diagnostic
slot. Lower latency, fewer kernels per step.
Floor at 0.1 on Kelly attenuation per
pearl_blend_formulas_must_have_permanent_floor — can't be 0, would
zero out position sizing permanently.
Pub launcher `launch_stacker_threshold_controller` in alpha_kernels.rs
with full safety asserts. Slot indices passed as i32 args (decouple
slot numbering from kernel).
GPU smoke test `stacker_threshold_controller_smoke_matches_hand_
computation` verifies 2-iteration sequence:
Iter 1 (Pearl A): observed=0.30 → ISV[545]=0.30; ISV[543]: 0.05 → 0.052
Iter 2 (Pearl D): observed=0.05 → ISV[545]=0.175 (α* hit floor 0.5)
ISV[543]: 0.052 → 0.05275
Both within 1e-5 tolerance. Anchor slot 544 unchanged.
`cargo test -p ml --lib alpha_kernels`: 6 pass (compile witness + 5 GPU
smokes including this one) on RTX 3050 Ti in 2.18s.
Audit doc docs/isv-slots.md updated per Invariant 7.
|
||
|
jgrusewski
|
5c0bcb1fdb |
fix(alpha): MBP-10 parser full-levels copy + fit_poisson L2 regularization
Two carried-over limitations from Phase E.0 / E.1 fixed and verified.
1. MBP-10 parser bug fix (`parse_mbp10_streaming` + `parse_mbp10_file`)
The DBN crate's `Mbp10Msg` carries the FULL post-update top-10 book
in `levels: [BidAskPair; 10]` per message — not just the single
update event's price/size. Previously the parser only called
`update_level(0, ...)` with the update event's fields, leaving
`current_snapshot.levels[1..10]` at default-empty. Downstream:
- OFI calculator reading L2-L5 got zeros → produced wrong OFI
features (the canonical Phase 1c/1d 81-dim feature stack has
multi-level OFI as features 0..5; with the bug these were
constant zero).
- microprice (`snapshot.levels[1]`) got zeros.
- FillModel L2/L3 fit observations got zeros, so L2/L3
coefficients were undefined (we worked around by replicating
L1 with attenuated intercept).
Fix: after `update_level(0, ...)`, copy fields from
`mbp10.levels[lvl]` into `current_snapshot.levels[lvl]` for `lvl
in 1..max_lvl`. Field-by-field copy preserves the existing scale
convention (raw 1e9 fixed-point i64). Applied to both streaming
and async file-parse code paths.
Comment "For simplicity, store all updates in level 0 / A full
implementation would maintain proper level ordering" removed.
2. fit_poisson L2 regularization
New `fit_poisson_l2(features, observed, max_iters, lr, l2_lambda)`
API (the old `fit_poisson` delegates with l2_lambda=0). L2 penalty
applies to slope coefficients β[1..5] but NOT to intercept β[0]
(penalizing the intercept biases toward p≈0.5 for all-zero-feature
samples, breaking the recovery test). Per-iteration update:
β[0] -= lr · grad[0] / n (intercept)
β[k] -= lr · (grad[k] / n + λ · β[k]) (slope, k ∈ 1..5)
Canonical motivation: on real 5.2M-trade ES.FUT data the
unregularized fitter converged to β_spread ≈ -40 (Task 5c commit
|
||
|
jgrusewski
|
cd5aa3402b |
feat(alpha): wire Phase 1d.3 stacker into smoke — H=600 VERDICT PASS
Phase E.1 Task 12b complete. The H=600 DQN smoke now consumes real
alpha_logit from the Phase 1d.3 stacker (Mamba2 + 7-input MLP stacker
trained for AUC=0.673 on test), and PASSES all four kill criteria:
Q_SPREAD_EMA = 10.92 ≥ 0.05 PASS
ACTION_ENTROPY_EMA = 1.97 ≥ 1.099 PASS
RETURN_VS_RANDOM_EMA = +1.043 ≥ 0.0 PASS ← jumped +3.62σ
EARLY_Q_MOVEMENT_EMA = 0.099 ≥ 0.01 PASS
Overall: PASS (H=6000 scale-up VIABLE)
Before/after comparison (same env, same DQN, only alpha_logit changed):
alpha_logit=0 alpha_logit=Phase1d.3
rollout_R_mean (final) -18,272 -18
RETURN_VS_RANDOM_EMA -2.58σ +1.04σ
Overall verdict FAIL PASS
The 1000× reduction in episode loss + the +3.62σ rvr swing definitively
proves the "first-best-action lock-in" hypothesis from the previous FAIL
analysis was a SYMPTOM, not the cause. The cause was alpha_logit=0
placeholder starving the policy of directional signal. With real Phase
1d.3 alpha, the linear Q-network learns to use it cleanly — no
NoisyNet, no MLP, no architectural change needed.
Integration pieces in this commit:
1. Cargo workspace registration: ml-alpha added as a workspace dep,
ml's manifest now depends on ml-alpha for FxCacheReader access.
(ml-alpha already depends only on ml-core, so no circular risk.)
2. alpha_dqn_h600_smoke.rs: two new CLI args
--fxcache-path <PATH> load snapshots from precomputed fxcache
(mid from raw_close, bid/ask synthesized
at fixed half-tick, 81-dim features extracted
for spread_bps / l1_imbalance / ofi / mid_drift)
--alpha-cache <PATH> load Phase 1d.3 stacker logit cache produced
by `alpha_train_stacker --alpha-cache-out`.
Each cache entry aligns to the corresponding
fxcache bar, populates SnapshotRow.alpha_logit
(and derives alpha_confidence = |sigmoid(z)-0.5|).
3. Snapshot source selection: in main(), --fxcache-path takes priority
when both paths are set; --alpha-cache requires --fxcache-path
(alignment guarantee). Original --mbp10-dir path unchanged for
non-cached runs.
4. Two new helper fns: load_alpha_cache (binary [u32 n] + [f32; n]
reader), load_snapshots_from_fxcache (FxCacheReader → Vec<SnapshotRow>
with synthesized bid/ask and alpha_logit/alpha_confidence from cache).
alpha_logits_cache.bin (7.6 MB, 1.97M f32 entries) is .gitignore'd —
regenerable from `cargo run -p ml-alpha --release --example
alpha_train_stacker -- --fxcache-path <FXC> --alpha-cache-out
config/ml/alpha_logits_cache.bin` (~2 min on RTX 3050 Ti).
Reproduction of this PASS verdict:
cargo run -p ml --release --example alpha_dqn_h600_smoke -- \
--fxcache-path /home/jgrusewski/Work/foxhunt/test_data/feature-cache/9297....fxcache \
--alpha-cache config/ml/alpha_logits_cache.bin \
--horizon 600 --n-episodes 1000
Total run time ~10s after fxcache load. Verdict + per-checkpoint KC
trajectory in config/ml/alpha_dqn_h600_smoke.json.
NEXT: Task 13 — scale to H=6000 (the production horizon). Per the plan,
PASS at H=600 unlocks H=6000.
|
||
|
jgrusewski
|
8958637c77 |
feat(alpha): stabilize alpha_dqn_h600_smoke — reward norm + target net + grad clip
Three stabilizers applied to the H=600 DQN smoke after initial run showed
unstable training (early_mvmt=2268× at lr=1e-6, NaN at lr=1e-4):
1. Reward normalization (--reward-scale, default 1000)
Rewards divided by scale BEFORE the Munchausen target. TD error
drops from ~1000 (raw reward magnitude at H=600) into O(1) target /
gradient / weight-update scale. Action selection + rollout-R
reporting use ORIGINAL rewards (so rvr math stays correct against
the Task 7c baseline).
2. Target network (--target-update-every, default 10 episodes)
Separate w_target_dev / b_target_dev buffers. Q_next(s') forward
uses target weights; SGD updates online only. Hard-update copies
online → target every K episodes. Breaks the V_soft(s') chase-its-
own-tail divergence of online-only Munchausen.
3. Gradient clipping (--grad-clip, default 1.0)
New `alpha_clip_inplace_kernel` in alpha_linear_q.cu (element-wise
clamp). Applied to dW and db after grad, before SGD. Safety net.
Diagnostic fix: weight_norm was direction-insensitive — orthogonal
rotations don't change ||W||_F, so early_mvmt read ≈0 even when training.
Switched to weight_distance_from_init = ||W_now − W_init||_F +
||b_now − b_init||_F (captures rotation). q_early = q_init + distance
so kernel's |q_early − q_init| / |q_init| ratio = distance / ||W_init||_F.
With lr bumped back up to 1e-4 (default for the stabilized config),
verified at horizon=100, n_episodes=200:
Q_SPREAD_EMA = 23.64 (≥ 0.05) PASS
ACTION_ENTROPY_EMA = 1.86 (≥ 1.0986) PASS
RETURN_VS_RANDOM_EMA = +0.586 (≥ 0.0) PASS
EARLY_Q_MOVEMENT_EMA = 0.0212 (≥ 0.01) PASS
Overall: PASS (H=6000 scale-up VIABLE)
early_mvmt grew monotonically (0.005 → 0.021) across the 200-episode
run — direction-sensitive diagnostic confirms genuine policy learning.
Audit doc docs/isv-slots.md updated per Invariant 7.
Next: H=600 / 1000-episode run on full data; if PASS holds, Task 13
(H=6000 scale-up) unlocks.
|
||
|
jgrusewski
|
fa30c2dd66 |
feat(alpha): alpha_dqn_h600_smoke — runnable Task 12 DQN smoke
Phase E.1 Task 12. Linear Q-network (W [9×10] + b [9], no hidden layer)
trained with ε-greedy + Munchausen target on the Phase E ExecutionEnv.
End-to-end runnable: load env, train, periodically launch
alpha_kill_criteria + apply_pearls_ad chain at episode boundaries, emit
PASS/FAIL verdict against the 4 kill criteria thresholds.
Pipeline per training step (all on GPU):
1. forward Q_current on s_batch via alpha_linear_q_forward
2. forward Q_next on s'_batch via alpha_linear_q_forward
3. alpha_munchausen_target → targets[batch]
4. alpha_linear_q_grad → dW, db (sparse over taken actions)
5. alpha_linear_q_sgd_step on W and b (separate launches)
6. every K episodes: kill_criteria + apply_pearls_ad chain → ISV[539..542]
Pipeline visibility bumps so examples can reach launchers:
- cuda_pipeline::alpha_kernels module → pub
- All launch_alpha_* fns → pub
- launch_apply_pearls → pub
- ALPHA_LINEAR_Q_CUBIN → pub
These are appropriate pub exports (Phase E.1 public API surface).
Initial micro-smoke (horizon=100, n_episodes=50, lr=1e-6):
Q_SPREAD_EMA = 3.12 (≥ 0.05) PASS
ACTION_ENTROPY_EMA = 2.12 (≥ 1.0986) PASS
RETURN_VS_RANDOM_EMA = +1.03 (≥ 0.0) PASS
EARLY_Q_MOVEMENT_EMA = 2268 (≥ 0.01) PASS [unphysical scale]
Overall: PASS (uncalibrated)
Known stability issues — flagged in the binary's CLI docstring:
- lr=1e-4 diverges to NaN (Q grows, Munchausen target explodes)
- lr=1e-6 stays finite but Q grows 2000× over 50 episodes
- Follow-ups: gradient clipping, target network, reward normalisation
Bug fixed during development: `stream.memcpy_htod(&host, &mut buf.clone())`
was uploading to a TEMPORARY clone (dropped immediately) — `kc_scalar_dev`
and `kc_action_counts_dev` never got their host data → entropy=0, early_mvmt=0,
rvr stuck at the alloc-zeros default. Fixed by removing `.clone()` and using
direct `&mut` refs.
Reads:
config/ml/alpha_fill_coeffs.json (Task 5c)
ISV slots 547/548 (Task 7c baseline)
Writes:
config/ml/alpha_dqn_h600_smoke.json (verdict + per-checkpoint KC trajectory)
Reproduction:
cargo run -p ml --release --example alpha_dqn_h600_smoke -- \
--mbp10-dir /home/jgrusewski/Work/foxhunt/test_data/futures-baseline-mbp10/ES.FUT \
--horizon 600 --n-episodes 1000
Audit doc docs/isv-slots.md updated per Invariant 7.
|
||
|
jgrusewski
|
36ab50814e |
feat(alpha): alpha_linear_q kernels + launchers for Task 12 DQN smoke
Phase E.1 Task 12a. Three new CUDA kernels for the H=600 DQN smoke
(Task 12 proper) that lands in a follow-up commit:
alpha_linear_q_forward_kernel Q = X · W^T + b
alpha_linear_q_grad_kernel dW, db sparse MSE-TD over taken actions
alpha_linear_q_sgd_step_kernel element-wise params -= lr · grad
Architecture: single linear layer, no hidden layer. The Phase E state
vector has meaningful direct features (alpha_logit, spread_bps, position,
ofi_sum_5, …) so linear Q can capture real relations like Q[Buy] ∝
alpha_logit. If linear can't pass the kill-criteria gate, no architecture
upgrade will save it — and the smoke proceeds with NoisyNet escalation
per the plan.
Sparse gradient: only the taken action contributes (standard DQN TD
loss). No atomicAdd needed — one thread per (i, j) loops over the batch
and adds only when actions[b] == i.
GPU contract:
- No host branches inside any kernel (graph-capture compatible)
- No atomicAdd (per feedback_no_atomicadd)
- All compute on GPU (forward, grad, weight update)
- Tiny launch overhead — fits per-step (batch=64 forward = 576 threads,
1 block; grad = 99 threads, 1 block)
Three pub(crate) Rust launchers in alpha_kernels.rs match the
launch_apply_pearls pattern. Cubin embedded via include_bytes!.
Smoke test `linear_q_forward_grad_sgd_round_trip_matches_hand_math`
exercises all three kernels end-to-end on a small (batch=2, state_dim=2,
n_actions=3) case with full hand-math:
Forward: Q = [[2.1, 3.2, 0.3], [4.1, 5.2, 0.3]] ✓
Grad: dW = [[-1.8, -2.7], [0.8, 1.0], [0, 0]]
db = [-0.9, 0.2, 0] ✓
SGD: W' = [[1.18, 0.27], [-0.08, 0.90], [0, 0]]
b' = [0.19, 0.18, 0.30] ✓
All within 1e-4 tolerance. `cargo test -p ml --lib alpha_kernels`:
5/5 pass on RTX 3050 Ti in 2.04s (compile witness + 4 GPU smokes).
Audit doc docs/isv-slots.md updated per Invariant 7.
|
||
|
jgrusewski
|
ebbd28437a |
test(alpha): chained pipeline smoke — Task 12 wiring validation
End-to-end test for the kernel COMPOSITION that the H=600 DQN smoke
(Task 12 proper) will use at each rollout boundary:
t+0 alpha_kill_criteria_compute_kernel → scratch[0..4]
t+1 apply_pearls_ad_kernel(n_slots=4) → ISV[539..543]
The two prior alpha_kernels smokes (munchausen + kill_criteria) validated
kernels in isolation. This smoke validates the COMPOSITION on the same
stream — failures here are different (stream-ordering, Pearls index base,
Wiener offset base, scratch visibility) and would silently break Task 12.
Two iterations with stationary synthetic inputs:
Iter 1 (Pearl A bootstrap)
prev_x_mean=0 AND x_lag=0 → ISV[539..542] populated with raw scratch
observations = [0.2041, 0.8980, 1.0670, 0.1] within 0.01 tolerance.
Anchor slots 547/548 remain at Task 7c values (-5185, 4953).
Iter 2 (Pearl D stationary)
dx_mean = dx_step = 0 → α* = 0 → ISV unchanged from iter 1 within
1e-4. Stationary signal stays at the bootstrap value.
Test would catch:
- Producer's scratch write not visible to applicator (stream-ordering)
- Wrong Pearls isv_idx_base / wiener_offset_base
- Pearl A sentinel detection broken (formula yields 0 at t=0)
- Wiener state corruption (iter 2 drifts from iter 1)
Reuses launch_apply_pearls from sp4_wiener_ema.rs (pub(crate)). Helper
fn run_chained_iter factors the producer→applicator sequence so the two
iterations are byte-identical apart from the Wiener state's evolution.
`cargo test -p ml --lib alpha_kernels`: 4 pass (compile witness + 2 prior
GPU smokes + this chained smoke) on RTX 3050 Ti in 1.89s total. Audit doc
docs/isv-slots.md updated per Invariant 7.
Remaining Task 12 work (full H=600 DQN trainer integration with this
pipeline at rollout boundaries) is queued for a dedicated session.
|
||
|
jgrusewski
|
697bb586d0 |
test(alpha): kill-criteria GPU smoke matching hand-computed observations
Adds the second of the two Phase E.1 kernel smoke tests in
alpha_kernels.rs (companion to the munchausen_target smoke from
|
||
|
jgrusewski
|
91d1a52b9c |
refactor(alpha): rename phase_e_* → alpha_* — system-scoped naming
The kill-criteria producer, Munchausen target kernel, Rust launchers,
fit/baseline binaries, and their output JSON artifacts are *durable
infrastructure* of the alpha trading system (live across Phase E/F/G/...),
not milestone-scoped to Phase E specifically. Aligns with the earlier
`phase_e_isv_slots.rs` → `alpha_isv_slots.rs` rename rationale.
What was renamed:
Code files:
crates/ml/src/cuda_pipeline/phase_e_kill_criteria.cu → alpha_kill_criteria.cu
crates/ml/src/cuda_pipeline/phase_e_munchausen_target.cu → alpha_munchausen_target.cu
crates/ml/src/cuda_pipeline/phase_e_kernels.rs → alpha_kernels.rs
crates/ml/examples/phase_e_fit_fill_model.rs → alpha_fit_fill_model.rs
crates/ml/examples/phase_e_random_baseline.rs → alpha_random_baseline.rs
Artifacts:
config/ml/phase_e_fill_coeffs.json → alpha_fill_coeffs.json
config/ml/phase_e_random_baseline.json → alpha_random_baseline.json
Kernel function names:
phase_e_kill_criteria_compute_kernel → alpha_kill_criteria_compute_kernel
phase_e_munchausen_target_kernel → alpha_munchausen_target_kernel
Rust launcher names:
launch_phase_e_kill_criteria → launch_alpha_kill_criteria
launch_phase_e_munchausen_target → launch_alpha_munchausen_target
Static cubin names:
PHASE_E_MUNCHAUSEN_TARGET_CUBIN → ALPHA_MUNCHAUSEN_TARGET_CUBIN
Historical milestone tags in doc-comments ("Phase E.1 Task N (2026-05-15)")
are RETAINED — they record WHEN the work landed and what plan it
implemented, which doesn't change with the system-scoped rename.
Plus: ADDS the alpha_munchausen_target GPU smoke test in alpha_kernels.rs.
End-to-end validates the launcher + kernel against hand-computed expected
values: batch=2 with one terminal sample; expected targets [29.8, 1.1];
got match within 0.05 tolerance on RTX 3050 Ti. PROVES the Task 9/10
kernels actually run on GPU.
All affected references updated in:
- build.rs (kernel compile list)
- mod.rs (module registration)
- state_reset_registry.rs (4 RegistryEntry descriptions for slots 539-542)
- alpha_isv_slots.rs (slot table comment)
- docs/isv-slots.md (audit-doc cross-references)
Verified:
cargo test -p ml --lib alpha_kernels: 2/2 pass (including GPU smoke)
cargo test -p ml --lib state_reset_registry: 10/10 pass
cargo build -p ml --release --example alpha_fit_fill_model --example alpha_random_baseline: clean
|
||
|
jgrusewski
|
feb2e8cc34 |
feat(alpha): phase_e_kernels — Rust launchers for Task 9 + Task 10 cubins
Phase E.1 Task 11. Two pub(crate) launchers expose the kill-criteria
producer (Task 9) and Munchausen target augmentation (Task 10) for use
by future trainer integration:
launch_phase_e_kill_criteria(stream, kernel, q_values_dev, action_counts_dev,
scalar_inputs_dev, isv_dev, batch, n_actions,
scratch_out_dev)
→ kicks the kill_criteria producer; caller chains apply_pearls_ad_kernel
(n_slots=4, isv_idx_base=ALPHA_ISV_BLOCK_LO=539) to smooth into slots
539..542 via Pearl A bootstrap + Pearl D Wiener-α.
launch_phase_e_munchausen_target(stream, kernel, q_next_dev, q_current_dev,
actions_dev, rewards_dev, dones_dev,
gamma, alpha_m, tau, log_clip_min,
target_out_dev, batch, n_actions)
→ one-thread-per-sample target augmentation; α_m/τ/log_clip_min are
scalar args so a downstream ISV-driven controller can tune them.
Plan deviation: Task 11 plan-spec said "replace hardcoded n_step=32,
gamma=0.999 literals" but grep across gpu_dqn_trainer.rs found ZERO such
literals — the trainer already reads gamma via read_isv_signal_at(
GAMMA_DIR_EFF_INDEX) and epsilon via read_isv_signal_at(AUX_TRUNK_EPS_
INDEX). The actual E.1 deliverable was Rust launchers for the new
cubins, which this commit lands.
Both launchers follow the launch_apply_pearls pattern in
sp4_wiener_ema.rs — pre-loaded CudaFunction as parameter, u64 device
pointers, debug_assert! guards.
Audit doc docs/isv-slots.md updated per Invariant 7.
Tested via `cargo test -p ml --lib phase_e_kernels` — 1 compile-witness
passes. Real GPU integration test in Task 12.
|
||
|
jgrusewski
|
b1ba41d403 |
feat(alpha): Munchausen DQN target term kernel (Vieillard et al. 2020)
Phase E.1 Task 10. Standalone target-augmentation kernel: m = α_m · max(τ · log π(a|s), log_clip_min) V_soft(s') = max(Q_next) + τ · log Σ exp((Q_next − max) / τ) target = r + m + γ · V_soft(s') (terminal: r + m) π(a|s) ∝ exp(Q_online(s, a) / τ) — softmax policy from the online net. Munchausen bonus is implicit KL regularisation between successive policies; soft-V replaces the hard max bootstrap with a τ-weighted softmax average. Both softmaxes are computed via log-sum-exp with the max-trick. This is essential at τ ≈ 0.03 where raw exp(Q/τ) would overflow f32 for any Q-spread > 25 nats. The kernel is one-thread-per-batch-sample, no atomicAdd, no host branches. α_m, τ, log_clip_min are kernel args (not hard-coded), so a Phase E.2+ controller can ISV-drive them. Typical Vieillard values: α_m=0.9, τ=0.03, log_clip_min=-1.0. Does NOT touch any ISV slot — pure target augmentation. Cubin: target/release/build/ml-*/out/phase_e_munchausen_target.cubin (12.8 KB). Launcher integration is Task 11 (consumes target_out where the C51/MSE loss kernels currently consume `r + γ · max_a' Q_target`). Audit doc docs/isv-slots.md updated per Invariant 7. |
||
|
jgrusewski
|
d493b729bf |
feat(alpha): phase_e_kill_criteria producer kernel (slots 539-542)
Phase E.0 Task 9. Single-block, single-thread CUDA producer that writes 4
raw scalar observations into a contiguous scratch_out[0..4] block:
scratch_out[0] → ISV[539] Q_SPREAD_EMA (kill: ≥ 0.05)
scratch_out[1] → ISV[540] ACTION_ENTROPY_EMA (kill: ≥ 0.5·ln(9) ≈ 1.10)
scratch_out[2] → ISV[541] RETURN_VS_RANDOM_EMA (kill: ≥ 0, i.e. ≥ random)
scratch_out[3] → ISV[542] EARLY_Q_MOVEMENT_EMA (kill: ≥ 0.01, learned)
Downstream `apply_pearls_ad_kernel` (n_slots=4) chained on the same stream
applies Pearl A first-observation bootstrap + Pearl D Wiener-α smoothing —
composes with the canonical val_sharpe_delta_compute_kernel pattern rather
than reimplementing Wiener math (per feedback_no_cpu_compute_strict — one
Wiener implementation in the codebase, the GPU one).
Inputs:
q_values[batch, n_actions] most-recent Q forward output (device)
action_counts[n_actions] empirical action histogram (device)
scalar_inputs[3] [rollout_R_mean, q_init_norm, q_early_norm]
via mapped-pinned (host writes between rollouts)
isv reads slots 547 + 548 (random baseline
mean/std — populated once by Task 7c,
TrainingPersist)
No atomicAdd (per feedback_no_atomicadd), no host branches
(per pearl_no_host_branches_in_captured_graph), no CPU compute
(per feedback_cpu_is_read_only). __threadfence_system() before exit for
the chained Pearls applicator's visibility guarantee.
Cubin produced: target/release/build/ml-*/out/phase_e_kill_criteria.cubin
(16.8 KB).
Launcher integration is Task 11 (separate commit so this task can stand
alone for cubin validation). Audit doc docs/isv-slots.md updated per
Invariant 7.
|
||
|
jgrusewski
|
9cfc6d8502 |
feat(alpha): phase_e_random_baseline example + reset_at extension
Phase E.0 Task 7b. Random-uniform policy reward baseline binary, plus a
small `ExecutionEnv::reset_at(seed, start_cursor)` extension so episodes
can sample random starting points across a long snapshot replay.
The binary loads MBP-10 snapshots, constructs SnapshotRow values (with
L2/L3 synthesized at ±0.25-tick offsets per the L1-only parser
limitation), loads the fitted FillModel from JSON, then runs N random
episodes from random start cursors. Reports mean / std / quintile
percentiles + kill threshold (mean + 2σ) for E.1 to exceed.
Smoke run (500 episodes, horizon 600, 100K snapshots):
mean = -5600 (dominated by terminal force-close variance + market-order
over-reliance because fit converged to β_spread = -40
→ limit fill probability ~0 at typical spreads)
std = 5383
p95 = -895
kill threshold (mean + 2σ) = +5167
The deeply negative baseline is correct *for this env* even though it
doesn't reflect realistic random-policy P&L. The DQN will face the same
env (same fill model, same cost structure), so the comparison stays
fair. Fitter regularisation (to prevent β_spread runaway) is a Phase E.1
follow-up.
Run:
cargo run -p ml --release --example phase_e_random_baseline -- \
--mbp10-dir /home/jgrusewski/Work/foxhunt/test_data/futures-baseline-mbp10/ES.FUT \
--fill-coeffs config/ml/phase_e_fill_coeffs.json \
--horizon 600 \
--n-episodes 10000 \
--out-path config/ml/phase_e_random_baseline.json
env.reset_at also called by reset() (1-line refactor); no behavior change.
|
||
|
jgrusewski
|
12151ccf6a |
feat(alpha): fitted FillModel coefficients from 500K ES.FUT snapshots
Phase E.0 Task 5c. Ran phase_e_fit_fill_model on the ES.FUT 2024-Q1 MBP-10 + trade tape (5.2M trades, 3.9M MBP-10 events, 500K snapshots accumulated at snapshot_interval=50 over a ~24-minute window). Total runtime ~80s. Empirical fill rates within 60s window: - bid_l1: 4.97% (matches L1 maker-side activity in trending market) - ask_l1: 71.34% (high — most 60s windows see an aggressive buy) Fitted L1 cloglog coefficients (all 5 features): BID L1: β_0=-0.213 β_spread=-2.064 β_imbal=-0.099 β_ofi=-0.006 β_logτ=-0.286 ASK L1: β_0=+0.016 β_spread=-40.336 β_imbal=+0.041 β_ofi=+0.652 β_logτ=-0.055 Sanity (sign checks all pass): - β_spread < 0 both sides (wider spread → fewer fills) ✓ - bid β_imbal < 0 (more bid stack → harder to get hit by sell) ✓ - ask β_ofi > 0 (buying pressure correlates with ask fills) ✓ - β_logτ < 0 both sides (quieter markets → slower execution) ✓ L1-only limitation: as documented in the binary header, the parser only populates levels[0]; L2/L3 in the JSON are L1 with β_0 -= ln(L+1) attenuation. Default --out-path bumped to config/ml/phase_e_fill_coeffs.json so future re-runs land in the same committed location. |
||
|
jgrusewski
|
3bdf74018d |
feat(alpha): phase_e_fit_fill_model example — cloglog FillModel calibration
Phase E.0 Task 5b. Calibrates FillModel coefficients from historical MBP-10
+ trade tape. Streams snapshots concurrently with time-sorted trades; per
snapshot, determines binary fill outcome ("would a posted L1 limit have
been hit within next --window-seconds?"), accumulates (FillFeatures, y),
calls fit_poisson (cloglog Bernoulli, see
|
||
|
jgrusewski
|
a1e3336b1f |
feat(alpha): ExecutionEnv — 10-dim state, 9-action gating, terminal-only reward
Phase E Task 6. Core environment for the execution-policy DQN.
Episode lifecycle: position-open signal → up to `horizon_snapshots` →
forced close. Within the episode the policy chooses 9 actions; illegal
actions degrade to Wait inside step() (soft masking) — the network's
Q-output is never masked, preserving clean C51 and Munchausen targets.
Reward is **terminal-only**: realized PnL minus fees, accumulated over
all fills and force-closed at terminal mid if still open. No dense
per-snapshot shaping per pearl_event_driven_reward_density_alignment
(the canonical SP11→SP12 lesson — dense shaping on event-driven
objectives creates exposure-positive bias).
Bugs in the plan's sketch, caught + fixed during implementation:
1. Synthetic test's bid/ask were CONSTANT while mid drifted — would
have made the "uptrend" test lose money. Fixed: drift bid/ask
together with mid.
2. L1/L2 closing path used Side::None for fill lookup → 0% fill rate
for closing limits. Fixed: closing-long → Side::Sell (post at ask),
closing-short → Side::Buy (post at bid).
3. `cursor`-based hash for fill PRNG was non-deterministic across
replay seeds. Replaced with self-contained SplitMix64 RNG state
(no `rand` dep added) — fill randomness is now bit-identical across
train/eval replays at the same seed.
4. `max_step_per_episode` and `mid_at_decision` fields stored but
never read — dead per feedback_no_stubs. Removed.
5. Test harness used n==horizon, which made the cursor-exhaustion
guard (`cursor+1 >= len`) fire before the planned FlatMarket call.
Fixed by setting n > horizon so the meaningful `ep.step >= horizon`
path terminates the episode.
6. `.unwrap()` in test bodies → `.expect("…")` per pre-commit policy.
7. Volume-weighted entry_price across multi-fill scaling (the plan
only handled the open-from-flat case).
5 unit tests:
- state_has_expected_dim_and_is_finite (STATE_DIM=10, no NaN)
- market_buy_then_market_sell_on_uptrend_profits (PnL math)
- illegal_action_degrades_to_wait (soft mask, no fee, no fill counted)
- terminal_force_close_pays_out_when_position_open (force-close at mid)
- rng_is_deterministic_across_resets (same seed → identical fills)
`cargo test -p ml --lib env`: 16 passed (4 action_space + 7 fill_model
+ 5 execution_env); full module compiles clean with no new warnings.
|
||
|
jgrusewski
|
d08ab461db |
feat(alpha): fit_poisson via cloglog likelihood + Serde derives on FillCoeffs
Phase E Task 5 (code portion; the 5.2M-trade fit run lands separately).
Diagnosis: the plan's draft used pure Poisson NLL with Bernoulli y∈{0,1},
which converges to λ = empirical rate ȳ. But the runtime fill_prob() uses
`p = 1 − exp(−λ)`, so a trained λ=0.4 → predicted p=0.33 → systematic
~30pp under-fill bias on every passive backtest order. Training and
inference must agree on what λ means.
Fix: switch the fitter to the **cloglog (complementary log-log) binary
likelihood**. Per-sample gradient:
∂L/∂β_k = (p − y) · (μ/p) · x_k
where μ = exp(β·x), p = 1 − exp(−μ)
The μ/p factor is the link derivative; p.max(1e-7) handles the μ→0 limit
in f32 (the analytical limit μ/p → 1 is achieved automatically because
both numerator and denominator vanish proportionally).
Recovery test verifies the fix: 1000 deterministic 40% fills, all-zero
features → fitter recovers β_0 ≈ ln(0.5108) ≈ -0.672 (the value at which
1 − exp(−exp(β_0)) = 0.4), within tolerance 0.05. Slope coefficients
stay near zero (features uninformative).
Also adds Serde derives on FillCoeffs / FillFeatures / FillModel for JSON
serialization (downstream when the calibration example lands).
Deferred: the calibration example (Steps 3-7 of the plan task) is held
back until paired with the actual 5M-trade fit run — the plan's example
has a placeholder loop that violates feedback_no_stubs, and the loader
lift from precompute_features.rs deserves a dedicated commit.
4 new tests (7 total in env::fill_model now):
- fit_recovers_baseline_when_features_uninformative
- fit_rejects_empty_and_mismatched_inputs
- coeffs_round_trip_through_json
`cargo test -p ml --lib env::fill_model`: 7 passed.
|
||
|
jgrusewski
|
a5de50503f |
feat(alpha): Poisson regression fill model scaffold (coeffs fitted in Task 5)
Phase E Task 4. Medium-tier fill simulator per the design memo.
For each (level ∈ {L1,L2,L3}, side ∈ {Bid,Ask}):
λ(features) = exp(β · [1, spread_bps, L1_imb, OFI_5, log(τ+1)])
Per-snapshot Bernoulli fill probability for a posted limit order:
p = 1 − exp(−λ)
Market orders fill immediately at the opposite-side L1 quote (no slippage
modeled at medium tier). Closing actions (Side::None) return λ=0 — they
are handled separately in the env.
Scaffolding only. Task 5 fits the 30 coefficients (6 distributions × 5
features) from the 5.2M-trade historical tape. The skeleton constructor
uses β=0 → λ=1 → p≈0.632, a stable sanity default for early smokes.
Numeric guard: `linear.exp().min(50.0)` caps λ to prevent f32 overflow
under outlier features before fitting lands. Fitted models should stay
well below this cap in practice.
4 unit tests:
- skeleton_has_uniform_fill_prob (β=0 → p≈0.632 within 0.01)
- fill_prob_bounded_under_outlier_features
- lambda_cap_prevents_f32_overflow (β=100 outlier path)
- side_none_returns_zero_rate
`cargo test -p ml --lib env::fill_model`: 4 passed.
|
||
|
jgrusewski
|
94f8f65571 |
feat(alpha): 9-action discrete execution action space + legality gating
Phase E Task 3. Introduces crates/ml/src/env/ for the execution-policy
environment. This commit lands the action space only; fill_model (Task 4)
and execution_env (Task 6) append to mod.rs in their respective commits
per feedback_wire_everything_up (no orphan declarations).
- 9-action discrete space: Wait + {Buy,Sell,Flat} × {Market,L1,L2 / L1}
- Soft action masking: illegal actions degrade to Wait in env::step
(not by masking Q-output) per design memo — preserves clean C51
categorical targets and Munchausen term (Task 10)
- `is_legal(position)` with position ∈ {-1, 0, +1} (sign only;
magnitude is decoupled into the Kelly layer in Task 11)
- repr(u8) discriminants round-trip with from_u8 for replay-buffer
storage by the existing GPU DQN trainer
4 unit tests:
- n_actions_matches_enum_cardinality
- legality_gating_by_position (covers flat/long/short × all 9 actions)
- decode_closing_flag_is_only_flat
- repr_u8_round_trip (replay-buffer contract)
`cargo test -p ml --lib env::action_space`: 4 passed.
|
||
|
jgrusewski
|
8bf8bdf874 |
feat(alpha): state_reset_registry entries + dispatch arms for slots 539..548
Per feedback_registry_entries_need_dispatch_arms — both must land in the same commit; the test `every_fold_and_soft_reset_entry_has_dispatch_arm` walks training_loop.rs::reset_named_state source and asserts coverage. - 10 RegistryEntry rows appended after SP22 block (full producer/consumer rationale per existing dense-description pattern) - 7 dispatch arms in reset_named_state (the 3 TrainingPersist anchors — slots 544/547/548 — are not dispatched per the existing convention) - All sentinels 0.0 per pearl_first_observation_bootstrap All 10 registry tests pass. Audit doc docs/isv-slots.md updated per Invariant-7. |
||
|
jgrusewski
|
aa5908aa53 |
feat(alpha): reserve ISV slot block 539..550 for alpha trading system
12 contiguous slots reserved for durable alpha-system infrastructure: - 539-542: diagnostics (Q-spread, action entropy, return-vs-random, early-Q-movement EMAs) — read by Phase E kill criteria - 543-546: stacker-threshold controller (threshold, target, observed-rate, Kelly attenuation) — engagement-rate self-correction - 547-548: random-uniform baseline (mean, std) — anchor for kill criterion 541 - 549-550: reserved spare (absorb growth without bumping ISV_TOTAL_DIM) Named `alpha_isv_slots` rather than `phase_e_isv_slots` because these slots are intended to outlive any individual Phase E/F/G milestone — the SP4..SP22 naming was iteration-scoped, but this block is system-scoped infrastructure. ISV_TOTAL_DIM bumped to 551. All indices validated in 4 unit tests (bounds, membership, uniqueness, total-dim coverage). Audit doc docs/isv-slots.md updated per Invariant-7. |
||
|
jgrusewski
|
db874b1841 |
feat(foxhuntq): Phase 1c snapshot-resolution alpha + leakage fix + variable-dim fxcache
Three things landing atomically because they're load-bearing for each other: 1. **Trend-scanning leakage fix** — trend_scanning.rs was emitting OLS slope+t-stat over a *forward* window [t, t+L]. With the Phase 1a label = sign(price[t+60] − price[t]), the forward feature window overlaps the label window, contaminating it. Purged walk-forward only sterilizes forward-looking *labels* that cross the train/val split, not forward-looking *features* that peek inside the same horizon the label measures. The leak inflated MLP accuracy from 0.49 (legacy 74-dim baseline) to 0.75 — vanished to 0.50 after switching to a trailing window. Bounded the perfect-fit t-stat sentinel from ±1e6 → ±20 (p<1e-30 is already meaningless); eliminated the 16k corruption-cap drops. 2. **Variable-dim alpha column** — fxcache schema now carries the alpha-feature width via metadata (`alpha_feature_dim`), not a compile-time constant. Same on-disk format hosts the 134-dim bar-level stack OR the 81-dim snapshot stack. Reader + auto-detect honor the metadata-declared dim; downstream MLP auto-sizes `in_dim`. Single schema, no forks. 3. **Snapshot pipeline (Phase 1c falsification)** — `snapshot_pipeline.rs`: 81-dim per-MBP10-snapshot extractor reusing 10 snapshot-native alpha blocks + 6 new snapshot-specific features (time-since-trade, time-since-snap, event-rate, spread-bps, L1-imbalance, microprice-mid drift). `precompute_features` gets `--row-unit snapshot` flag; emits one fxcache row per LOB update (1.97M rows from MBP-10 data vs 206K for bar mode). **Smoke verdict on real data** (ES.FUT, 1.97M snapshots, 384K val): - Bar-level honest alpha: accuracy=0.5005, AUC=0.5043 (no signal) - **Snapshot-level alpha**: accuracy=0.5241, AUC=0.6849 (real signal, 384K val) - GBM corroboration: accuracy=0.5401 (non-linear partitioning sees more) - Horizon decay: alpha peaks at K=20-50 snapshots (~5-25ms), gone by K=500 - Regime-conditional: spread-Q4 quintile hits 0.752 accuracy on 76k samples Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
19bb3bc3c8 |
fix(sp22-vnext): aux_outcome CF half mirror on-policy (NOT all -1)
Smoke train-k95mj epoch 0 + epoch 1 printed
`trade_outcome_ce=0.000e0` — K=3 CE EMA stuck at Pearl A sentinel
across all training steps.
Root cause chain:
1. insert_batch is called with bs = total = 4_096_000 (cf-mult-
expanded), replay cap = 300_000 (L40S GpuProfile). Tail-clip:
off = 3_796_000, slice(off..) takes the last 300K elements.
2. Prior fix (
|
||
|
jgrusewski
|
878cc9ba72 |
feat(sp22-vnext): F-3c follow-up — K=3 CE EMA in stdout HEALTH_DIAG aux line
The F-3c commit (
|
||
|
jgrusewski
|
256a5fa5ab |
fix(sp22-vnext): K=3 aux_outcome_labels CF-half buffer underrun
Phase B4b-2 introduced `exp_aux_to_label_per_sample` sized
`[alloc_episodes × alloc_timesteps]` (no cf-mult expansion — the
B4b-1 per-step kernel only writes the on-policy half). The batch
finalisation cloned into the emitted `aux_outcome_labels` with size
`total = base_total × 2` (cf-mult expanded), so
`dtod_clone_i32(src, total, ...)` invoked `src.slice(..total)` on a
half-sized source → `CudaSlice::try_slice` returned `None` → the
internal `unwrap()` panicked at cudarc safe/core.rs:1648.
Repro: workflow train-xzv56 panicked after rollout completed
(timestep=999) on fold 0; rollout itself ran clean, the OOM from
the prior commit is gone.
Fix: replace the single `dtod_clone_i32` with a 3-step build:
1. alloc_zeros::<i32>(total)
2. cuMemsetD32Async(ptr, 0xFFFFFFFFu32, total, stream) — fills
all `total` slots with i32 mask sentinel -1 (byte pattern
0xFFFFFFFF reinterprets as i32(-1))
3. memcpy_dtod the first `base_total` real labels from
exp_aux_to_label_per_sample into the on-policy half
CF half remains at -1. The K=3 sparse-CE loss masks `label == -1`
out of the mean and B_valid count, so CF samples contribute zero
gradient — exactly the semantic we want (CF actions have no
observed trade-close outcome to predict against).
Lib suite: 1016/0 green. Audit doc updated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
0acf77e656 |
config(dqn): strip H100-tuned VRAM overrides from dqn-production.toml
dqn-production.toml hard-coded three H100-tuned values that shadow
GpuProfile auto-detection: batch_size=16384, buffer_size=500K,
gpu_n_episodes=4096. After the L40S-default flip lands (prior commit
|
||
|
jgrusewski
|
bbd52c3aa7 |
feat(sp22-vnext): FoldReset registry entries for B5b/C collector buffers
Two collector-side device buffers used by the K=3 trade-outcome head
were missing FoldReset registry coverage:
- prev_aux_outcome_probs [alloc_episodes × 3]
TRUE stale-read risk. Producer writes end-of-step; consumer
(experience_state_gather) reads start-of-next-step into
state[121..124). Without FoldReset the new fold's step-0 state
gather would inject the previous fold's last-step softmax probs
into the first batch's state slots.
- exp_aux_to_input_buf [alloc_episodes × 262]
Cleanliness-only. Concat kernel overwrites all 262 columns every
step before the K=3 forward reads them, so no steady-state stale-
read risk. Registered for parity with the rest of the K=3
pipeline + to satisfy feedback_registry_entries_need_dispatch_
arms (the pin test asserts every registry entry has a matching
dispatch arm in reset_named_state).
Both fields promoted to pub(crate) on GpuExperienceCollector so
reset_named_state can reach them. Matching dispatch arms added with
the standard memset_zeros pattern (is_win_per_env / hold_baseline_
buffer style).
Tests: All 10 state_reset_registry tests pass, including the critical
every_fold_and_soft_reset_entry_has_dispatch_arm pin test that walks
the dispatch body and validates parity with registry entries. Full
lib suite 1015/1 (the failing test is the pre-existing
test_dqn_checkpoint_round_trip NoisyLinear flake — pred1/pred2 sign
mismatch surfacing ~30-50% of full-suite runs, documented in
project_sp22_h6_vnext_resume memory as unrelated to this work).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
4b40710b7c |
feat(sp22-vnext): Phase B5b-2 collector trade plan forward — resolves K=3 asymmetry
Phase B5b's K=3 input concat passed plan_params=NULL because the collector had no trade plan launch. Trainer-side K=3 forward trained on real plan_params while the collector queried at plan_params=0 — documented train/inference asymmetry on the plan-conditioning surface. Phase B5b-2 mirrors the (now-corrected) trainer `launch_trade_plan_forward` chain on the collector inside the rollout step: 1. SGEMM: hidden[N, AH] = h_s2_q[N, SH2] @ W_fc[AH, SH2]^T 2. bias+relu in-place on hidden 3. SGEMM: pre_out[N, 6] = hidden[N, AH] @ W_out[6, AH]^T 4. trade_plan_activate → exp_plan_params[N, 6] Weight resolution uses the same `aux_w_ptrs` array the K=3 forward already consumes (`f32_weight_ptrs_from_base`); indices 91-94 match the corrected trainer-side reads. The `trade_plan_activate` kernel is loaded from `EXPERIENCE_KERNELS_CUBIN` (same cubin the rest of `exp_module_extra` uses; the trainer loads it from there too). The K=3 concat now takes `exp_plan_params.raw_ptr()` instead of NULL — both sides see f(h_s2; W_plan_*_init), symmetry restored. Plan tensors at [91..94] still have no backward (no Adam updates), so the plan-head weights stay at Xavier cold-start forever. This commit delivers the symmetry the K=3 head requires, not a learned plan signal — adding a real plan-head backward is a follow-up project. New struct fields on GpuExperienceCollector: - exp_trade_plan_hidden_buf [alloc_episodes × adv_h] - exp_trade_plan_pre_out_buf [alloc_episodes × 6] - exp_plan_params [alloc_episodes × 6] - exp_trade_plan_activate_kernel: CudaFunction Lib test suite: 1016/0 green maintained. Audit doc updated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
fb9b62a1f9 |
fix(plan-head): trainer trade plan forward reads from wrong param indices
`launch_trade_plan_forward` read `w_fc/b_fc/w_out/b_out` from `padded_byte_offset(¶m_sizes, [82..86))` — i.e., the ISV-conditioning + recursive-confidence tensors (`b_isv_gate`, `w_isv_gamma`, `b_isv_gamma`, `w_conf_fc`). The actual plan tensors live at `[91..95)` per `compute_param_sizes` and the matching Xavier init block. No backward exists for the plan head, so the plan tensors at `[91..94]` sat at cold-start Xavier values forever while `plan_params` was being driven by whichever ISV/conf weights happened to occupy the wrong offsets. Every downstream consumer (`backtest_plan_kernel`, `plan_isv` slots in `experience_kernels`, `compute_plan_params` in `q_value_provider`, regime gating, and the SP22 H6 vNext B5b concat path) was therefore conditioning on noise correlated with ISV optimisation, not on a learned plan. Discovered while preparing Phase B5b-2 (collector trade plan launch). Two-line index fix + diagnostic comment + audit doc entry. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
cdd3dc6edb |
feat(sp22-vnext): Phase F-3c — HEALTH_DIAG snap + console line for K=3 CE EMA
Completes the F-3 observability story. Smoke runs now print
aux_trade_outcome_ce_ema every epoch in the standard HEALTH_DIAG output
— no ISV inspector required.
Changes:
health_diag.rs:
- New field aux_trade_outcome_ce: f32 appended at END of
HealthDiagSnapshot per "Field order is stable; adding fields
appends to the end" doc rule. Existing fields' byte offsets
preserved → no kernel-side word offset re-validation.
- snapshot_size_is_stable pin: 149 × 4 → 150 × 4 = 600 bytes.
health_diag_kernel.cu:
- New WORD_AUX_TRADE_OUTCOME_CE = 149 (appended at end).
- WORD_TOTAL = 150 (was 149); static_assert bumped.
- New kernel arg aux_trade_outcome_ce_idx appended after
moe_lambda_eff_idx.
- New mirror write after the existing MoE mirrors. Stream-implicit
ordering: aux_outcome_ce_ema_update (F-3b) fires before
health_diag_isv_mirror, so the read picks up the just-updated EMA.
gpu_health_diag.rs:
- launch_isv_mirror gets new aux_trade_outcome_ce_idx: i32 arg.
gpu_dqn_trainer.rs:
- launch_health_diag_isv_mirror passes
AUX_TRADE_OUTCOME_CE_EMA_INDEX as the new arg.
training_loop.rs:
- Per-epoch metrics push appends ("aux_trade_outcome_ce_ema",
ISV[538]) to the standard out vec. Console / CSV automatically
includes the new column.
End-to-end F-3 chain now closed:
K=3 fwd → aux_to_loss_scalar_buf → aux_outcome_ce_ema_update
→ ISV[538] → health_diag_isv_mirror → snap.aux_trade_outcome_ce
→ training_loop metrics → console.
Operator sees CE every epoch:
- Cold-start: 0.000
- After bootstrap: ~1.098 (= ln(3))
- After training: ideally 0.5-0.7 (head learning)
Phase F end-to-end ready. The vNext stack has:
- Full GPU kernel chain (A2-A5 + D + plan-conditioning)
- Full Rust wireup (B0-B4, B4b-1/2, C-1/C-2, B5b)
- Real labels reaching trainer
- K=3 → policy via state slots + atom-shift
- End-to-end CE observability
Verification:
- cargo check -p ml clean.
- cargo test -p ml --lib → 1016/0 green (incl. bumped
snapshot_size_is_stable byte-size pin test).
Audit: docs/dqn-wire-up-audit.md Phase F-3c section.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
02479c885d |
feat(sp22-vnext): Phase F-3b — K=3 CE EMA launcher wireup + reset registry
Completes the Phase F-3 observability chain. Kernel + ISV slot landed
in F-3a; this commit wires the launcher into the trainer's per-step
training graph and registers the FoldReset entry.
Changes:
- gpu_dqn_trainer.rs:
- New pub(crate) static AUX_OUTCOME_CE_EMA_CUBIN embed
- New aux_outcome_ce_ema_kernel: CudaFunction field on GpuDqnTrainer
- Constructor loads kernel handle + struct-init
- New launch_aux_outcome_ce_ema() method (single-thread launch,
ISV slot 538, α=0.05)
- training_loop.rs:
- Launch call appended after launch_aux_heads_loss_ema()
- New dispatch arm "aux_trade_outcome_ce_ema" in reset_named_state
- state_reset_registry.rs:
- New RegistryEntry for "aux_trade_outcome_ce_ema" (FoldReset
sentinel 0.0)
End-to-end observability now live: K=3 head's batch-mean sparse-CE
flows into ISV[538] every step via Pearl A-bootstrapped EMA.
Smoke runs can read this slot to verify learning:
- Cold-start: 0.0
- After first step with B_valid > 0: bootstrap to first observation
(~1.098 = ln(3) for uniform K=3 prediction)
- Healthy learning: monotonic decrease toward 0.5-0.7 over epochs
- Falsification: pinned at ~1.098 for many epochs
Phase F prep complete. The trade-outcome aux head's:
- Forward chain (A2-A5 + B0-B4)
- Real labels reach trainer (B4b-1/2)
- K=3 → policy state (C-1/C-2)
- K=3 → Q-target atom-shift (D)
- Plan-conditioning (B5b)
- Smoke observability (F-3 + F-3b)
are all wired. Phase F deployment (argo-train.sh smoke) is next.
Verification:
- cargo check -p ml clean.
- cargo test -p ml --lib → 1016/0 green (incl. every_fold_and_soft_
reset_entry_has_dispatch_arm pin test).
Audit: docs/dqn-wire-up-audit.md Phase F-3b section.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
de64935b78 |
feat(sp22-vnext): Phase F-3 — K=3 CE EMA producer + ISV slot
Adds observability infrastructure for Phase F smoke validation: new ISV slot AUX_TRADE_OUTCOME_CE_EMA_INDEX = 538 tracks the K=3 head's batch-mean sparse cross-entropy EMA. Producer kernel registered + cubin-built; launcher wireup follows in Phase F-3b commit. Changes: - sp22_isv_slots.rs: new pub const AUX_TRADE_OUTCOME_CE_EMA_INDEX = 538 - gpu_dqn_trainer.rs ISV_TOTAL_DIM: 538 → 539 - NEW kernel aux_outcome_ce_ema_kernel.cu: single-thread single-block direct-to-ISV EMA writer with Pearl A first-observation bootstrap. Fixed α=0.05 (slow-moving observability). NULL-tolerant + NaN-guarded. - build.rs: kernel registered; cubin compiles (3.2 KB) Why dedicated kernel (not extension of aux_heads_loss_ema_update): The K=2/K=5 EMA writes through Pearls A+D's 2-stage producer scratch. Extending it would require allocating a new scratch slot, threading Pearls A+D mapping, and touching apply_pearls_ad_kernel. The K=3 head's CE is OBSERVABILITY ONLY in this commit — no Pearls A+D adaptive α needed. Minimum-scope direct-to-ISV path. Re-routable through Pearls A+D if K=3 CE later becomes a controller anchor. Smoke validation signal interpretation: - Cold-start: ISV[538] = 0.0 (FoldReset sentinel) - First step with B_valid > 0: Pearl A bootstrap → ISV[538] = first CE - Typical uniform-K=3 cold-start CE: ln(3) ≈ 1.098 - Learning signal: CE drops from ~1.098 toward 0.5-0.7 over epochs - Falsification signal: CE pinned at ln(3) for many epochs → head can't learn (label noise / under-capacity / outcomes unconditional) Phase F-3b follow-up: - Trainer launcher call after aux_heads_loss_ema_update - HEALTH_DIAG snap layout extension + console column - Reset registry entry (FoldReset sentinel 0) Verification: - cargo check -p ml clean. - cargo test -p ml --lib → 1016/0 green. Audit: docs/dqn-wire-up-audit.md Phase F-3 section. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
93aa4cd6a2 |
feat(sp22-vnext): Phase D — 12-weight W atom-shift (4 actions × 3 outcomes)
THE K=3 HEAD NOW DIRECTLY MODULATES Q-TARGETS. Phase D extends the
Phase 3 atom-shift mechanism from per-action W[4] reading single state
slot 121 to per-(action, outcome) W[4, 3] = 12 weights reading 3 state
slots [121..124).
Mathematical change: shift[a, b] now sums Σ_k W[a*K+k] × state[121+k]
= W[a, 0]*p_Profit + W[a, 1]*p_Stop + W[a, 2]*p_Timeout.
5 atom-shift kernel sites updated (all coordinated):
1. experience_kernels.cu::compute_expected_q (replay path)
2. experience_kernels.cu::mag_concat_qdir (rollout path)
3. experience_kernels.cu::quantile_q_select
4. c51_loss_kernel.cu loss numerator (next_state CVaR side)
5. c51_loss_kernel.cu Bellman target (online + target combined)
Each site replaces `W[a] × state_121` with `Σ_k W[a*K+k] × state[121+k]`
unrolled 3 times. State hoist points lift 1 scalar → 3-element array.
5 supporting kernel updates:
- aux_w_prior_init_kernel.cu: writes 12 K=3 structural priors instead
of 4. Spec prior matrix:
Short × {Profit=+0.5, Stop=-0.5, Timeout=0}
Hold × {Profit= 0, Stop=+0.5, Timeout=0}
Long × {Profit=+0.5, Stop=-0.5, Timeout=0}
Flat × {Profit= 0, Stop=+0.5, Timeout=0}
Block dim bumped 4 → 12.
- c51_aux_dw_kernel.cu: grid bumped (4,1,1) → (b0_size×K=12,1,1).
blockIdx.x decoded as (a, k); reads state slot 121+k, writes
dw_aux[a*K + k]. New kernel arg aux_outcome_k=3.
- adam_w_aux_kernel.cu: W_AUX_DIM 4 → 12. Block dim 12 threads.
Trainer-side buffer resizes:
w_aux_to_q_dir [4] → [12]
adam_m_w_aux [4] → [12]
adam_v_w_aux [4] → [12]
dw_aux_buf [4] → [12]
Rust launcher updates:
- c51_aux_dw_kernel launch: grid (4,1,1) → (12,1,1) + new aux_kto arg
- adam_w_aux_kernel launch: block (4,1,1) → (12,1,1)
- aux_w_prior_init launch: block (4,1,1) → (12,1,1)
Cold-start gracefulness preserved: state[121..124] = 0.0 at step 0
(no K=3 prediction yet from C-1 producer). Σ_k W[a*K+k] × 0 = 0 →
zero atom-shift across all actions. After step 1+ when C-1 producer
fires, real softmax probs activate the prior W's structural bias and
Adam refines from there.
End-to-end K=3 → Q-target chain now active:
K=3 fwd (B3/B4) → softmax → C-1 producer → prev_aux_outcome_probs
→ C-2 state gather → state[121..124) → Phase D atom-shift
→ Q-target z_n + Σ_k W[a*K+k] × prob[k]
→ Bellman target + argmax + action_select all see aux's outcome
prediction.
The K=3 head now influences policy via TWO paths: state input (Phase
C-2) AND Q-target modulation (Phase D).
Verification:
- cargo check -p ml clean.
- cargo test -p ml --lib → 1016/0 green.
Remaining vNext work:
- Phase E: dW backward gradient validation tests
- Phase F: Validation smoke at structural prior — decisive spec test
- B5b-2 (deferred): collector trade plan launch
Audit: docs/dqn-wire-up-audit.md Phase D section.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
d2331f2f62 |
feat(sp22-vnext): Phase B5b — full plan-conditioning integration
The K=3 head's input is now [h_s2_aux (256) || plan_params (6)] = 262- dim, matching the spec's intended architecture. Implements the input concat in both the trainer's replay-batch path and the collector's rollout-step path, with appropriate handling of the backward-side stride mismatch. NEW kernel strided_row_saxpy_kernel.cu: row-truncating SAXPY that accumulates first n_cols_copy columns per row of src [B, src_cols] into dst [B, dst_cols] scaled by alpha. Handles stride mismatch (src_cols != dst_cols). Needed because backward emits dh_s2_aux_to_buf [B, 262] but dh_s2_aux_accum [B, 256] only consumes first 256 cols. PLAN_PARAM_DIM = 6 constant in gpu_aux_heads.rs. Ops struct updates: - AuxTradeOutcomeForwardOps gains concat_kernel + launch_concat() - AuxTradeOutcomeBackwardOps gains strided_saxpy_kernel + launch_strided_row_saxpy() - Both load new cubins in new() Weight tensor resize: - sizes[163] = H × (SH2 + PLAN_PARAM_DIM) = 128 × 262 = 33,536 floats - fan_dims[163] = (H, SH2 + PLAN_PARAM_DIM) Trainer changes: - New aux_to_input_buf [B × 262] field - aux_dh_s2_to_buf resized to [B × 262] - aux_partial_to_w1 resized to [B × H × 262] - max_aux_tensor_len bumped for param_grad_final scratch Trainer forward (aux_heads_forward): - Concat h_s2_aux + plan_params_buf → aux_to_input_buf - forward() with SH2_TOTAL=262 Trainer backward (aux_heads_backward): - backward() with SH2_TOTAL=262; reads aux_to_input_buf - saxpy_f32_kernel SAXPY for dh_s2_aux REPLACED by launch_strided_row_saxpy: copies only first SH2=256 cols per row; trailing 6 cols (plan_params gradient) are dropped — STOP-GRAD on trade plan head from aux loss. Collector forward (rollout): - New exp_aux_to_input_buf [N × 262] field - Concat with plan_params_ptr = 0 (NULL) → zero-fill trailing 6 cols - forward() with SH2_TOTAL=262 Train/inference asymmetry (documented): - Trainer: real plan_params from trade plan head output - Collector: zeros (no trade plan launch in collector) - The head is trained on real plan-conditional outcomes but queried at rollout time with plan_params=0. Phase B5b-2 follow-up would add a trade plan launch to the collector to resolve. Deferred — current state is functional, head still receives plan signal during training. Stop-grad on plan_params: backward writes full [B, 262] gradient, but strided SAXPY only copies first 256 cols. Trade plan head weights NOT trained by K=3 aux loss in this commit. Verification: - cargo check -p ml clean. - cargo test -p ml --lib → 1016/0 green. Phase D next: 12-weight W atom-shift (4 actions × 3 outcomes). Audit: docs/dqn-wire-up-audit.md Phase B5b section. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
53462a28d9 |
feat(sp22-vnext): Phase C-2 — state gather flip K=2 single-slot → K=3 3-slot
THE K=3 HEAD NOW REACHES THE POLICY. Phase C-2 is the consumer-side flip — slot 121's semantic changes from K=2's recentered p_up to K=3's p_Profit, and slots [122, 123] gain new meaning as (p_Stop, p_Timeout). Architectural constraint: aux_dir_prob_per_env (K=2 buffer) is ALSO consumed by experience_env_step's β reward at lines 2335 + 3724-3725. Cannot repurpose that pointer to point at the K=3 [N, 3] buffer — env_step β consumer would read wrong-stride memory. This commit adds SEPARATE arg aux_outcome_probs_per_env [N, 3] to state_gather kernels with NULL fallback. Branching semantic at state_gather read site: - aux_outcome_probs_per_env != NULL → K=3 active path: writes slots [121..124) via new assemble_state_outcome_k3 helper - aux_outcome_probs_per_env == NULL → K=2 fallback: writes slot 121 via existing assemble_state from the recentered scalar Changes: - state_layout.cuh: NEW __device__ helper assemble_state_outcome_k3 — mirrors assemble_state except padding slots [121..124) get p_Profit/p_Stop/p_Timeout (raw softmax probs [0, 1]), slots [124..128) zero for 8-alignment. - experience_kernels.cu: training-side experience_state_gather + eval-side backtest_state_gather both get new trailing arg aux_outcome_probs_per_env (NULL-tolerant). Read site branches: K=3 reads 3 floats / env → assemble_state_outcome_k3; K=2 fallback preserves legacy assemble_state call. - gpu_experience_collector.rs: training launcher passes self.prev_aux_outcome_probs.raw_ptr() → K=3 active in training. - gpu_backtest_evaluator.rs: eval launcher passes NULL → K=2 fallback in eval (eval has no aux producer infra yet). K=2 head still alive: - prev_aux_dir_prob still populated by aux_softmax_to_per_env_kernel - experience_env_step still reads it for β reward (independent consumer untouched) - EGF chain still reads exp_aux_nb_softmax_buf - Only K=2's slot 121 contribution to policy state is suppressed End-to-end K=3 chain now active: Label producer (A2) → per-(env, t) ring (B4b-1) → replay buffer scatter (B4b-2) → PER direct gather → trainer aux_to_label_buf → loss_reduce (B4) sparse CE on real labels → backward (B4) per-sample partials → Adam SAXPY (B1+B4) updates W1/b1/W2/b2 at [163..167) PARALLEL: Collector rollout K=3 forward (B3) → softmax tile → C-1 producer → prev_aux_outcome_probs [N, 3] → C-2 state gather → state[121..124] → policy reads in next step The K=3 head closes the loop: learns from real labels via replay, AND predictions reach policy via state assembly. Trainable + observable. Verification: - cargo check -p ml clean. - cargo test -p ml --lib → 1016/0 green. Remaining vNext work: - Phase D: 12-weight W atom-shift (4 actions × 3 outcomes) - Phase E: dW backward + Adam for W[4, 3] - Phase F: Validation smoke at structural prior - B5b (deferred): plan_params input concat Audit: docs/dqn-wire-up-audit.md Phase C-2 section. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
3286dc7dee |
feat(sp22-vnext): Phase C-1 — K=3 softmax → per-env 3-slot cache (producer)
First half of Phase C. Lands the producer side of the K=3 trade- outcome aux head's state bridge: new kernel populates a per-env 3-slot cache from the K=3 softmax tile every rollout step. The consumer side (state gather reading from this cache → state slots [121..124)) lands in Phase C-2. Mirrors the K=2 head's existing aux_softmax_to_per_env_kernel exactly at K=3: - K=2: prev_aux_dir_prob[env] = 2*softmax[env, 1] - 1 (recentered) - K=3: prev_aux_outcome_probs[env, k] = softmax[env, k] for k in [0, 3) Changes: - state_layout.rs: 3 new constants AUX_OUTCOME_PROFIT_INDEX = 121, AUX_OUTCOME_STOP_INDEX = 122, AUX_OUTCOME_TIMEOUT_INDEX = 123. PROFIT_INDEX aliases AUX_DIR_PROB_INDEX (same value, different semantic). Phase C-2 flips slot 121's meaning from K=2's recentered p_up to K=3's p_Profit. - aux_outcome_softmax_to_per_env_kernel.cu: new kernel + cubin. - gpu_dqn_trainer.rs: new SP22_AUX_OUTCOME_SOFTMAX_TO_PER_ENV_CUBIN embed. - gpu_experience_collector.rs: 2 new struct fields (cache buffer + kernel handle); cubin load + alloc in constructor; struct-init; per-step launch in rollout loop after K=3 forward. - build.rs: kernel registered. Encoding shift K=2 → K=3: K=2 used recentered [-1, +1] to match "no signal = 0" baseline of every other slot. K=3 keeps raw softmax probabilities [0, 1]. Cold-start sentinel 0.0 for all 3 slots = "no prediction yet" (mask). The 3-slot natural distribution is more informative than a scalar. Dead-code status: producer populates cache every step but experience_state_gather doesn't read from it yet — state slot 121 still receives K=2's prev_aux_dir_prob write. Phase C-2 swaps the state gather's source from K=2 cache to K=3 cache (3-slot write). Why split C into C-1 + C-2: experience_state_gather is a hot-path kernel with many consumers. Updating it touches training collector, eval-side backtest evaluator, Rust launcher arg list. C-2 lands that as an atomic state-semantic flip; C-1 lands the GPU-side scaffolding independently so the producer chain can be validated first. Verification: - cargo check -p ml clean. - cargo test -p ml --lib → 1016/0 green. Audit: docs/dqn-wire-up-audit.md Phase C-1 section. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
68f0481a9e |
feat(sp22-vnext): Phase B5a — input concat kernel scaffolding
Lands the plan-conditioning concat kernel for the K=3 trade-outcome forward as reusable scaffolding. Phase B5b (integration) is deferred with rationale: Phase C (state slots) is more critical for testing the K=3 head's effect on policy behavior, and can land independently of the plan-conditioning refinement. NEW kernel aux_to_input_concat_kernel.cu: - Writes [B, SH2+P] from h_s2_aux [B, 256] || plan_params [B, 6] - Pure GPU map; one thread per output element, no atomicAdd - NULL-tolerant on plan_params (zeros trailing P cols when source unavailable, e.g., collector cold-start where the trade plan head doesn't run) - Registered in build.rs; cubin compiles (5.7 KB). Dead code at this commit — no Rust launcher yet. Why B5 is split + B5b deferred: Full Phase B5 (integration) requires three coordinated changes: 1. Forward path: bump aux_to_fwd.forward() to SH2=262 + 262-dim input 2. Backward stride mismatch: backward emits dh_s2_aux_to_buf [B, 262], but dh_s2_aux_accum (input to aux trunk backward) is [B, 256]. A direct SAXPY mismatches row strides (262 vs 256) and corrupts the trunk's upstream gradient. Needs a strided-SAXPY kernel. 3. Collector-path plan_params unavailability: trade plan head only runs trainer-side. Workarounds: zero-fill, add trade plan to collector, or skip K=3 forward in collector. All have trade-offs. Phase B5b would need (1) strided-SAXPY kernel and (2) collector plan_params decision. Real work but NOT on the critical path for testing the K=3 head's effect on WR. Why Phase C should land first: The K=3 head currently trains on real labels (post-B4b) but doesn't influence policy behavior. Phase C wires the head's softmax into state slots [121..124) = (p_Profit, p_Stop, p_Timeout), replacing the K=2 single-slot 121 = 2*p_up - 1. WITH Phase C the policy reads aux's outcome predictions as state features → behavior changes → testable. Without Phase C, validation runs would show "K=3 head trains and converges" but predictions don't reach the policy → WR signal isn't a function of K=3 at all. We'd be testing nothing. Recommendation: skip the full B5 for now, do Phase C next, then Phase D (atom-shift). Phase B5b (plan-conditioning) is a refinement we add IF Phase C/D's no-plan-params version shows promise but plateaus below the WR ≥ 0.55 target. Audit: docs/dqn-wire-up-audit.md Phase B5a section. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |