fa30c2dd66c14a94cd166a810cb9eba6ef8ccb73
5053 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
jgrusewski
|
fa30c2dd66 |
feat(alpha): alpha_dqn_h600_smoke — runnable Task 12 DQN smoke
Phase E.1 Task 12. Linear Q-network (W [9×10] + b [9], no hidden layer)
trained with ε-greedy + Munchausen target on the Phase E ExecutionEnv.
End-to-end runnable: load env, train, periodically launch
alpha_kill_criteria + apply_pearls_ad chain at episode boundaries, emit
PASS/FAIL verdict against the 4 kill criteria thresholds.
Pipeline per training step (all on GPU):
1. forward Q_current on s_batch via alpha_linear_q_forward
2. forward Q_next on s'_batch via alpha_linear_q_forward
3. alpha_munchausen_target → targets[batch]
4. alpha_linear_q_grad → dW, db (sparse over taken actions)
5. alpha_linear_q_sgd_step on W and b (separate launches)
6. every K episodes: kill_criteria + apply_pearls_ad chain → ISV[539..542]
Pipeline visibility bumps so examples can reach launchers:
- cuda_pipeline::alpha_kernels module → pub
- All launch_alpha_* fns → pub
- launch_apply_pearls → pub
- ALPHA_LINEAR_Q_CUBIN → pub
These are appropriate pub exports (Phase E.1 public API surface).
Initial micro-smoke (horizon=100, n_episodes=50, lr=1e-6):
Q_SPREAD_EMA = 3.12 (≥ 0.05) PASS
ACTION_ENTROPY_EMA = 2.12 (≥ 1.0986) PASS
RETURN_VS_RANDOM_EMA = +1.03 (≥ 0.0) PASS
EARLY_Q_MOVEMENT_EMA = 2268 (≥ 0.01) PASS [unphysical scale]
Overall: PASS (uncalibrated)
Known stability issues — flagged in the binary's CLI docstring:
- lr=1e-4 diverges to NaN (Q grows, Munchausen target explodes)
- lr=1e-6 stays finite but Q grows 2000× over 50 episodes
- Follow-ups: gradient clipping, target network, reward normalisation
Bug fixed during development: `stream.memcpy_htod(&host, &mut buf.clone())`
was uploading to a TEMPORARY clone (dropped immediately) — `kc_scalar_dev`
and `kc_action_counts_dev` never got their host data → entropy=0, early_mvmt=0,
rvr stuck at the alloc-zeros default. Fixed by removing `.clone()` and using
direct `&mut` refs.
Reads:
config/ml/alpha_fill_coeffs.json (Task 5c)
ISV slots 547/548 (Task 7c baseline)
Writes:
config/ml/alpha_dqn_h600_smoke.json (verdict + per-checkpoint KC trajectory)
Reproduction:
cargo run -p ml --release --example alpha_dqn_h600_smoke -- \
--mbp10-dir /home/jgrusewski/Work/foxhunt/test_data/futures-baseline-mbp10/ES.FUT \
--horizon 600 --n-episodes 1000
Audit doc docs/isv-slots.md updated per Invariant 7.
|
||
|
jgrusewski
|
36ab50814e |
feat(alpha): alpha_linear_q kernels + launchers for Task 12 DQN smoke
Phase E.1 Task 12a. Three new CUDA kernels for the H=600 DQN smoke
(Task 12 proper) that lands in a follow-up commit:
alpha_linear_q_forward_kernel Q = X · W^T + b
alpha_linear_q_grad_kernel dW, db sparse MSE-TD over taken actions
alpha_linear_q_sgd_step_kernel element-wise params -= lr · grad
Architecture: single linear layer, no hidden layer. The Phase E state
vector has meaningful direct features (alpha_logit, spread_bps, position,
ofi_sum_5, …) so linear Q can capture real relations like Q[Buy] ∝
alpha_logit. If linear can't pass the kill-criteria gate, no architecture
upgrade will save it — and the smoke proceeds with NoisyNet escalation
per the plan.
Sparse gradient: only the taken action contributes (standard DQN TD
loss). No atomicAdd needed — one thread per (i, j) loops over the batch
and adds only when actions[b] == i.
GPU contract:
- No host branches inside any kernel (graph-capture compatible)
- No atomicAdd (per feedback_no_atomicadd)
- All compute on GPU (forward, grad, weight update)
- Tiny launch overhead — fits per-step (batch=64 forward = 576 threads,
1 block; grad = 99 threads, 1 block)
Three pub(crate) Rust launchers in alpha_kernels.rs match the
launch_apply_pearls pattern. Cubin embedded via include_bytes!.
Smoke test `linear_q_forward_grad_sgd_round_trip_matches_hand_math`
exercises all three kernels end-to-end on a small (batch=2, state_dim=2,
n_actions=3) case with full hand-math:
Forward: Q = [[2.1, 3.2, 0.3], [4.1, 5.2, 0.3]] ✓
Grad: dW = [[-1.8, -2.7], [0.8, 1.0], [0, 0]]
db = [-0.9, 0.2, 0] ✓
SGD: W' = [[1.18, 0.27], [-0.08, 0.90], [0, 0]]
b' = [0.19, 0.18, 0.30] ✓
All within 1e-4 tolerance. `cargo test -p ml --lib alpha_kernels`:
5/5 pass on RTX 3050 Ti in 2.04s (compile witness + 4 GPU smokes).
Audit doc docs/isv-slots.md updated per Invariant 7.
|
||
|
jgrusewski
|
ebbd28437a |
test(alpha): chained pipeline smoke — Task 12 wiring validation
End-to-end test for the kernel COMPOSITION that the H=600 DQN smoke
(Task 12 proper) will use at each rollout boundary:
t+0 alpha_kill_criteria_compute_kernel → scratch[0..4]
t+1 apply_pearls_ad_kernel(n_slots=4) → ISV[539..543]
The two prior alpha_kernels smokes (munchausen + kill_criteria) validated
kernels in isolation. This smoke validates the COMPOSITION on the same
stream — failures here are different (stream-ordering, Pearls index base,
Wiener offset base, scratch visibility) and would silently break Task 12.
Two iterations with stationary synthetic inputs:
Iter 1 (Pearl A bootstrap)
prev_x_mean=0 AND x_lag=0 → ISV[539..542] populated with raw scratch
observations = [0.2041, 0.8980, 1.0670, 0.1] within 0.01 tolerance.
Anchor slots 547/548 remain at Task 7c values (-5185, 4953).
Iter 2 (Pearl D stationary)
dx_mean = dx_step = 0 → α* = 0 → ISV unchanged from iter 1 within
1e-4. Stationary signal stays at the bootstrap value.
Test would catch:
- Producer's scratch write not visible to applicator (stream-ordering)
- Wrong Pearls isv_idx_base / wiener_offset_base
- Pearl A sentinel detection broken (formula yields 0 at t=0)
- Wiener state corruption (iter 2 drifts from iter 1)
Reuses launch_apply_pearls from sp4_wiener_ema.rs (pub(crate)). Helper
fn run_chained_iter factors the producer→applicator sequence so the two
iterations are byte-identical apart from the Wiener state's evolution.
`cargo test -p ml --lib alpha_kernels`: 4 pass (compile witness + 2 prior
GPU smokes + this chained smoke) on RTX 3050 Ti in 1.89s total. Audit doc
docs/isv-slots.md updated per Invariant 7.
Remaining Task 12 work (full H=600 DQN trainer integration with this
pipeline at rollout boundaries) is queued for a dedicated session.
|
||
|
jgrusewski
|
697bb586d0 |
test(alpha): kill-criteria GPU smoke matching hand-computed observations
Adds the second of the two Phase E.1 kernel smoke tests in
alpha_kernels.rs (companion to the munchausen_target smoke from
|
||
|
jgrusewski
|
91d1a52b9c |
refactor(alpha): rename phase_e_* → alpha_* — system-scoped naming
The kill-criteria producer, Munchausen target kernel, Rust launchers,
fit/baseline binaries, and their output JSON artifacts are *durable
infrastructure* of the alpha trading system (live across Phase E/F/G/...),
not milestone-scoped to Phase E specifically. Aligns with the earlier
`phase_e_isv_slots.rs` → `alpha_isv_slots.rs` rename rationale.
What was renamed:
Code files:
crates/ml/src/cuda_pipeline/phase_e_kill_criteria.cu → alpha_kill_criteria.cu
crates/ml/src/cuda_pipeline/phase_e_munchausen_target.cu → alpha_munchausen_target.cu
crates/ml/src/cuda_pipeline/phase_e_kernels.rs → alpha_kernels.rs
crates/ml/examples/phase_e_fit_fill_model.rs → alpha_fit_fill_model.rs
crates/ml/examples/phase_e_random_baseline.rs → alpha_random_baseline.rs
Artifacts:
config/ml/phase_e_fill_coeffs.json → alpha_fill_coeffs.json
config/ml/phase_e_random_baseline.json → alpha_random_baseline.json
Kernel function names:
phase_e_kill_criteria_compute_kernel → alpha_kill_criteria_compute_kernel
phase_e_munchausen_target_kernel → alpha_munchausen_target_kernel
Rust launcher names:
launch_phase_e_kill_criteria → launch_alpha_kill_criteria
launch_phase_e_munchausen_target → launch_alpha_munchausen_target
Static cubin names:
PHASE_E_MUNCHAUSEN_TARGET_CUBIN → ALPHA_MUNCHAUSEN_TARGET_CUBIN
Historical milestone tags in doc-comments ("Phase E.1 Task N (2026-05-15)")
are RETAINED — they record WHEN the work landed and what plan it
implemented, which doesn't change with the system-scoped rename.
Plus: ADDS the alpha_munchausen_target GPU smoke test in alpha_kernels.rs.
End-to-end validates the launcher + kernel against hand-computed expected
values: batch=2 with one terminal sample; expected targets [29.8, 1.1];
got match within 0.05 tolerance on RTX 3050 Ti. PROVES the Task 9/10
kernels actually run on GPU.
All affected references updated in:
- build.rs (kernel compile list)
- mod.rs (module registration)
- state_reset_registry.rs (4 RegistryEntry descriptions for slots 539-542)
- alpha_isv_slots.rs (slot table comment)
- docs/isv-slots.md (audit-doc cross-references)
Verified:
cargo test -p ml --lib alpha_kernels: 2/2 pass (including GPU smoke)
cargo test -p ml --lib state_reset_registry: 10/10 pass
cargo build -p ml --release --example alpha_fit_fill_model --example alpha_random_baseline: clean
|
||
|
jgrusewski
|
feb2e8cc34 |
feat(alpha): phase_e_kernels — Rust launchers for Task 9 + Task 10 cubins
Phase E.1 Task 11. Two pub(crate) launchers expose the kill-criteria
producer (Task 9) and Munchausen target augmentation (Task 10) for use
by future trainer integration:
launch_phase_e_kill_criteria(stream, kernel, q_values_dev, action_counts_dev,
scalar_inputs_dev, isv_dev, batch, n_actions,
scratch_out_dev)
→ kicks the kill_criteria producer; caller chains apply_pearls_ad_kernel
(n_slots=4, isv_idx_base=ALPHA_ISV_BLOCK_LO=539) to smooth into slots
539..542 via Pearl A bootstrap + Pearl D Wiener-α.
launch_phase_e_munchausen_target(stream, kernel, q_next_dev, q_current_dev,
actions_dev, rewards_dev, dones_dev,
gamma, alpha_m, tau, log_clip_min,
target_out_dev, batch, n_actions)
→ one-thread-per-sample target augmentation; α_m/τ/log_clip_min are
scalar args so a downstream ISV-driven controller can tune them.
Plan deviation: Task 11 plan-spec said "replace hardcoded n_step=32,
gamma=0.999 literals" but grep across gpu_dqn_trainer.rs found ZERO such
literals — the trainer already reads gamma via read_isv_signal_at(
GAMMA_DIR_EFF_INDEX) and epsilon via read_isv_signal_at(AUX_TRUNK_EPS_
INDEX). The actual E.1 deliverable was Rust launchers for the new
cubins, which this commit lands.
Both launchers follow the launch_apply_pearls pattern in
sp4_wiener_ema.rs — pre-loaded CudaFunction as parameter, u64 device
pointers, debug_assert! guards.
Audit doc docs/isv-slots.md updated per Invariant 7.
Tested via `cargo test -p ml --lib phase_e_kernels` — 1 compile-witness
passes. Real GPU integration test in Task 12.
|
||
|
jgrusewski
|
b1ba41d403 |
feat(alpha): Munchausen DQN target term kernel (Vieillard et al. 2020)
Phase E.1 Task 10. Standalone target-augmentation kernel: m = α_m · max(τ · log π(a|s), log_clip_min) V_soft(s') = max(Q_next) + τ · log Σ exp((Q_next − max) / τ) target = r + m + γ · V_soft(s') (terminal: r + m) π(a|s) ∝ exp(Q_online(s, a) / τ) — softmax policy from the online net. Munchausen bonus is implicit KL regularisation between successive policies; soft-V replaces the hard max bootstrap with a τ-weighted softmax average. Both softmaxes are computed via log-sum-exp with the max-trick. This is essential at τ ≈ 0.03 where raw exp(Q/τ) would overflow f32 for any Q-spread > 25 nats. The kernel is one-thread-per-batch-sample, no atomicAdd, no host branches. α_m, τ, log_clip_min are kernel args (not hard-coded), so a Phase E.2+ controller can ISV-drive them. Typical Vieillard values: α_m=0.9, τ=0.03, log_clip_min=-1.0. Does NOT touch any ISV slot — pure target augmentation. Cubin: target/release/build/ml-*/out/phase_e_munchausen_target.cubin (12.8 KB). Launcher integration is Task 11 (consumes target_out where the C51/MSE loss kernels currently consume `r + γ · max_a' Q_target`). Audit doc docs/isv-slots.md updated per Invariant 7. |
||
|
jgrusewski
|
d493b729bf |
feat(alpha): phase_e_kill_criteria producer kernel (slots 539-542)
Phase E.0 Task 9. Single-block, single-thread CUDA producer that writes 4
raw scalar observations into a contiguous scratch_out[0..4] block:
scratch_out[0] → ISV[539] Q_SPREAD_EMA (kill: ≥ 0.05)
scratch_out[1] → ISV[540] ACTION_ENTROPY_EMA (kill: ≥ 0.5·ln(9) ≈ 1.10)
scratch_out[2] → ISV[541] RETURN_VS_RANDOM_EMA (kill: ≥ 0, i.e. ≥ random)
scratch_out[3] → ISV[542] EARLY_Q_MOVEMENT_EMA (kill: ≥ 0.01, learned)
Downstream `apply_pearls_ad_kernel` (n_slots=4) chained on the same stream
applies Pearl A first-observation bootstrap + Pearl D Wiener-α smoothing —
composes with the canonical val_sharpe_delta_compute_kernel pattern rather
than reimplementing Wiener math (per feedback_no_cpu_compute_strict — one
Wiener implementation in the codebase, the GPU one).
Inputs:
q_values[batch, n_actions] most-recent Q forward output (device)
action_counts[n_actions] empirical action histogram (device)
scalar_inputs[3] [rollout_R_mean, q_init_norm, q_early_norm]
via mapped-pinned (host writes between rollouts)
isv reads slots 547 + 548 (random baseline
mean/std — populated once by Task 7c,
TrainingPersist)
No atomicAdd (per feedback_no_atomicadd), no host branches
(per pearl_no_host_branches_in_captured_graph), no CPU compute
(per feedback_cpu_is_read_only). __threadfence_system() before exit for
the chained Pearls applicator's visibility guarantee.
Cubin produced: target/release/build/ml-*/out/phase_e_kill_criteria.cubin
(16.8 KB).
Launcher integration is Task 11 (separate commit so this task can stand
alone for cubin validation). Audit doc docs/isv-slots.md updated per
Invariant 7.
|
||
|
jgrusewski
|
4f71ab32ae |
feat(alpha): random-uniform policy baseline (10K episodes, horizon 600)
Phase E.0 Task 7c. Ran phase_e_random_baseline against the fitted L1 FillModel on 500K MBP-10 snapshots from ES.FUT 2024-Q1. Completed in ~2 minutes (snapshot load dominated; episode loop ~150ms total). Results: mean reward = -5185.13 std reward = 4952.85 p05 = -13972.31 p25 = -7251.85 p50 (median) = -2804.56 p75 = -1787.90 p95 = -954.75 (best 5% of random episodes still lose) kill threshold = +4720.57 (= mean + 2σ; E.1 DQN must exceed) avg fills/ep = 139.22 (~1 fill every 4.3 steps) These numbers feed ISV slots: 547 (RANDOM_BASELINE_MEAN_INDEX) = -5185.13 548 (RANDOM_BASELINE_STD_INDEX) = 4952.85 Interpretation: the broken fitter (β_spread = -40 → near-zero limit fill probability at typical spreads) causes the random policy to over-rely on market orders, paying full spread + fee on every flip. With 139 fills per episode this compounds into the strongly-negative baseline. The baseline is *still meaningful* — the DQN will face the same env and the same fill model, so a DQN that beats this learns something real. Open follow-up for Phase E.1: regularise fit_poisson (add L2 penalty on β to prevent runaway β_spread on wide-spread tail samples), then re-run both Task 5 and Task 7. Until then, the current baseline is the operational reference point. |
||
|
jgrusewski
|
9cfc6d8502 |
feat(alpha): phase_e_random_baseline example + reset_at extension
Phase E.0 Task 7b. Random-uniform policy reward baseline binary, plus a
small `ExecutionEnv::reset_at(seed, start_cursor)` extension so episodes
can sample random starting points across a long snapshot replay.
The binary loads MBP-10 snapshots, constructs SnapshotRow values (with
L2/L3 synthesized at ±0.25-tick offsets per the L1-only parser
limitation), loads the fitted FillModel from JSON, then runs N random
episodes from random start cursors. Reports mean / std / quintile
percentiles + kill threshold (mean + 2σ) for E.1 to exceed.
Smoke run (500 episodes, horizon 600, 100K snapshots):
mean = -5600 (dominated by terminal force-close variance + market-order
over-reliance because fit converged to β_spread = -40
→ limit fill probability ~0 at typical spreads)
std = 5383
p95 = -895
kill threshold (mean + 2σ) = +5167
The deeply negative baseline is correct *for this env* even though it
doesn't reflect realistic random-policy P&L. The DQN will face the same
env (same fill model, same cost structure), so the comparison stays
fair. Fitter regularisation (to prevent β_spread runaway) is a Phase E.1
follow-up.
Run:
cargo run -p ml --release --example phase_e_random_baseline -- \
--mbp10-dir /home/jgrusewski/Work/foxhunt/test_data/futures-baseline-mbp10/ES.FUT \
--fill-coeffs config/ml/phase_e_fill_coeffs.json \
--horizon 600 \
--n-episodes 10000 \
--out-path config/ml/phase_e_random_baseline.json
env.reset_at also called by reset() (1-line refactor); no behavior change.
|
||
|
jgrusewski
|
12151ccf6a |
feat(alpha): fitted FillModel coefficients from 500K ES.FUT snapshots
Phase E.0 Task 5c. Ran phase_e_fit_fill_model on the ES.FUT 2024-Q1 MBP-10 + trade tape (5.2M trades, 3.9M MBP-10 events, 500K snapshots accumulated at snapshot_interval=50 over a ~24-minute window). Total runtime ~80s. Empirical fill rates within 60s window: - bid_l1: 4.97% (matches L1 maker-side activity in trending market) - ask_l1: 71.34% (high — most 60s windows see an aggressive buy) Fitted L1 cloglog coefficients (all 5 features): BID L1: β_0=-0.213 β_spread=-2.064 β_imbal=-0.099 β_ofi=-0.006 β_logτ=-0.286 ASK L1: β_0=+0.016 β_spread=-40.336 β_imbal=+0.041 β_ofi=+0.652 β_logτ=-0.055 Sanity (sign checks all pass): - β_spread < 0 both sides (wider spread → fewer fills) ✓ - bid β_imbal < 0 (more bid stack → harder to get hit by sell) ✓ - ask β_ofi > 0 (buying pressure correlates with ask fills) ✓ - β_logτ < 0 both sides (quieter markets → slower execution) ✓ L1-only limitation: as documented in the binary header, the parser only populates levels[0]; L2/L3 in the JSON are L1 with β_0 -= ln(L+1) attenuation. Default --out-path bumped to config/ml/phase_e_fill_coeffs.json so future re-runs land in the same committed location. |
||
|
jgrusewski
|
3bdf74018d |
feat(alpha): phase_e_fit_fill_model example — cloglog FillModel calibration
Phase E.0 Task 5b. Calibrates FillModel coefficients from historical MBP-10
+ trade tape. Streams snapshots concurrently with time-sorted trades; per
snapshot, determines binary fill outcome ("would a posted L1 limit have
been hit within next --window-seconds?"), accumulates (FillFeatures, y),
calls fit_poisson (cloglog Bernoulli, see
|
||
|
jgrusewski
|
a1e3336b1f |
feat(alpha): ExecutionEnv — 10-dim state, 9-action gating, terminal-only reward
Phase E Task 6. Core environment for the execution-policy DQN.
Episode lifecycle: position-open signal → up to `horizon_snapshots` →
forced close. Within the episode the policy chooses 9 actions; illegal
actions degrade to Wait inside step() (soft masking) — the network's
Q-output is never masked, preserving clean C51 and Munchausen targets.
Reward is **terminal-only**: realized PnL minus fees, accumulated over
all fills and force-closed at terminal mid if still open. No dense
per-snapshot shaping per pearl_event_driven_reward_density_alignment
(the canonical SP11→SP12 lesson — dense shaping on event-driven
objectives creates exposure-positive bias).
Bugs in the plan's sketch, caught + fixed during implementation:
1. Synthetic test's bid/ask were CONSTANT while mid drifted — would
have made the "uptrend" test lose money. Fixed: drift bid/ask
together with mid.
2. L1/L2 closing path used Side::None for fill lookup → 0% fill rate
for closing limits. Fixed: closing-long → Side::Sell (post at ask),
closing-short → Side::Buy (post at bid).
3. `cursor`-based hash for fill PRNG was non-deterministic across
replay seeds. Replaced with self-contained SplitMix64 RNG state
(no `rand` dep added) — fill randomness is now bit-identical across
train/eval replays at the same seed.
4. `max_step_per_episode` and `mid_at_decision` fields stored but
never read — dead per feedback_no_stubs. Removed.
5. Test harness used n==horizon, which made the cursor-exhaustion
guard (`cursor+1 >= len`) fire before the planned FlatMarket call.
Fixed by setting n > horizon so the meaningful `ep.step >= horizon`
path terminates the episode.
6. `.unwrap()` in test bodies → `.expect("…")` per pre-commit policy.
7. Volume-weighted entry_price across multi-fill scaling (the plan
only handled the open-from-flat case).
5 unit tests:
- state_has_expected_dim_and_is_finite (STATE_DIM=10, no NaN)
- market_buy_then_market_sell_on_uptrend_profits (PnL math)
- illegal_action_degrades_to_wait (soft mask, no fee, no fill counted)
- terminal_force_close_pays_out_when_position_open (force-close at mid)
- rng_is_deterministic_across_resets (same seed → identical fills)
`cargo test -p ml --lib env`: 16 passed (4 action_space + 7 fill_model
+ 5 execution_env); full module compiles clean with no new warnings.
|
||
|
jgrusewski
|
d08ab461db |
feat(alpha): fit_poisson via cloglog likelihood + Serde derives on FillCoeffs
Phase E Task 5 (code portion; the 5.2M-trade fit run lands separately).
Diagnosis: the plan's draft used pure Poisson NLL with Bernoulli y∈{0,1},
which converges to λ = empirical rate ȳ. But the runtime fill_prob() uses
`p = 1 − exp(−λ)`, so a trained λ=0.4 → predicted p=0.33 → systematic
~30pp under-fill bias on every passive backtest order. Training and
inference must agree on what λ means.
Fix: switch the fitter to the **cloglog (complementary log-log) binary
likelihood**. Per-sample gradient:
∂L/∂β_k = (p − y) · (μ/p) · x_k
where μ = exp(β·x), p = 1 − exp(−μ)
The μ/p factor is the link derivative; p.max(1e-7) handles the μ→0 limit
in f32 (the analytical limit μ/p → 1 is achieved automatically because
both numerator and denominator vanish proportionally).
Recovery test verifies the fix: 1000 deterministic 40% fills, all-zero
features → fitter recovers β_0 ≈ ln(0.5108) ≈ -0.672 (the value at which
1 − exp(−exp(β_0)) = 0.4), within tolerance 0.05. Slope coefficients
stay near zero (features uninformative).
Also adds Serde derives on FillCoeffs / FillFeatures / FillModel for JSON
serialization (downstream when the calibration example lands).
Deferred: the calibration example (Steps 3-7 of the plan task) is held
back until paired with the actual 5M-trade fit run — the plan's example
has a placeholder loop that violates feedback_no_stubs, and the loader
lift from precompute_features.rs deserves a dedicated commit.
4 new tests (7 total in env::fill_model now):
- fit_recovers_baseline_when_features_uninformative
- fit_rejects_empty_and_mismatched_inputs
- coeffs_round_trip_through_json
`cargo test -p ml --lib env::fill_model`: 7 passed.
|
||
|
jgrusewski
|
a5de50503f |
feat(alpha): Poisson regression fill model scaffold (coeffs fitted in Task 5)
Phase E Task 4. Medium-tier fill simulator per the design memo.
For each (level ∈ {L1,L2,L3}, side ∈ {Bid,Ask}):
λ(features) = exp(β · [1, spread_bps, L1_imb, OFI_5, log(τ+1)])
Per-snapshot Bernoulli fill probability for a posted limit order:
p = 1 − exp(−λ)
Market orders fill immediately at the opposite-side L1 quote (no slippage
modeled at medium tier). Closing actions (Side::None) return λ=0 — they
are handled separately in the env.
Scaffolding only. Task 5 fits the 30 coefficients (6 distributions × 5
features) from the 5.2M-trade historical tape. The skeleton constructor
uses β=0 → λ=1 → p≈0.632, a stable sanity default for early smokes.
Numeric guard: `linear.exp().min(50.0)` caps λ to prevent f32 overflow
under outlier features before fitting lands. Fitted models should stay
well below this cap in practice.
4 unit tests:
- skeleton_has_uniform_fill_prob (β=0 → p≈0.632 within 0.01)
- fill_prob_bounded_under_outlier_features
- lambda_cap_prevents_f32_overflow (β=100 outlier path)
- side_none_returns_zero_rate
`cargo test -p ml --lib env::fill_model`: 4 passed.
|
||
|
jgrusewski
|
94f8f65571 |
feat(alpha): 9-action discrete execution action space + legality gating
Phase E Task 3. Introduces crates/ml/src/env/ for the execution-policy
environment. This commit lands the action space only; fill_model (Task 4)
and execution_env (Task 6) append to mod.rs in their respective commits
per feedback_wire_everything_up (no orphan declarations).
- 9-action discrete space: Wait + {Buy,Sell,Flat} × {Market,L1,L2 / L1}
- Soft action masking: illegal actions degrade to Wait in env::step
(not by masking Q-output) per design memo — preserves clean C51
categorical targets and Munchausen term (Task 10)
- `is_legal(position)` with position ∈ {-1, 0, +1} (sign only;
magnitude is decoupled into the Kelly layer in Task 11)
- repr(u8) discriminants round-trip with from_u8 for replay-buffer
storage by the existing GPU DQN trainer
4 unit tests:
- n_actions_matches_enum_cardinality
- legality_gating_by_position (covers flat/long/short × all 9 actions)
- decode_closing_flag_is_only_flat
- repr_u8_round_trip (replay-buffer contract)
`cargo test -p ml --lib env::action_space`: 4 passed.
|
||
|
jgrusewski
|
8bf8bdf874 |
feat(alpha): state_reset_registry entries + dispatch arms for slots 539..548
Per feedback_registry_entries_need_dispatch_arms — both must land in the same commit; the test `every_fold_and_soft_reset_entry_has_dispatch_arm` walks training_loop.rs::reset_named_state source and asserts coverage. - 10 RegistryEntry rows appended after SP22 block (full producer/consumer rationale per existing dense-description pattern) - 7 dispatch arms in reset_named_state (the 3 TrainingPersist anchors — slots 544/547/548 — are not dispatched per the existing convention) - All sentinels 0.0 per pearl_first_observation_bootstrap All 10 registry tests pass. Audit doc docs/isv-slots.md updated per Invariant-7. |
||
|
jgrusewski
|
aa5908aa53 |
feat(alpha): reserve ISV slot block 539..550 for alpha trading system
12 contiguous slots reserved for durable alpha-system infrastructure: - 539-542: diagnostics (Q-spread, action entropy, return-vs-random, early-Q-movement EMAs) — read by Phase E kill criteria - 543-546: stacker-threshold controller (threshold, target, observed-rate, Kelly attenuation) — engagement-rate self-correction - 547-548: random-uniform baseline (mean, std) — anchor for kill criterion 541 - 549-550: reserved spare (absorb growth without bumping ISV_TOTAL_DIM) Named `alpha_isv_slots` rather than `phase_e_isv_slots` because these slots are intended to outlive any individual Phase E/F/G milestone — the SP4..SP22 naming was iteration-scoped, but this block is system-scoped infrastructure. ISV_TOTAL_DIM bumped to 551. All indices validated in 4 unit tests (bounds, membership, uniqueness, total-dim coverage). Audit doc docs/isv-slots.md updated per Invariant-7. |
||
|
jgrusewski
|
9c26e78cdc |
docs(phase-e): implementation plan for execution-layer RL policy
32 tasks across 5 milestones (E.0 foundation → E.4 shadow-mode), with locked design decisions from three rounds of focused research memos: - Q1 (fill sim): medium-tier Poisson regression from 5.2M trade tape - Q2 (reward): terminal-only, n-step credit (consumes ISV slot 517) - Q3 (alpha trust): implicit calibrated trust via state features - Q4 (state window): current snapshot + 2 short-horizon scalars - Q5 (sizing): hybrid decoupled fractional Kelly × Phase E attenuation Trainer choice: DQN primary (Rainbow + Munchausen target), PPO control on H=600 truncated only if kill criteria fire. Exploration: ε-greedy with kill-criteria gate at end of week 2; NoisyNet escalation (4-6 days due to dead scaffolding in our codebase) if criteria fail; RND beyond that. ISV consumption: 5 existing slots (n_step=517, γ=43-46, ε=41, Kelly=280, reward_caps=452-453); new block 539..550 reserved for Phase E (10 in active use, 2 spare). One new controller (stacker-threshold engagement-rate-self-correction at slot 543). Hardcoded by design: Kelly contract cap (Category-1 safety), kill-criteria thresholds (circuit breakers). All other knobs are ISV-driven per pearl_controller_anchors_isv_driven. Decisive gates at week 1 (H=600 kill criteria), week 4 (composition backtest Sharpe at half-tick > 0), and week 5 (shadow-vs-backtest PnL within 30%). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
e190ecfa61 |
feat(ml-alpha): backtest cost sweep + annualised Sharpe (Phase 1d.4)
Initial single-cost backtest at τ=0.25 cost=0.25 revealed the binding
constraint: mean_ret = -0.176, pre-cost EV ≈ +0.074, so the model has
a real directional edge but K=6000 price moves are too small to clear
1-tick round-trip cost. The cost-vs-edge balance is the real Phase 1d.4
verdict question, not whether the model has signal.
Two enhancements per the insight block in the previous run:
1. **Cost sweep**: GpuBacktest::run now takes `costs: &[f32]` and
produces (C × T) rows instead of T. The smoke runs at five costs:
- 0.0 frictionless upper bound (theoretical max Sharpe)
- 0.0625 quarter-tick (very aggressive execution)
- 0.125 half-tick (professional desk)
- 0.25 1 tick = $12.50/contract (retail / pessimistic)
- 0.50 2 ticks (very pessimistic)
Tells us the break-even cost where Sharpe crosses zero.
2. **Annualised Sharpe**: per-trade Sharpe × sqrt(trades_per_year).
trades_per_year = n_trades × (seconds_per_year / test_time_span_seconds).
test_time_span_seconds derived from first/last test sequence end-bar
timestamps via FxCacheReader::record_timestamp. Standard Sharpe-time-
scaling assumption (trades roughly i.i.d.); imperfect when signals
cluster in correlated regimes, but the right ballpark for comparison
with industry benchmarks.
Output adds per-cost-band "best operating point" tables plus a clear
"REALISTIC VERDICT" line at cost=0.125 (half-tick — what a professional
desk would actually pay) with three gates:
- Sharpe_ann > 2.0 → deployable
- 0.5 < Sharpe_ann ≤ 2.0 → marginal
- Sharpe_ann ≤ 0.5 → fail at realistic cost
The "FRICTIONLESS UPPER BOUND" line reports the intrinsic edge — what
the model could theoretically achieve at zero cost. Even if realistic
Sharpe fails, this number tells us whether the model has anything to
optimise toward at deployment.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
20c361a300 |
feat(ml-alpha): Phase 1d.4 GPU-native backtest — proper SSM-stacker Sharpe verdict
Four new kernels in mamba2_alpha_kernel.cu:
- backtest_per_trade_pnl : [T, N] per-trade PnL with threshold filter
- backtest_sum_reduce_f32 : block tree-reduce returns per threshold (T scalars)
- backtest_sum_squared_reduce : block tree-reduce returns² per threshold (T scalars)
- backtest_sum_reduce_i32 : block tree-reduce trade counts per threshold
All atomicAdd-free via block tree-reduce in shared memory (per
feedback_no_atomicadd). Single kernel launch handles the full
threshold sweep across all sequences via grid_x=T, grid_y=ceil(N/256).
New module crates/ml-alpha/src/backtest.rs:
- GpuBacktest::from_block(&Mamba2Block) — reuses cubin already loaded
- GpuBacktest::run(probs, prices_t, prices_kt, thresholds, cost) → Vec<BacktestStats>
- Returns: n_trades, mean_ret, std_ret, Sharpe (per-trade unannualised),
hit_rate, total_pnl per threshold
Wired into phase1d_long_horizon.rs after the stacker eval:
- Convert stacker_logits → probs via sigmoid
- Upload probs + end-bar prices + (end-bar + horizon) prices to GPU
- Sweep thresholds [0.00, 0.02, 0.05, 0.10, 0.15, 0.20, 0.25]
- Print per-threshold table + best Sharpe operating point
- GATE: per-trade Sharpe > 1.5 = deployable, 0.5-1.5 = marginal, < 0.5 = fail
Cost model: 0.25 price units round-trip = 1 ES.FUT tick = $12.50/contract.
Tunable via --cost-per-trade. Realistic for retail flow; brokers can
trade at half-tick or better.
GPU-pure on the hot path: kernels do per-trade math + reductions;
host only receives T (= 7 here) scalars per metric for final Sharpe
arithmetic. No GPU↔CPU roundtrip per trade.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
149e1ae7b7 |
fix(ml-alpha): make stacker always-on (clap default_value_t=true bug)
Bool flag with `default_value_t = true` doesn't accept `--stacker true` on the command line in clap — it expects either the flag alone (which inverts) or a custom action. Cleanest fix: drop the flag entirely; the stacker block always runs when cal_frac is in (0, 1). Phase 1d.3 stacker delivered the key result: - Stacker test AUC: 0.7078 (raw Mamba: 0.6619, +4.6pts) - Stacker test accuracy: 0.6683 (raw Mamba: 0.6187, +5.0pts) - Stacker test Brier: 0.2088 (raw Mamba: 0.2299, well below chance 0.25) - Stacker spread-Q4 accuracy: 0.8164 (raw Mamba spread-Q4: 0.7467, +7pts intra-regime) See pearl_stacker_beats_threshold_gate_with_regime_info.md for full write-up and design implications for Phase 1d.4 backtest. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
ab4b7c64cb |
feat(ml-alpha): GPU-native stacked regime head on top of Mamba2 (Phase 1d.3)
Builds a second-stage MLP that takes [mamba_logit, 6 Block-S features] as input (7 dims) and learns the joint alpha-and-regime score in one calibrated output. Trains on the cal half of val (same 50/50 split as Platt/isotonic so all comparisons are on the same held-out test bars). Architecture: 7 → hidden_dim → 1 sigmoid, GELU activation, BCE-with-logits loss, AdamW. Uses the existing GPU-native `MlpModel` from crates/ml-alpha/src/mlp.rs — same primitives used for the Phase 1c MLP baseline. No CPU compute on the hot path (per feedback_cpu_is_read_only); all weights, activations, gradients, optimizer state on GPU; host writes the input matrix to a pinned buffer once per batch via GpuTensor::from_host. The Block-S columns are z-score normalised using cal-half statistics (then applied to the full val matrix) before training; mamba_logit is left raw since it's already close to standard-normal scale via the Mamba's natural calibration (see pearl_mamba_sss_state_yields_native_calibration). After training, reports stacker held-out accuracy + AUC + Brier + log-loss, plus stratified accuracy by Block-S feature so we can see whether the stacker absorbed the regime conditioning (uniform accuracy across quintiles) or just sharpened the Q4-gate (still elevated in Q4). Why this matters for production deployment per pearl_mamba_inherits_regime_structure: - Single calibrated score for conformal coverage gating downstream - Retrainable when market regimes drift - Captures interactions between regime features that a static threshold AND can't (e.g., spread-Q4 only when book is balanced) - Replaces the planned Phase 1d.3 dual-head architecture with a smaller stacked-generalisation approach (no separate regime classifier) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
635e2c8b48 |
feat(ml-alpha): Phase 1d.2 smoke + Block-S stratified accuracy diagnostic
After calibration, also stratify val accuracy across the 6 Block-S features (time_since_trade, time_since_snap, book_event_rate, spread_bps, L1_imbalance, micro_mid_drift) by sampling each val sequence's END BAR feature value, then running `metrics_detail::stratified_accuracy` per column with 5 quintile bins. Tells us whether the K=6000 Mamba alpha concentrates in specific book regimes (justifying an explicit regime head per Phase 1d.3) or is uniform across regimes (allowing direct backtest in Phase 1d.4). The Phase 1c stateless MLP showed strong stratification (spread-Q4 hit 0.752 acc on 76K samples while middle quintiles fell below 0.50); this run tests whether the Mamba inherits or transcends that pattern. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
d57026b0ba |
feat(ml-alpha): Phase 1d.2 smoke + post-hoc Platt/Isotonic calibration
Extends phase1d_long_horizon with a 50/50 val split (cal/test halves):
fit Platt and Isotonic on cal, evaluate on held-out test. Reports both
uncalibrated AND calibrated metrics (accuracy, AUC, Brier, log-loss).
Hypothesis: the K=6000 Mamba result (AUC=0.66 / acc=0.62 from the
uncalibrated
|
||
|
jgrusewski
|
4cf9499b58 |
feat(ml-alpha): Phase 1d.2 multi-minute label + smoke (K=6000 architectural test)
The DECISIVE gate for FoxhuntQ-Δ's two-head architecture. The K-sweep
(commit
|
||
|
jgrusewski
|
ab6922a199 |
feat(ml-alpha): Phase 1d.1 Mamba2 smoke example + first-shot verdict
Trains the from-scratch GPU-pure Mamba2 block against the snapshot fxcache, gathers sequence batches via end-bar lookup into train/val labels, runs AdamW for N epochs, computes val AUC. First-shot result (epochs=3, stride=8, lr=1e-3, hidden=64, state=16, seq_len=32): - Train BCE: 2.338 → 1.164 → 0.957 (monotone, still dropping) - Val accuracy: 0.5645 (beats MLP 0.5241) - Val AUC: 0.5684 (below MLP 0.6849) Interpretation: undertrained (loss curve still descending steeply; stride=8 sees only 1/8 of data; lr=1e-3 conservative given the training-loop unit test converged at lr=1e-2). Not yet a clean GATE FAIL — needs a retry with stride=2, lr=3e-3, epochs=10-20 before declaring the model class has a ceiling below the stateless MLP baseline. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
eb8c251afb |
feat(ml-alpha): Mamba2AdamW optimizer + end-to-end training-loop validation (Phase 1d.1, session 4)
GPU-pure AdamW for Mamba2Block's nine parameter tensors with bias-corrected
moment updates, decoupled weight decay, and host-side L2 grad clipping
(reads all 9 grad norms once, multiplies a single scale factor into the
kernel). Adam state (m, v) allocated once at optimizer construction;
reused across all training steps.
New kernel `mamba2_alpha_adamw_step` added to ml-alpha's cubin (no
cross-crate cubin loading; ml-alpha stays self-contained per its crate
invariant).
Borrow-checker gotcha worth flagging: `step()` mutably borrows each of
the 9 per-param `AdamState` fields in turn, plus the param itself.
Tried `apply()` as a method on `&self` — conflicts with `&mut self.s_*`.
Resolved by extracting `adamw_apply` as a free function taking (stream,
kernel, config) by reference; lets the caller mutably borrow distinct
state fields while sharing immutable references to the surroundings.
**The end-to-end training-loop test is the analytical-gradient validation:**
- 20 AdamW steps on a fixed batch (n_batch=4, seq_len=8, in_dim=4,
hidden=8, state=4) with binary labels (half +1, half 0)
- Asserts ≥15 of 20 steps have monotonically-decreasing BCE loss
- Asserts final loss < 0.65 (below the chance baseline ln(2) ≈ 0.693)
If backward had a sign flip, scale error, or wrong reduction axis
anywhere across:
- BCE-with-logits derivative (sigmoid(z) - y) / N
- Output projection cuBLAS sgemm (dY^T @ X for dw_out; dY @ W for dx)
- Scan backward kernel (per-channel scratch d_a/d_b/d_w_c + d_h_s2
identity passthrough)
- Reduction kernels (sum over j for d_a/d_b, sum over i for d_w_c)
- A/B projection backwards + branch-sum to recover d_x
- Input projection backward
- AdamW with bias correction + decoupled weight decay
…loss would NOT decrease monotonically. It does. The full backward
chain is correct.
Tests (10 passing on real GPU):
- training_loop_decreases_loss (THE end-to-end validation)
- backward_returns_finite_grads
- backward_rejects_wrong_d_logit_shape
- forward_train_returns_cache
- forward_shape_and_finite
- forward_rejects_wrong_shape
- config_rejects_seq_len_over_32
- config_rejects_state_over_16
- config_rejects_zero_dims
- constructs_and_loads_kernels
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
88a6db6eae |
feat(ml-alpha): Mamba2 backward pass — analytical, GPU-pure (Phase 1d.1, session 3)
End-to-end analytical backward through all six stages of the forward
chain. No atomicAdd, no SPSA, no host roundtrip during gradient flow —
each scratch buffer is per-channel-unique so concurrent writes don't
collide; cross-channel reduction is a separate kernel.
New API:
- GpuLinear::backward_with_slices(dy, act, weight, cublas, stream)
→ LinearGrads{dw, db, dx}
Mirror of forward_with_slices added to ml-core; no GpuVarStore lookup.
- Mamba2BackwardGrads holds dw_in/db_in/dw_a/db_a/dw_b/db_b/dw_c/dw_out/db_out
- Mamba2Block::backward(&cache, &d_logit) → Mamba2BackwardGrads
Chain (reverse of forward):
6′. W_out backward (cuBLAS sgemm) → d_h_enriched, dw_out, db_out
5′. mamba2_alpha_scan_bwd kernel → d_a/d_b/d_w_c per-channel/sample
+ d_h_s2 (identity passthrough)
↳ mamba2_alpha_reduce_d_proj × 2 → d_a_proj, d_b_proj [N, K, state]
↳ mamba2_alpha_reduce_d_w_c → dw_c [hidden, state]
3′. W_b backward → d_x_from_b, dw_b, db_b
2′. W_a backward → d_x_from_a, dw_a, db_a
↳ d_x = d_x_from_a + d_x_from_b
1′. W_in backward → dw_in, db_in (d_input discarded)
Memory cost per backward call (for Phase 1d.1 sizes N=64, sh2=32,
K=16, state=8): ~1.1 MiB scratch — fits comfortably on RTX 3050.
Tests (9 passing on real GPU):
- Backward produces all 9 grads with correct shapes
- All grads finite through the 6-stage chain
- dw_in non-zero (gradient flows to the input projection, proving the
full chain is wired — not silently zero-ing somewhere)
- Backward rejects wrong d_logit shape
- + all previously-passing forward / config / shape tests
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
bf6ed42acf |
fix(ml-alpha): backward kernel concerns — atomicAdd-free per-channel scratch + forward cache (Phase 1d.1)
Addresses four concerns surfaced after the forward-pass commit:
1. Backward kernel was scaffolded with `if (j==0)` to dodge atomicAdd,
but that drops contributions from j>0 channels. Rewritten so every
(i, j) thread writes its UNIQUE slot in per-channel scratch:
d_a_per_channel[N, sh2, K, state_d]
d_b_per_channel[N, sh2, K, state_d]
Followed by a unified reduction kernel mamba2_alpha_reduce_d_proj
that sums over j → d_a_proj / d_b_proj [N, K, state_d]. Same kernel
handles both call sites (DRY).
2. d_w_c gradient already had the right pattern (d_w_c_per_sample +
mamba2_alpha_reduce_d_w_c); kept as-is. All three gradient outputs
now follow the same atomicAdd-free scratch+reduce structure per
feedback_no_atomicadd.
3. `forward()` was discarding LinearActivations which the backward path
needs. New `Mamba2ForwardCache` struct carries (input_2d, x, a_proj,
b_proj, h_enriched) — everything backward needs to recover gradients
through the four projections + scan. `forward_train()` returns
`(logit, cache)`; `forward()` thin-wraps and discards the cache for
inference.
4. `x_hist[32 * 16]` in the backward kernel was hardcoded; configs with
seq_len > 32 would silently corrupt. Added MAMBA2_KERNEL_SEQ_MAX=32
constant + config validation. Backward kernel header documents both
limits explicitly.
Tests (7 passing on real GPU):
- forward_train returns cache with correct shapes for all 5 tensors
- seq_len > 32 rejected at config validation
- state_dim > 16 rejected
- forward output [B, 1] all finite
- forward rejects wrong in_dim / seq_len
- kernel handles all 4 functions resolve (fwd / bwd / reduce_d_proj /
reduce_d_w_c) + param-count sanity
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
c3e769b4b6 |
feat(ml-alpha): Mamba2 forward pass — GPU-pure end-to-end (Phase 1d.1, session 2)
Forward inference for the supervised snapshot stream — no ISV, no
temporal_weight, no NULL-pointer dispatch. Clean rewrite of the DQN
mamba2 kernel into a purpose-built alpha kernel.
New kernel `crates/ml-alpha/cuda/mamba2_alpha_kernel.cu` with three
extern "C" symbols:
- mamba2_alpha_scan_fwd — selective SSM scan over K timesteps with
sigmoid-gated state update; cheaper than
the DQN variant (no ISV stability scaling,
no per-position temporal_weight)
- mamba2_alpha_scan_bwd — analytical backward (scaffolded; full
gradient wiring lands in session 3)
- mamba2_alpha_reduce_d_w_c — block tree-reduce over batch for the
W_c gradient (no atomicAdd — per
feedback_no_atomicadd)
build.rs swapped from ../ml/src/cuda_pipeline/mamba2_temporal_kernel.cu
to the local cuda/mamba2_alpha_kernel.cu. ml-alpha no longer depends
on ml's CUDA source — fully self-contained alpha-stack.
Forward pipeline:
1. cuBLAS sgemm: input [B,K,in] @ W_in.T + b_in → x [B,K,hidden]
2. cuBLAS sgemm: x @ W_a.T + b_a → a_proj [B,K,state]
3. cuBLAS sgemm: x @ W_b.T + b_b → b_proj [B,K,state]
4. zero-init h_s2, h_enriched [B, hidden]
5. scan kernel: (a_proj, b_proj, W_c, h_s2) → h_enriched
6. cuBLAS sgemm: h_enriched @ W_out.T + b_out → logit [B, 1]
All on GPU; output is a [N] CudaSlice<f32> of raw logits. Caller
sigmoids + thresholds (or feeds directly into BCE-with-logits).
Tests (5 passing on real GPU):
- forward [4, 16, 81] → logit [4, 1], all finite
- reject wrong in_dim
- reject wrong seq_len
- reject state_dim > 16
- reject zero dims
- + parameter-count sanity
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
1f05d6cb80 |
feat(ml-alpha): from-scratch Mamba2 block — foundation (Phase 1d.1, session 1)
GPU-pure stateful encoder skeleton for the snapshot-stream falsification. This session lands the build infrastructure + weight allocation + kernel loading; forward/backward + training loop in follow-up sessions. - build.rs compiles `../ml/src/cuda_pipeline/mamba2_temporal_kernel.cu` to `mamba2_temporal_kernel.cubin` in OUT_DIR (rerun-if-env-changed=CUDA_COMPUTE_CAP per the L40S/H100 cubin-staleness pattern). Zero header dependencies → single nvcc invocation; no NVRTC. - `Mamba2Block` holds all parameters on GPU (`OwnedGpuLinear` from ml-core for the projection layers, raw `CudaSlice<f32>` for `W_c` which the kernel reads directly). Xavier init via ml-core, which uses pinned host buffers for the seed transfer. - Both `mamba2_scan_projected_fwd` and `mamba2_scan_projected_bwd` kernel symbols resolve at construction; forward and backward paths in follow-up. - State dim hardcoded at ≤16 in the kernel; config validation rejects >16. Tests (3 passing on real GPU): - Reject state_dim > 16 - Reject zero dims - Constructs + loads both kernels + correct param count (8417 for 81×64×16×1) Aligns with project memories: - feedback_no_nvrtc: pre-compiled cubin via build.rs - feedback_no_htod_htoh_only_mapped_pinned: pinned via ml-core init helpers - ml-alpha invariant: no `ml`/`ml-supervised` dep (only the .cu source file) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
6ac9b36782 |
feat(ml-alpha): phase1d_calibrate smoke — gate PASS
Platt scaling drops held-out Brier from 0.346 → 0.221 (chance=0.250); log-loss 1.096 → 0.632. Both Platt and isotonic land below chance baseline. AUC-accuracy gap was pure miscalibration, not fundamental misexpression. Learned: Platt a=0.32 (raw logits too extreme), b=0.89 (positive offset needed). Trained MLP underconfidence on positives compounds with negative prior. Proceed to 1d.1. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
7e77add1e1 |
feat(ml-alpha): Platt + isotonic calibrators for Phase 1d.0
- Calibrator trait + PlattScaler (2-param logistic, BCE gradient descent) - IsotonicCalibrator (pool-adjacent-violators algorithm) - Tests: invariant-based (clean separation, monotonicity) per pearl_tests_must_prove_not_lock_observations Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
5d79bf0b22 |
docs(phase1d): implementation plan for regime-gated tick reasoning + memory accumulator
26 tasks across 5 milestones (1d.0 through 1d.4) with decisive falsification
gates at each. Anchors to commit
|
||
|
jgrusewski
|
db874b1841 |
feat(foxhuntq): Phase 1c snapshot-resolution alpha + leakage fix + variable-dim fxcache
Three things landing atomically because they're load-bearing for each other: 1. **Trend-scanning leakage fix** — trend_scanning.rs was emitting OLS slope+t-stat over a *forward* window [t, t+L]. With the Phase 1a label = sign(price[t+60] − price[t]), the forward feature window overlaps the label window, contaminating it. Purged walk-forward only sterilizes forward-looking *labels* that cross the train/val split, not forward-looking *features* that peek inside the same horizon the label measures. The leak inflated MLP accuracy from 0.49 (legacy 74-dim baseline) to 0.75 — vanished to 0.50 after switching to a trailing window. Bounded the perfect-fit t-stat sentinel from ±1e6 → ±20 (p<1e-30 is already meaningless); eliminated the 16k corruption-cap drops. 2. **Variable-dim alpha column** — fxcache schema now carries the alpha-feature width via metadata (`alpha_feature_dim`), not a compile-time constant. Same on-disk format hosts the 134-dim bar-level stack OR the 81-dim snapshot stack. Reader + auto-detect honor the metadata-declared dim; downstream MLP auto-sizes `in_dim`. Single schema, no forks. 3. **Snapshot pipeline (Phase 1c falsification)** — `snapshot_pipeline.rs`: 81-dim per-MBP10-snapshot extractor reusing 10 snapshot-native alpha blocks + 6 new snapshot-specific features (time-since-trade, time-since-snap, event-rate, spread-bps, L1-imbalance, microprice-mid drift). `precompute_features` gets `--row-unit snapshot` flag; emits one fxcache row per LOB update (1.97M rows from MBP-10 data vs 206K for bar mode). **Smoke verdict on real data** (ES.FUT, 1.97M snapshots, 384K val): - Bar-level honest alpha: accuracy=0.5005, AUC=0.5043 (no signal) - **Snapshot-level alpha**: accuracy=0.5241, AUC=0.6849 (real signal, 384K val) - GBM corroboration: accuracy=0.5401 (non-linear partitioning sees more) - Horizon decay: alpha peaks at K=20-50 snapshots (~5-25ms), gone by K=500 - Regime-conditional: spread-Q4 quintile hits 0.752 accuracy on 76k samples Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
2a2f16b944 |
design(foxhuntq): architectural pivot away from per-bar DQN to decoupled Belief Bus + Conformal DRL
After 16+ SP-runs producing WR pinned at ~0.435 — and a session-end smoke
showing SP22 H6 vNext K=3 head architecture moves WR to 0.458 only via
degenerate Hold-collapse (PF erodes 1.45 → 1.08) — pivot to a research-
honest architecture: distributional supervised alpha + meta-labeling gate +
Coverage-Gated Kelly execution, integrated through a novel GPU-native
publish-subscribe Belief Bus substrate.
This is a DESIGN doc only. No implementation yet. v1 → v4 evolution captured
in the doc itself; v4 is research-honest with explicit prior-work citations:
- Bellemare/Dabney distributional RL (already shipped in foxhunt SP5+)
- Lopez de Prado triple-barrier + purging + meta-labeling
- Vovk/Romano/Gibbs-Candès conformal prediction foundations
- Sun-Yu 2025 NeurIPS CPTC (change-point-aware CP)
- Gan et al. 2025 NeurIPS arXiv:2510.26026 (CP for infinite-horizon RL —
we PORT Algorithm 1 directly in Phase 7, not invent)
- Zhu-Zhu ICML 2025 AlphaQCM (QCM variance estimation, adopted)
- Berti-Kasneci 2025 TLOB (motivates MLP baseline)
Honest novelty narrowed to three claims after literature review:
1. Belief Bus substrate — GPU-native pub/sub bus with per-slot
distributional semantics + conformal coverage + causal DAG metadata.
Extends our existing 539-slot ISV pattern (already novel architecture
vs published trading systems). The substrate integration is not in
literature.
2. Application domain — imbalance-bar HFT futures + MBP-10 microstructure +
triple-barrier labels. Existing distributional CP + DRL papers use
daily stocks, general RL benchmarks, or alpha formula discovery.
3. Adaptive controllers + per-slot conformal coverage — every adaptive
quantity in the system (Kelly priors, reward caps, Adam β1, regime
probabilities) gets conformal coverage attached. Not seen in
literature.
Tiered success criteria recalibrated per CFTC 2014 E-mini HFT study
(median firms hit ~55% WR / PF 1.2-1.4):
- Minimum viable: WR ≥ 50% AND PF ≥ 1.4 → deploy
- Goal: WR ≥ 53% AND PF ≥ 1.7
- Stretch: WR ≥ 55% AND PF ≥ 2.0 (original v1 target — aggressive
top-quartile HFT)
Eight phases with explicit falsification gates:
Phase 0: Purged walk-forward + bar audit (Lopez de Prado hygiene)
Phase 1a: MLP baseline alpha (cheapest falsification)
Phase 1b: TLOB/Mamba2/Liquid encoders
Phase 1C (conditional): tick-resolution if bar fails
Phase 2: Multi-head IQN + QCM + class weights
Phase 3: Belief Bus substrate
Phase 4: CPTC calibration
Phase 5: Coverage-Gated Kelly execution (deployment trigger if viable)
Phase 6: Production wiring + 2-week shadow mode
Phase 7 (optional): Port arXiv:2510.26026 conformal-DRL Q-residual
Phase 7 specifically detailed with concrete Algorithm 1 port (~1100 LOC
total), tunable params (k=5-10 ours vs 1-5 paper, due to γ=0.99 vs 0.8),
and falsification gate (empirical coverage ≥ 88% + PF improvement ≥ 0.2).
Deferred indefinitely (research-grade risk too high):
- Neural SDE (training instability per Kidger 2021)
- Hawkes process bar replacement (O(N²) MLE prohibitive at HFT scale)
- Multi-asset portfolio
- Learned in-trade exit head
Total minimum-viable path: Phases 0-6 ~6-8 weeks engineering + 20 hours
L40S compute. Falsification gates at every step.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
19bb3bc3c8 |
fix(sp22-vnext): aux_outcome CF half mirror on-policy (NOT all -1)
Smoke train-k95mj epoch 0 + epoch 1 printed
`trade_outcome_ce=0.000e0` — K=3 CE EMA stuck at Pearl A sentinel
across all training steps.
Root cause chain:
1. insert_batch is called with bs = total = 4_096_000 (cf-mult-
expanded), replay cap = 300_000 (L40S GpuProfile). Tail-clip:
off = 3_796_000, slice(off..) takes the last 300K elements.
2. Prior fix (
|
||
|
jgrusewski
|
878cc9ba72 |
feat(sp22-vnext): F-3c follow-up — K=3 CE EMA in stdout HEALTH_DIAG aux line
The F-3c commit (
|
||
|
jgrusewski
|
256a5fa5ab |
fix(sp22-vnext): K=3 aux_outcome_labels CF-half buffer underrun
Phase B4b-2 introduced `exp_aux_to_label_per_sample` sized
`[alloc_episodes × alloc_timesteps]` (no cf-mult expansion — the
B4b-1 per-step kernel only writes the on-policy half). The batch
finalisation cloned into the emitted `aux_outcome_labels` with size
`total = base_total × 2` (cf-mult expanded), so
`dtod_clone_i32(src, total, ...)` invoked `src.slice(..total)` on a
half-sized source → `CudaSlice::try_slice` returned `None` → the
internal `unwrap()` panicked at cudarc safe/core.rs:1648.
Repro: workflow train-xzv56 panicked after rollout completed
(timestep=999) on fold 0; rollout itself ran clean, the OOM from
the prior commit is gone.
Fix: replace the single `dtod_clone_i32` with a 3-step build:
1. alloc_zeros::<i32>(total)
2. cuMemsetD32Async(ptr, 0xFFFFFFFFu32, total, stream) — fills
all `total` slots with i32 mask sentinel -1 (byte pattern
0xFFFFFFFF reinterprets as i32(-1))
3. memcpy_dtod the first `base_total` real labels from
exp_aux_to_label_per_sample into the on-policy half
CF half remains at -1. The K=3 sparse-CE loss masks `label == -1`
out of the mean and B_valid count, so CF samples contribute zero
gradient — exactly the semantic we want (CF actions have no
observed trade-close outcome to predict against).
Lib suite: 1016/0 green. Audit doc updated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
0acf77e656 |
config(dqn): strip H100-tuned VRAM overrides from dqn-production.toml
dqn-production.toml hard-coded three H100-tuned values that shadow
GpuProfile auto-detection: batch_size=16384, buffer_size=500K,
gpu_n_episodes=4096. After the L40S-default flip lands (prior commit
|
||
|
jgrusewski
|
8b8bb1af70 |
infra(argo): default GPU pool to ci-training-l40s (sm_89) per feedback_default_to_l40s_pool
SP-chain training has been standardising on L40S since 2026-05-09, but
every invocation required an explicit `--gpu-pool ci-training-l40s`
override. The 2026-05-04 train-mnpf7 incident (sm_90 cubins deployed
to an L40S device, then resubmitted with the explicit override) was
the last incident in a long line of "forgot the pool flag" friction.
`feedback_default_to_l40s_pool.md` codified the user preference; this
commit lands the default in the actual invocation paths.
Changes:
- infra/k8s/argo/train-template.yaml: gpu-pool default H100 → L40S
- infra/k8s/argo/train-multi-seed-template.yaml: same + cuda-compute
-cap default 90 → 89
- scripts/argo-train.sh: docstring / --help / compute-cap fallback
case all flip to L40S as the bare default; H100 becomes opt-in via
`--gpu-pool ci-training-h100` for 80 GB / sm_90 workloads
- scripts/argo-test.sh: --help text aligned
Other architectural defaults (data-source=mbp10 per
feedback_mbp10_mandatory; imbalance-bar-threshold=20.0 per the 2026-
05-10 OOM-prevention fix) are already correct in the template.
Verified via `argo-train.sh dqn --branch sp20-aux-h-fixed --sha HEAD
--baseline --dry-run` — rendered workflow shows cuda-compute-cap=89,
no explicit gpu-pool override (template default L40S in effect).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
bbd52c3aa7 |
feat(sp22-vnext): FoldReset registry entries for B5b/C collector buffers
Two collector-side device buffers used by the K=3 trade-outcome head
were missing FoldReset registry coverage:
- prev_aux_outcome_probs [alloc_episodes × 3]
TRUE stale-read risk. Producer writes end-of-step; consumer
(experience_state_gather) reads start-of-next-step into
state[121..124). Without FoldReset the new fold's step-0 state
gather would inject the previous fold's last-step softmax probs
into the first batch's state slots.
- exp_aux_to_input_buf [alloc_episodes × 262]
Cleanliness-only. Concat kernel overwrites all 262 columns every
step before the K=3 forward reads them, so no steady-state stale-
read risk. Registered for parity with the rest of the K=3
pipeline + to satisfy feedback_registry_entries_need_dispatch_
arms (the pin test asserts every registry entry has a matching
dispatch arm in reset_named_state).
Both fields promoted to pub(crate) on GpuExperienceCollector so
reset_named_state can reach them. Matching dispatch arms added with
the standard memset_zeros pattern (is_win_per_env / hold_baseline_
buffer style).
Tests: All 10 state_reset_registry tests pass, including the critical
every_fold_and_soft_reset_entry_has_dispatch_arm pin test that walks
the dispatch body and validates parity with registry entries. Full
lib suite 1015/1 (the failing test is the pre-existing
test_dqn_checkpoint_round_trip NoisyLinear flake — pred1/pred2 sign
mismatch surfacing ~30-50% of full-suite runs, documented in
project_sp22_h6_vnext_resume memory as unrelated to this work).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
4b40710b7c |
feat(sp22-vnext): Phase B5b-2 collector trade plan forward — resolves K=3 asymmetry
Phase B5b's K=3 input concat passed plan_params=NULL because the collector had no trade plan launch. Trainer-side K=3 forward trained on real plan_params while the collector queried at plan_params=0 — documented train/inference asymmetry on the plan-conditioning surface. Phase B5b-2 mirrors the (now-corrected) trainer `launch_trade_plan_forward` chain on the collector inside the rollout step: 1. SGEMM: hidden[N, AH] = h_s2_q[N, SH2] @ W_fc[AH, SH2]^T 2. bias+relu in-place on hidden 3. SGEMM: pre_out[N, 6] = hidden[N, AH] @ W_out[6, AH]^T 4. trade_plan_activate → exp_plan_params[N, 6] Weight resolution uses the same `aux_w_ptrs` array the K=3 forward already consumes (`f32_weight_ptrs_from_base`); indices 91-94 match the corrected trainer-side reads. The `trade_plan_activate` kernel is loaded from `EXPERIENCE_KERNELS_CUBIN` (same cubin the rest of `exp_module_extra` uses; the trainer loads it from there too). The K=3 concat now takes `exp_plan_params.raw_ptr()` instead of NULL — both sides see f(h_s2; W_plan_*_init), symmetry restored. Plan tensors at [91..94] still have no backward (no Adam updates), so the plan-head weights stay at Xavier cold-start forever. This commit delivers the symmetry the K=3 head requires, not a learned plan signal — adding a real plan-head backward is a follow-up project. New struct fields on GpuExperienceCollector: - exp_trade_plan_hidden_buf [alloc_episodes × adv_h] - exp_trade_plan_pre_out_buf [alloc_episodes × 6] - exp_plan_params [alloc_episodes × 6] - exp_trade_plan_activate_kernel: CudaFunction Lib test suite: 1016/0 green maintained. Audit doc updated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
fb9b62a1f9 |
fix(plan-head): trainer trade plan forward reads from wrong param indices
`launch_trade_plan_forward` read `w_fc/b_fc/w_out/b_out` from `padded_byte_offset(¶m_sizes, [82..86))` — i.e., the ISV-conditioning + recursive-confidence tensors (`b_isv_gate`, `w_isv_gamma`, `b_isv_gamma`, `w_conf_fc`). The actual plan tensors live at `[91..95)` per `compute_param_sizes` and the matching Xavier init block. No backward exists for the plan head, so the plan tensors at `[91..94]` sat at cold-start Xavier values forever while `plan_params` was being driven by whichever ISV/conf weights happened to occupy the wrong offsets. Every downstream consumer (`backtest_plan_kernel`, `plan_isv` slots in `experience_kernels`, `compute_plan_params` in `q_value_provider`, regime gating, and the SP22 H6 vNext B5b concat path) was therefore conditioning on noise correlated with ISV optimisation, not on a learned plan. Discovered while preparing Phase B5b-2 (collector trade plan launch). Two-line index fix + diagnostic comment + audit doc entry. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
cdd3dc6edb |
feat(sp22-vnext): Phase F-3c — HEALTH_DIAG snap + console line for K=3 CE EMA
Completes the F-3 observability story. Smoke runs now print
aux_trade_outcome_ce_ema every epoch in the standard HEALTH_DIAG output
— no ISV inspector required.
Changes:
health_diag.rs:
- New field aux_trade_outcome_ce: f32 appended at END of
HealthDiagSnapshot per "Field order is stable; adding fields
appends to the end" doc rule. Existing fields' byte offsets
preserved → no kernel-side word offset re-validation.
- snapshot_size_is_stable pin: 149 × 4 → 150 × 4 = 600 bytes.
health_diag_kernel.cu:
- New WORD_AUX_TRADE_OUTCOME_CE = 149 (appended at end).
- WORD_TOTAL = 150 (was 149); static_assert bumped.
- New kernel arg aux_trade_outcome_ce_idx appended after
moe_lambda_eff_idx.
- New mirror write after the existing MoE mirrors. Stream-implicit
ordering: aux_outcome_ce_ema_update (F-3b) fires before
health_diag_isv_mirror, so the read picks up the just-updated EMA.
gpu_health_diag.rs:
- launch_isv_mirror gets new aux_trade_outcome_ce_idx: i32 arg.
gpu_dqn_trainer.rs:
- launch_health_diag_isv_mirror passes
AUX_TRADE_OUTCOME_CE_EMA_INDEX as the new arg.
training_loop.rs:
- Per-epoch metrics push appends ("aux_trade_outcome_ce_ema",
ISV[538]) to the standard out vec. Console / CSV automatically
includes the new column.
End-to-end F-3 chain now closed:
K=3 fwd → aux_to_loss_scalar_buf → aux_outcome_ce_ema_update
→ ISV[538] → health_diag_isv_mirror → snap.aux_trade_outcome_ce
→ training_loop metrics → console.
Operator sees CE every epoch:
- Cold-start: 0.000
- After bootstrap: ~1.098 (= ln(3))
- After training: ideally 0.5-0.7 (head learning)
Phase F end-to-end ready. The vNext stack has:
- Full GPU kernel chain (A2-A5 + D + plan-conditioning)
- Full Rust wireup (B0-B4, B4b-1/2, C-1/C-2, B5b)
- Real labels reaching trainer
- K=3 → policy via state slots + atom-shift
- End-to-end CE observability
Verification:
- cargo check -p ml clean.
- cargo test -p ml --lib → 1016/0 green (incl. bumped
snapshot_size_is_stable byte-size pin test).
Audit: docs/dqn-wire-up-audit.md Phase F-3c section.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
02479c885d |
feat(sp22-vnext): Phase F-3b — K=3 CE EMA launcher wireup + reset registry
Completes the Phase F-3 observability chain. Kernel + ISV slot landed
in F-3a; this commit wires the launcher into the trainer's per-step
training graph and registers the FoldReset entry.
Changes:
- gpu_dqn_trainer.rs:
- New pub(crate) static AUX_OUTCOME_CE_EMA_CUBIN embed
- New aux_outcome_ce_ema_kernel: CudaFunction field on GpuDqnTrainer
- Constructor loads kernel handle + struct-init
- New launch_aux_outcome_ce_ema() method (single-thread launch,
ISV slot 538, α=0.05)
- training_loop.rs:
- Launch call appended after launch_aux_heads_loss_ema()
- New dispatch arm "aux_trade_outcome_ce_ema" in reset_named_state
- state_reset_registry.rs:
- New RegistryEntry for "aux_trade_outcome_ce_ema" (FoldReset
sentinel 0.0)
End-to-end observability now live: K=3 head's batch-mean sparse-CE
flows into ISV[538] every step via Pearl A-bootstrapped EMA.
Smoke runs can read this slot to verify learning:
- Cold-start: 0.0
- After first step with B_valid > 0: bootstrap to first observation
(~1.098 = ln(3) for uniform K=3 prediction)
- Healthy learning: monotonic decrease toward 0.5-0.7 over epochs
- Falsification: pinned at ~1.098 for many epochs
Phase F prep complete. The trade-outcome aux head's:
- Forward chain (A2-A5 + B0-B4)
- Real labels reach trainer (B4b-1/2)
- K=3 → policy state (C-1/C-2)
- K=3 → Q-target atom-shift (D)
- Plan-conditioning (B5b)
- Smoke observability (F-3 + F-3b)
are all wired. Phase F deployment (argo-train.sh smoke) is next.
Verification:
- cargo check -p ml clean.
- cargo test -p ml --lib → 1016/0 green (incl. every_fold_and_soft_
reset_entry_has_dispatch_arm pin test).
Audit: docs/dqn-wire-up-audit.md Phase F-3b section.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
de64935b78 |
feat(sp22-vnext): Phase F-3 — K=3 CE EMA producer + ISV slot
Adds observability infrastructure for Phase F smoke validation: new ISV slot AUX_TRADE_OUTCOME_CE_EMA_INDEX = 538 tracks the K=3 head's batch-mean sparse cross-entropy EMA. Producer kernel registered + cubin-built; launcher wireup follows in Phase F-3b commit. Changes: - sp22_isv_slots.rs: new pub const AUX_TRADE_OUTCOME_CE_EMA_INDEX = 538 - gpu_dqn_trainer.rs ISV_TOTAL_DIM: 538 → 539 - NEW kernel aux_outcome_ce_ema_kernel.cu: single-thread single-block direct-to-ISV EMA writer with Pearl A first-observation bootstrap. Fixed α=0.05 (slow-moving observability). NULL-tolerant + NaN-guarded. - build.rs: kernel registered; cubin compiles (3.2 KB) Why dedicated kernel (not extension of aux_heads_loss_ema_update): The K=2/K=5 EMA writes through Pearls A+D's 2-stage producer scratch. Extending it would require allocating a new scratch slot, threading Pearls A+D mapping, and touching apply_pearls_ad_kernel. The K=3 head's CE is OBSERVABILITY ONLY in this commit — no Pearls A+D adaptive α needed. Minimum-scope direct-to-ISV path. Re-routable through Pearls A+D if K=3 CE later becomes a controller anchor. Smoke validation signal interpretation: - Cold-start: ISV[538] = 0.0 (FoldReset sentinel) - First step with B_valid > 0: Pearl A bootstrap → ISV[538] = first CE - Typical uniform-K=3 cold-start CE: ln(3) ≈ 1.098 - Learning signal: CE drops from ~1.098 toward 0.5-0.7 over epochs - Falsification signal: CE pinned at ln(3) for many epochs → head can't learn (label noise / under-capacity / outcomes unconditional) Phase F-3b follow-up: - Trainer launcher call after aux_heads_loss_ema_update - HEALTH_DIAG snap layout extension + console column - Reset registry entry (FoldReset sentinel 0) Verification: - cargo check -p ml clean. - cargo test -p ml --lib → 1016/0 green. Audit: docs/dqn-wire-up-audit.md Phase F-3 section. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
93aa4cd6a2 |
feat(sp22-vnext): Phase D — 12-weight W atom-shift (4 actions × 3 outcomes)
THE K=3 HEAD NOW DIRECTLY MODULATES Q-TARGETS. Phase D extends the
Phase 3 atom-shift mechanism from per-action W[4] reading single state
slot 121 to per-(action, outcome) W[4, 3] = 12 weights reading 3 state
slots [121..124).
Mathematical change: shift[a, b] now sums Σ_k W[a*K+k] × state[121+k]
= W[a, 0]*p_Profit + W[a, 1]*p_Stop + W[a, 2]*p_Timeout.
5 atom-shift kernel sites updated (all coordinated):
1. experience_kernels.cu::compute_expected_q (replay path)
2. experience_kernels.cu::mag_concat_qdir (rollout path)
3. experience_kernels.cu::quantile_q_select
4. c51_loss_kernel.cu loss numerator (next_state CVaR side)
5. c51_loss_kernel.cu Bellman target (online + target combined)
Each site replaces `W[a] × state_121` with `Σ_k W[a*K+k] × state[121+k]`
unrolled 3 times. State hoist points lift 1 scalar → 3-element array.
5 supporting kernel updates:
- aux_w_prior_init_kernel.cu: writes 12 K=3 structural priors instead
of 4. Spec prior matrix:
Short × {Profit=+0.5, Stop=-0.5, Timeout=0}
Hold × {Profit= 0, Stop=+0.5, Timeout=0}
Long × {Profit=+0.5, Stop=-0.5, Timeout=0}
Flat × {Profit= 0, Stop=+0.5, Timeout=0}
Block dim bumped 4 → 12.
- c51_aux_dw_kernel.cu: grid bumped (4,1,1) → (b0_size×K=12,1,1).
blockIdx.x decoded as (a, k); reads state slot 121+k, writes
dw_aux[a*K + k]. New kernel arg aux_outcome_k=3.
- adam_w_aux_kernel.cu: W_AUX_DIM 4 → 12. Block dim 12 threads.
Trainer-side buffer resizes:
w_aux_to_q_dir [4] → [12]
adam_m_w_aux [4] → [12]
adam_v_w_aux [4] → [12]
dw_aux_buf [4] → [12]
Rust launcher updates:
- c51_aux_dw_kernel launch: grid (4,1,1) → (12,1,1) + new aux_kto arg
- adam_w_aux_kernel launch: block (4,1,1) → (12,1,1)
- aux_w_prior_init launch: block (4,1,1) → (12,1,1)
Cold-start gracefulness preserved: state[121..124] = 0.0 at step 0
(no K=3 prediction yet from C-1 producer). Σ_k W[a*K+k] × 0 = 0 →
zero atom-shift across all actions. After step 1+ when C-1 producer
fires, real softmax probs activate the prior W's structural bias and
Adam refines from there.
End-to-end K=3 → Q-target chain now active:
K=3 fwd (B3/B4) → softmax → C-1 producer → prev_aux_outcome_probs
→ C-2 state gather → state[121..124) → Phase D atom-shift
→ Q-target z_n + Σ_k W[a*K+k] × prob[k]
→ Bellman target + argmax + action_select all see aux's outcome
prediction.
The K=3 head now influences policy via TWO paths: state input (Phase
C-2) AND Q-target modulation (Phase D).
Verification:
- cargo check -p ml clean.
- cargo test -p ml --lib → 1016/0 green.
Remaining vNext work:
- Phase E: dW backward gradient validation tests
- Phase F: Validation smoke at structural prior — decisive spec test
- B5b-2 (deferred): collector trade plan launch
Audit: docs/dqn-wire-up-audit.md Phase D section.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
d2331f2f62 |
feat(sp22-vnext): Phase B5b — full plan-conditioning integration
The K=3 head's input is now [h_s2_aux (256) || plan_params (6)] = 262- dim, matching the spec's intended architecture. Implements the input concat in both the trainer's replay-batch path and the collector's rollout-step path, with appropriate handling of the backward-side stride mismatch. NEW kernel strided_row_saxpy_kernel.cu: row-truncating SAXPY that accumulates first n_cols_copy columns per row of src [B, src_cols] into dst [B, dst_cols] scaled by alpha. Handles stride mismatch (src_cols != dst_cols). Needed because backward emits dh_s2_aux_to_buf [B, 262] but dh_s2_aux_accum [B, 256] only consumes first 256 cols. PLAN_PARAM_DIM = 6 constant in gpu_aux_heads.rs. Ops struct updates: - AuxTradeOutcomeForwardOps gains concat_kernel + launch_concat() - AuxTradeOutcomeBackwardOps gains strided_saxpy_kernel + launch_strided_row_saxpy() - Both load new cubins in new() Weight tensor resize: - sizes[163] = H × (SH2 + PLAN_PARAM_DIM) = 128 × 262 = 33,536 floats - fan_dims[163] = (H, SH2 + PLAN_PARAM_DIM) Trainer changes: - New aux_to_input_buf [B × 262] field - aux_dh_s2_to_buf resized to [B × 262] - aux_partial_to_w1 resized to [B × H × 262] - max_aux_tensor_len bumped for param_grad_final scratch Trainer forward (aux_heads_forward): - Concat h_s2_aux + plan_params_buf → aux_to_input_buf - forward() with SH2_TOTAL=262 Trainer backward (aux_heads_backward): - backward() with SH2_TOTAL=262; reads aux_to_input_buf - saxpy_f32_kernel SAXPY for dh_s2_aux REPLACED by launch_strided_row_saxpy: copies only first SH2=256 cols per row; trailing 6 cols (plan_params gradient) are dropped — STOP-GRAD on trade plan head from aux loss. Collector forward (rollout): - New exp_aux_to_input_buf [N × 262] field - Concat with plan_params_ptr = 0 (NULL) → zero-fill trailing 6 cols - forward() with SH2_TOTAL=262 Train/inference asymmetry (documented): - Trainer: real plan_params from trade plan head output - Collector: zeros (no trade plan launch in collector) - The head is trained on real plan-conditional outcomes but queried at rollout time with plan_params=0. Phase B5b-2 follow-up would add a trade plan launch to the collector to resolve. Deferred — current state is functional, head still receives plan signal during training. Stop-grad on plan_params: backward writes full [B, 262] gradient, but strided SAXPY only copies first 256 cols. Trade plan head weights NOT trained by K=3 aux loss in this commit. Verification: - cargo check -p ml clean. - cargo test -p ml --lib → 1016/0 green. Phase D next: 12-weight W atom-shift (4 actions × 3 outcomes). Audit: docs/dqn-wire-up-audit.md Phase B5b section. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |