67dd414cb3
Per-param-group Adam hyperparameters from per-group gradient
direction-stability EMA + L2 norm. 8 SP4 param groups × 3 hparams = 24
ISV slots [226..250).
Two-kernel chain:
grad_cosine_sim_update — per-group cosine_sim of curr vs prev grads
+ L2 norm; writes back grad_curr → grad_prev
for next step's comparison.
pearl_4_adam_hparams_update — β1/β2/ε from cosine + l2_norm.
Structural envelopes (Invariant 1 anchors per spec line 88):
β1 ∈ [0.85, 0.95]; β2 ∈ [0.99, 0.9995]; ε ∈ [1e-10, 1e-6]
ALPHA_META migration (Option A — atomic shared-contract migration per
feedback_no_partial_refactor): apply_pearls_ad_kernel.cu and the
launch_apply_pearls Rust wrapper now take alpha_meta as a parameter.
All 45 existing call sites (SP4 + SP5 producers) pass the prior default
1.0e-3 explicitly via crate::cuda_pipeline::sp4_wiener_ema::ALPHA_META.
Pearl 4's apply_pearls calls pass 5.0e-4 (half the default per spec
line 89) to limit β change rate.
Theoretical caveat: adaptive β breaks Adam's constant-β convergence
proof. Mitigations: structural envelopes + halved ALPHA_META + Pearls
A+D smoothing. Fall-back path defined: revert + ε-only adaptive
variant if Layer C destabilization observed.
New mapped-pinned buffer: grad_prev_buf_per_group [TOTAL_PARAMS] for
cosine-sim previous-step direction storage. Mapped-pinned i32 mirror
of grad_buf layout.
producer_step_scratch_buf grew 131 → 171 (40 new outputs: 8 cosine_sim
+ 8 l2_norm + 24 β1/β2/ε). wiener_state_buf already at 543 (A1 sized
for entire SP5 block).
StateResetRegistry: 4 new FoldReset entries: sp5_adam_beta1,
sp5_adam_beta2, sp5_adam_eps, sp5_grad_prev_buf. All sentinel 0 →
bootstrap to envelope midpoint via Pearls A+D + cosine_sim=0 fall-back
on first step of new fold.
Adam-launcher consumer migration deferred to Layer B.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ml
10-model ML ensemble for the Foxhunt HFT system, built on Candle v0.9.1.
Models
- DQN (Rainbow) — deep Q-network with prioritized replay, dueling heads, noisy nets
- PPO — proximal policy optimization with GAE, LSTM policies, clip-higher
- TFT — temporal fusion transformer for multi-horizon forecasting
- Mamba2 — state space model for sequence prediction
- Liquid Networks — biologically inspired networks for non-stationary data
- TLOB — transformer-based limit order book analysis
- KAN — Kolmogorov-Arnold networks
- xLSTM — extended LSTM architecture
- TGGN — temporal graph neural network
- Diffusion — diffusion-based generative model
Key Modules
ensemble— model ensemble coordination and confidence aggregationhyperopt— PSO-based hyperparameter optimization with per-model adapterstrainers— unified training loops (DQN, PPO, supervised)inference—InferenceAdaptertrait for predictioncheckpoint— model checkpointing and restorationevaluation— walk-forward evaluation pipeline
Usage
use ml::dqn::DQN;
use ml::ppo::PpoTrainer;