8aef59f735
Replaces eval-mode Boltzmann softmax with strict argmax + uniform-sample- among-tied-indices (|q_a − q_b| < 1e-6). Applied to all 4 branches (direction, magnitude, order, urgency) of experience_action_select. Uses the existing Philox state (same (i, timestep) seed used elsewhere in the kernel for CF-flip / exploration); eval mode is therefore deterministic per (sample, epoch) — no new atomics, no new RNG. Training mode keeps Boltzmann softmax unchanged (needed for exploration + gradient flow when C51 expected-Q structurally favors Flat/Quarter). Root-cause re-diagnosis (commit7e78bf4f8+41b0c559c): Task 2.0 instrumentation + bug fixes revealed Track 1 H4 was a measurement artefact produced by two silent bugs in per_branch_grad_norms. Real mag/dir gradient ratio is 50-400× (magnitude over-fed, NOT starved). The genuine observable problem is H10 — argmax tie-break at eval, which is what this commit closes. Why the previous eval-mode Boltzmann collapsed: with Q-range ≪ tau floor (0.01), softmax was effectively uniform at eval, but cumulative sampling with a single Philox draw deterministically picked the first index over every tied bin — hence eval_dist [eq=1.000 eh=0.000 ef=0.000] across all 20 baseline epochs despite training-mode ent_mag ≈ 0.99. Strict argmax honours the learned preference where it exists, tie-break only applies on genuine ties < 1e-6. Results from 20-epoch smoke (magnitude_distribution): Baseline (HEAD pre-fix): training dist [dist_q=0.577 dist_h=0.166 dist_f=0.257] eval dist [eq=1.000 eh=0.000 ef=0.000] Post-fix (this commit): training dist [dist_q=0.577 dist_h=0.166 dist_f=0.257] (unchanged — eval-only path) eval dist [eq=0.580 eh=0.420 ef=0.000] eh + ef = 0.420 ≥ 0.30 regression gate → PASS. Training-mode distribution is bit-identical as expected: the fix only touches the eval_mode == 1 branch of experience_action_select. Regression assertion in magnitude_distribution smoke requires eh + ef ≥ 0.30 (the production threshold from Task 2.9 §2.9 gate). Closes Track 1 H10 CONFIRMED verdict. With H4 REJECTED by Task 2.0's real data, H10 is now Phase 2's primary fix. If future training regresses this ratio, the fallback is the "delete magnitude branch" path (H9 spec §5.1 proposed fix — 108 → 36 action space).
ml
10-model ML ensemble for the Foxhunt HFT system, built on Candle v0.9.1.
Models
- DQN (Rainbow) — deep Q-network with prioritized replay, dueling heads, noisy nets
- PPO — proximal policy optimization with GAE, LSTM policies, clip-higher
- TFT — temporal fusion transformer for multi-horizon forecasting
- Mamba2 — state space model for sequence prediction
- Liquid Networks — biologically inspired networks for non-stationary data
- TLOB — transformer-based limit order book analysis
- KAN — Kolmogorov-Arnold networks
- xLSTM — extended LSTM architecture
- TGGN — temporal graph neural network
- Diffusion — diffusion-based generative model
Key Modules
ensemble— model ensemble coordination and confidence aggregationhyperopt— PSO-based hyperparameter optimization with per-model adapterstrainers— unified training loops (DQN, PPO, supervised)inference—InferenceAdaptertrait for predictioncheckpoint— model checkpointing and restorationevaluation— walk-forward evaluation pipeline
Usage
use ml::dqn::DQN;
use ml::ppo::PpoTrainer;