7ed4e5ca90
Five reward computation fixes in experience_env_step CUDA kernel: 1. Replace CUSUM vol proxy with ATR(14): CUSUM at feature[41] is a binary direction indicator [-1,1,0], NOT volatility. When CUSUM≈0, vol_proxy became 0.0001 causing 10000x reward amplification. ATR(14) at feature[9] is actual realized volatility — reverse the safe_normalize encoding (ln(atr)+7)/16 to recover atr_pct = exp(norm*16-7) / price. 2. Move loss aversion BEFORE squash: previously applied after hard clamp, creating asymmetric [-15, +10] range making expected reward negative even for fair strategies. Now applied pre-squash for smooth asymmetry. 3. Replace hard clamp with tanh soft squash: fmaxf(-10, fminf(10, reward)) destroyed tail information (1% and 5% wins both → 10.0). tanh preserves that larger wins produce proportionally larger rewards. 4. Remove turnover penalty: the 0.05*|delta|/max_position penalty double- counted transaction costs already deducted from cash via Almgren-Chriss impact model at line ~679, over-penalizing necessary rebalancing. 5. Clarify CUSUM spread_scale usage: CUSUM at feature[41] is correctly used as market-stress proxy for spread widening in tx cost computation — this is distinct from the (now-fixed) vol proxy for reward normalization. Also: annotate min_hold_bars=5 as hyperopt candidate. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ml
10-model ML ensemble for the Foxhunt HFT system, built on Candle v0.9.1.
Models
- DQN (Rainbow) — deep Q-network with prioritized replay, dueling heads, noisy nets
- PPO — proximal policy optimization with GAE, LSTM policies, clip-higher
- TFT — temporal fusion transformer for multi-horizon forecasting
- Mamba2 — state space model for sequence prediction
- Liquid Networks — biologically inspired networks for non-stationary data
- TLOB — transformer-based limit order book analysis
- KAN — Kolmogorov-Arnold networks
- xLSTM — extended LSTM architecture
- TGGN — temporal graph neural network
- Diffusion — diffusion-based generative model
Key Modules
ensemble— model ensemble coordination and confidence aggregationhyperopt— PSO-based hyperparameter optimization with per-model adapterstrainers— unified training loops (DQN, PPO, supervised)inference—InferenceAdaptertrait for predictioncheckpoint— model checkpointing and restorationevaluation— walk-forward evaluation pipeline
Usage
use ml::dqn::DQN;
use ml::ppo::PpoTrainer;