7c19b59033
Add `GpuIqnHead::sync_target_from_online` (DtoD copy of online_params → target_params) and call it from `FusedTrainingCtx::reset_for_fold` alongside the existing IQN Adam reset. Root cause: at fold boundary the main DQN does shrink-and-perturb + hard target sync (gpu_dqn_trainer.rs:12495 — the latter explicitly added to prevent fold-1 grad explosion). The IQN head was added later but only had Polyak EMA `target_ema_update` — no hard-sync existed. Polyak at τ=0.005/step closes ~0.5%/step, far too slow to close the fold-boundary online↔target gap before inflated Bellman TD-error compounds geometrically through the IQN backward pass. Bug signature this resolves: F0 ep5 Q=+0.79 (healthy) F1 ep1 Q=+0.82 (boundary fine; c51_alpha warmup blends IQN low) F1 ep2 Q=+2.23 (drift starts — c51_alpha ramp + IQN online↔target gap widens) F1 ep3 Q=+4.05 F1 ep4 Q=+10.66 (geometric ~2.3×/epoch) F1 ep5 NaN at step 5 (fp32 overflow past atom support) Per `feedback_no_partial_refactor.md`: when a shared contract (fold-boundary state migration) changes, every consumer must migrate together. The main DQN sync was added explicitly — the IQN, TLOB, IQL, and aux Adam states were added later but never extended to that contract. This commit closes the IQN gap (primary). Aux Adam + IQN readiness gaps follow in subsequent commits. Touched: - gpu_iqn_head.rs: new `sync_target_from_online` (mirrors gpu_dqn_trainer.rs:12495 exactly) - fused_training.rs: call the new sync inside the existing IQN reset block - docs/dqn-wire-up-audit.md: Invariant 7 entry covering Fix 1 of 3 cargo check -p ml --lib clean at 13 warnings (workspace baseline). No fingerprint change. DtoD memcpy only — no HtoD/HtoH/DtoH paths. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ml
10-model ML ensemble for the Foxhunt HFT system, built on Candle v0.9.1.
Models
- DQN (Rainbow) — deep Q-network with prioritized replay, dueling heads, noisy nets
- PPO — proximal policy optimization with GAE, LSTM policies, clip-higher
- TFT — temporal fusion transformer for multi-horizon forecasting
- Mamba2 — state space model for sequence prediction
- Liquid Networks — biologically inspired networks for non-stationary data
- TLOB — transformer-based limit order book analysis
- KAN — Kolmogorov-Arnold networks
- xLSTM — extended LSTM architecture
- TGGN — temporal graph neural network
- Diffusion — diffusion-based generative model
Key Modules
ensemble— model ensemble coordination and confidence aggregationhyperopt— PSO-based hyperparameter optimization with per-model adapterstrainers— unified training loops (DQN, PPO, supervised)inference—InferenceAdaptertrait for predictioncheckpoint— model checkpointing and restorationevaluation— walk-forward evaluation pipeline
Usage
use ml::dqn::DQN;
use ml::ppo::PpoTrainer;