62b5a50e8b
Smoke v1 (train-grfcw) evaluate phase failed with "Failed to load DQN
checkpoint" for fold 0 and fold 1. MinIO log inspection confirmed
checkpoints WERE saved (1431144 bytes each) — the failure was
eval-side shape mismatch.
Root cause:
- Training uses STATE_DIM=128 (ml_core::state_layout), num_actions=108
(factored b0*b1*b2*b3=4*3*3*3), num_order_types=3,
num_urgency_levels=3.
- evaluate_baseline CLI defaults: --feature-dim=54, --num-actions=5
(legacy from pre-branching DQN era).
- Loading 128-state-dim 108-action checkpoint into 54-feature 5-action
net → tensor shape mismatch → `load_from_safetensors` returned
parse error → `with_context(...)` wrapped it as the generic "Failed
to load DQN checkpoint" message, hiding the actual shape error.
- Both GPU and CPU eval paths hit the same root cause.
Fix:
Both eval paths now call `DQNConfig::from_safetensors_file(&ckpt_path)`
to read architecture-critical fields from the checkpoint's embedded
metadata (state_dim, num_actions, hidden_dims, num_order_types,
num_urgency_levels, dueling_hidden_dim, num_atoms, gamma). Eval-time
fields (LR, epsilon, buffer caps) overridden; hyperopt-derived gamma/
v_min/v_max applied if present in hyperopt config.
Older checkpoints without embedded metadata fall back to CLI-args-built
config + warn! log. All production SP21+ checkpoints embed metadata
via the existing DQNConfig::checkpoint_metadata path.
Files changed:
- crates/ml/examples/evaluate_baseline.rs: shape-aware config for both
dqn_eval_gpu_path (line ~1238) and dqn_eval_cpu_path (line ~1029)
- docs/dqn-wire-up-audit.md: 2026-05-11 audit entry
Verification:
- cargo check -p ml --examples --features cuda: 0 errors
- cargo test -p ml --lib financials: 7/7 (unchanged)
- cargo test -p ml --lib sp21_isv_slots: 4/4 (unchanged)
Behavioral gate: smoke v3 (train-psf86, in-flight on 2937da889) won't
have this fix; smoke v4 dispatch on this commit will validate
evaluate phase succeeds for all folds. Look for
"[DQN GPU] Architecture from checkpoint: ..." log line per fold.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ml
10-model ML ensemble for the Foxhunt HFT system, built on Candle v0.9.1.
Models
- DQN (Rainbow) — deep Q-network with prioritized replay, dueling heads, noisy nets
- PPO — proximal policy optimization with GAE, LSTM policies, clip-higher
- TFT — temporal fusion transformer for multi-horizon forecasting
- Mamba2 — state space model for sequence prediction
- Liquid Networks — biologically inspired networks for non-stationary data
- TLOB — transformer-based limit order book analysis
- KAN — Kolmogorov-Arnold networks
- xLSTM — extended LSTM architecture
- TGGN — temporal graph neural network
- Diffusion — diffusion-based generative model
Key Modules
ensemble— model ensemble coordination and confidence aggregationhyperopt— PSO-based hyperparameter optimization with per-model adapterstrainers— unified training loops (DQN, PPO, supervised)inference—InferenceAdaptertrait for predictioncheckpoint— model checkpointing and restorationevaluation— walk-forward evaluation pipeline
Usage
use ml::dqn::DQN;
use ml::ppo::PpoTrainer;