2e4c7ebf60
Phase 3a builds on the Phase A foundation (464bc5f7a). Migrates the 7-component reward_components_per_sample contract atomically across producer + readers + buffer alloc, and installs the β producer at training-side trade-close. α kernels exist (compiled in Phase A) but are not yet launched in captured graphs — that's Phase 3b alongside the SP11 controller extension and A2 eval-side aux infrastructure. Why split into 3a/3b ──────────────────── Full Phase 3 (α + β + SP11 controller + A2) is ~25-35 hr engineering spanning ~19 files. Phase 3a is the SAFE atomic contract migration (7-stride buffer + β producer; no α captured-graph integration yet) — runtime-equivalent to Phase 2 (β no-op at scale_β=0 sentinel; α kernels loaded but never launched). This commits the foundation + contract change as a clean checkpoint per `feedback_no_partial_refactor` (the 7-component contract spans every consumer; partial migration would produce stride mismatches; this commit migrates ALL consumers that read the buffer). Files ───── - crates/ml/src/cuda_pipeline/experience_kernels.cu: Preamble doc → 7-component layout. Buffer stride `* 6 +` → `* 7 +` (~14 sites, atomic). 7th-slot init at the per-step zero block (`rc[6] = 0.0f`). β producer at the segment_complete branch: r_aux_align = scale_β * max(0, aux × pos_sign) * max(0, pnl) with NULL-safe fallbacks (aux_dir_prob_per_env NULL OR isv_signals_ptr NULL → β no-op). Two new kernel args: aux_dir_prob_per_env + aux_align_scale_idx (slot index for scale_β, decoupled per the loss_cap_idx pattern). - crates/ml/src/cuda_pipeline/reward_component_ema_kernel.cu: Stride `idx * 6` → `idx * 7` (3 sites). Iteration stays c=0..5; the 7th component (aux_align) is intentionally NOT EMA'd here. A dedicated reward_aux_align_ema_kernel writing directly to ISV[REWARD_AUX_ALIGN_EMA_INDEX=536] is Phase 3b scope (avoids extending the apply_pearls_ad chain). Preamble doc updated. - crates/ml/src/cuda_pipeline/reward_decomp_diag_kernel.cu: #define RCP_NUM_COMPS 6 → 7. The kernel's per-bin abs-sum (col 3) now naturally includes r_aux_align; popart/micro/ opp_cost per-bin means unchanged. - crates/ml/src/cuda_pipeline/reward_component_mag_ratio_compute_kernel.cu: Documentation only: aux_align excluded from the 6-axis cf_others ratio (non-contiguous with cf_others_base_slot at 64..68; aux_align EMA lives at ISV[536]). - crates/ml/src/cuda_pipeline/gpu_experience_collector.rs: Buffer alloc `total_output * 6` → `* 7` (critical for runtime safety — partial migration would produce OOB writes since experience_env_step writes to `out_off * 7 + N`). experience_env_step launcher gains 2 new `.arg(...)` calls passing `self.prev_aux_dir_prob.raw_ptr()` and `SP22_AUX_ALIGN_SCALE_INDEX as i32`. - docs/dqn-wire-up-audit.md: Phase 3a entry documenting the partial commit + Phase 3b remaining-work breakdown. Verification ──────────── - cargo check -p ml --features cuda: 0 errors, 21 pre-existing warnings (Phase 2 baseline parity). - All nvcc cubins recompile (experience_kernels, reward_component_ema, reward_decomp_diag, reward_component_mag_ratio_compute, plus Phase A's aux_to_q_dir_bias_kernel + backward). - Runtime equivalent to Phase 2: β no-op (scale_β=0 sentinel since SP11 controller not yet extended), α no-op (kernels dead-code until Phase 3b wires them into captured graphs). Phase 3b scope (resume in fresh session) ──────────────────────────────────────── - B6: SP11 controller extension (w_aux_align emit at ISV[537]) - B7: HEALTH_DIAG snap layout extension - B9: α plumbing — W_aux_to_Q_dir param + Adam + captured-graph forward + backward integration in gpu_dqn_trainer.rs - B10/B11: HEALTH_DIAG print-line extensions - C1: α forward in collector's rollout-time captured graph - D1-D7: A2 eval-side aux trunk + α + state-gather wiring - E + F: verification gates + atomic Phase F commit + smoke + verdict Refs ──── - docs/plans/2026-05-12-sp22-h6-phase3-alpha-beta.md (spec) - docs/plans/2026-05-13-sp22-h6-phase3-alpha-beta-runbook.md (runbook) -464bc5f7a(Phase A foundation) - pearl_no_partial_refactor (atomic 7-component contract migration) - pearl_event_driven_reward_density_alignment (β at segment_complete) - pearl_one_unbounded_signal_per_reward (β bounded by scale_β + alignment caps) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ml
10-model ML ensemble for the Foxhunt HFT system, built on Candle v0.9.1.
Models
- DQN (Rainbow) — deep Q-network with prioritized replay, dueling heads, noisy nets
- PPO — proximal policy optimization with GAE, LSTM policies, clip-higher
- TFT — temporal fusion transformer for multi-horizon forecasting
- Mamba2 — state space model for sequence prediction
- Liquid Networks — biologically inspired networks for non-stationary data
- TLOB — transformer-based limit order book analysis
- KAN — Kolmogorov-Arnold networks
- xLSTM — extended LSTM architecture
- TGGN — temporal graph neural network
- Diffusion — diffusion-based generative model
Key Modules
ensemble— model ensemble coordination and confidence aggregationhyperopt— PSO-based hyperparameter optimization with per-model adapterstrainers— unified training loops (DQN, PPO, supervised)inference—InferenceAdaptertrait for predictioncheckpoint— model checkpointing and restorationevaluation— walk-forward evaluation pipeline
Usage
use ml::dqn::DQN;
use ml::ppo::PpoTrainer;