5b394f1035
Closes the Phase 1.3.b deferred per-env redesign per feedback_no_partial_refactor. Path A (env-0-canonical, commit132609724) made every downstream consumer read env-0's DD context; production envs each have their own DD trajectory, so when env-3 was at 30% DD and env-0 was at ATH, the model learning from env-3's transitions saw dd_pct=0 and silently skipped the recovery shaping. Path B threads each env's actual DD context through the reward shaping atomically: (1) dd_state_kernel reshape — grid [n_envs, 1, 1], one thread per env, writes 6 scalars per env to a new per-env tile dd_state_per_env [n_envs * 6]. No more ISV scalar writes. (2) NEW dd_state_reduce_kernel — single-block tree-reduce (no atomicAdd; BLOCK=256 with strided initial pass for n_envs up to 32768 on H100). Mean-aggregates per-env tile → 6 scalar ISV slots [401..407) for HEALTH_DIAG diagnostic. Max-aggregates DD_PERSISTENCE → new slot DD_PERSISTENCE_MAX_INDEX=443 for plasticity injection trigger (one shared advantage-head ⇒ global firing ⇒ max-aggregate is the only correct rule). ISV_TOTAL_DIM 443→444; SP15_SLOT_END 443→444; SP15_SLOT_COUNT 46→47. (3) compute_sp15_final_reward_kernel migration — per-(i,t) per-env DD lookup via env_id = (idx % (N*L)) / L. Helpers sp15_dd_asymmetric_reward and sp15_dd_penalty migrated to take dd_pct + dd_current as scalar parameters (the kernel reads from the per-env tile, threads scalars in). On-policy + CF threads at the same (i,t) read the SAME tile entry (one DD trajectory per env, shared across slot kinds). (4) plasticity_injection_kernel migration — persistence read switched from ISV[404] (mean) to ISV[443] (max). One set of advantage weights ⇒ ANY env exceeding the threshold should arm the gate. (5) Per-env tile owned by GpuExperienceCollector (not the trainer) — the collector knows alloc_episodes (= n_envs); the trainer's batch_size is a different quantity. Reset to zero via the sp15_dd_state_per_env registry-arm dispatch. (6) Per-step launch order: dd_state → dd_state_reduce → alpha_split_producer → final_reward, all on the same stream (CUDA serialises producer→consumer without explicit event sync). (7) HEALTH_DIAG semantic shift (documented breaking change): slots 401-406 now report cross-env mean, not env-0 value. For n_envs=1 smoke configs the mean equals env-0's value (bit-stable migration). (8) Layout fingerprint break: added markers DD_PERSISTENCE_MAX=443; ISV_TOTAL_DIM=444; DD_STATE_PER_ENV=sp15_phase_1_3_b_followup. Pre-followup checkpoints will not load (greenfield OK per spec Q1). (9) 6 oracle tests migrated + 1 NEW behavioral test `dd_state_per_env_diverge_independently` — two-env config (env-0 in recovery, env-1 deepening) verifies independent trajectories + mean-aggregate + max-aggregate semantics. Phase 1.3.b-followup-B (separate split): dd_trajectory_decreasing_kernel + per_insert_pa migration is NOT in scope. The current Wave 4.3 reads ISV[439] at insert-batch time (epoch end) — applied uniformly to ALL inserted transitions (a known pre-existing limitation). Proper fix requires per-(env, t) trajectory buffer [N*L] + env_id-aware lookup in per_insert_pa via env_id = (j % (N*L)) / L. That's a different contract change; splitting preserves no-partial-refactor within each migration. Atomic per feedback_no_partial_refactor: all 5 consumers of single-env- canonical DD slots (final_reward kernel + plasticity kernel + HEALTH_DIAG diagnostic + 6 oracle tests + new behavioral test) migrate to per-env tile lookup in this commit; the new DD_PERSISTENCE_MAX ISV slot lands with its sole consumer (plasticity). Verified: SQLX_OFFLINE=true cargo check -p ml --features cuda clean; cargo check -p ml --features cuda --tests clean; CUDA_COMPUTE_CAP=86 cargo test -p ml --test sp15_phase1_oracle_tests --features cuda -- --ignored: 17/17 oracle tests green (2 dd_state incl. new per-env behavioral + 5 plasticity + 3 dd_trajectory + 6 final_reward + 1 per_sampler); cargo test -p ml --features cuda --lib: 947 pass / 12 fail HOLDS the483cef454baseline (no new regressions). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ml
10-model ML ensemble for the Foxhunt HFT system, built on Candle v0.9.1.
Models
- DQN (Rainbow) — deep Q-network with prioritized replay, dueling heads, noisy nets
- PPO — proximal policy optimization with GAE, LSTM policies, clip-higher
- TFT — temporal fusion transformer for multi-horizon forecasting
- Mamba2 — state space model for sequence prediction
- Liquid Networks — biologically inspired networks for non-stationary data
- TLOB — transformer-based limit order book analysis
- KAN — Kolmogorov-Arnold networks
- xLSTM — extended LSTM architecture
- TGGN — temporal graph neural network
- Diffusion — diffusion-based generative model
Key Modules
ensemble— model ensemble coordination and confidence aggregationhyperopt— PSO-based hyperparameter optimization with per-model adapterstrainers— unified training loops (DQN, PPO, supervised)inference—InferenceAdaptertrait for predictioncheckpoint— model checkpointing and restorationevaluation— walk-forward evaluation pipeline
Usage
use ml::dqn::DQN;
use ml::ppo::PpoTrainer;