4d4cd996db
Closes the Phase 3.2 forward-reference loop in the SP20 aggregator.
Previously `out_inputs->per_bar_hold_reward = 0.0f` was hardcoded; the
per-bar Hold opportunity-cost producer existed
(experience_kernels.cu:3823 — `per_bar_opp_cost = -aux_conf × cost_scale`)
and wrote to `hold_baseline_buffer` and `r_micro` directly, but never
reached HOLD_REWARD_EMA. Result: HOLD_REWARD_EMA frozen at sentinel 0.0
across all observed epochs; the SP20 reward centering loop
(`r_micro += per_bar_opp_cost - HOLD_REWARD_EMA`) stayed uncentered,
biasing the policy's reward signal away from zero-mean.
T3.3 wireup mirrors the T2.2 alpha refactor: a per-env scratch buffer
that the producer writes at every step (alongside the existing
`hold_baseline_buffer` write), and the aggregator reads at the same
step. Sums opp_cost over Hold-direction envs only; emits the mean as
`per_bar_hold_reward`. The HOLD_REWARD_EMA's gate (`hold_fraction > 0.5f`
from T3.2) preserves the strict-majority semantic from before.
Atomic across producer site, kernel signature, aggregator, launcher,
collector, and tests (per feedback_no_partial_refactor):
- experience_kernels.cu — new `float* per_bar_opp_cost_per_env`
kernel arg, NULL-tolerant write at line ~3823.
- sp20_aggregate_inputs_kernel.cu — new arg, 6th shmem stripe
(`sh_opp_cost_sum`), per-thread Hold-gated accumulation, tree
reduction extended to 6 stripes, output `per_bar_hold_reward =
opp_cost_sum / hold_count` when hold_count > 0 else 0.
- sp20_aggregate_inputs.rs — launcher signature, dynamic_shmem_bytes
5→6 stripes, doc table, internal test.
- gpu_experience_collector.rs — new `per_bar_opp_cost_per_env:
CudaSlice<f32>` field, alloc, struct construction, kernel-arg
pass at both experience_env_step and sp20_aggregate_inputs
launches (raw_ptr).
- sp20_aggregate_inputs_test.rs — helper renamed
`run_kernel_with_is_win_and_opp_cost`, 4 existing sites pass
NULL, 2 NEW oracle tests verifying real producer + NULL fallback.
- sp20_phase1_4_wireup_test.rs — NULL fallback at the wireup site;
HOLD_REWARD_EMA-stays-at-zero assertion remains valid.
Verification:
- cargo check -p ml --tests: passes (warnings only)
- cargo test -p ml --test sp20_aggregate_inputs_test --features cuda
-- --ignored: 12/12 GPU oracle tests pass on RTX 3050 Ti, including
both new T3.3 tests (per_bar_hold_reward_means_over_hold_envs_only,
null_per_bar_opp_cost_emits_zero).
- cargo test -p ml --test sp20_phase1_4_wireup_test --features cuda
-- --ignored: 2/2 pass under the new NULL-tolerant contract.
Plan reference: docs/plans/2026-05-10-sp21-train-eval-coherence-isv-defrost.md
Tier 3 status: T3.1 ✓, T3.2 ✓, T3.3 ✓ (this commit), T3.4 withdrawn,
T3.5 cascade-pending, T3.6 withdrawn.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ml
10-model ML ensemble for the Foxhunt HFT system, built on Candle v0.9.1.
Models
- DQN (Rainbow) — deep Q-network with prioritized replay, dueling heads, noisy nets
- PPO — proximal policy optimization with GAE, LSTM policies, clip-higher
- TFT — temporal fusion transformer for multi-horizon forecasting
- Mamba2 — state space model for sequence prediction
- Liquid Networks — biologically inspired networks for non-stationary data
- TLOB — transformer-based limit order book analysis
- KAN — Kolmogorov-Arnold networks
- xLSTM — extended LSTM architecture
- TGGN — temporal graph neural network
- Diffusion — diffusion-based generative model
Key Modules
ensemble— model ensemble coordination and confidence aggregationhyperopt— PSO-based hyperparameter optimization with per-model adapterstrainers— unified training loops (DQN, PPO, supervised)inference—InferenceAdaptertrait for predictioncheckpoint— model checkpointing and restorationevaluation— walk-forward evaluation pipeline
Usage
use ml::dqn::DQN;
use ml::ppo::PpoTrainer;