06dbb78ffc
Lands the per-bar Hold opportunity-cost dual emission at
experience_env_step's per-bar branches (positioned-non-event ~3650 +
flat-non-event ~3737), atomically with the trade-close consumer
wiring at the segment_complete branch (~3289 — replaces the Phase 2
Task 2.2 forward-reference placeholder `hold_baseline = 0.0f`) per
`feedback_no_partial_refactor`.
Spec §4.2 dual-emission contract:
per_bar_opp_cost = -aux_conf × cost_scale
Path 1 (Hold-only): r_micro/r_opp_cost += per_bar_opp_cost
- ISV[HOLD_REWARD_EMA]
(centered for Q-target stability)
Path 2 (always): hold_baseline_buffer[env, t % 30] = per_bar_opp_cost
(uncentered, consumed at trade close)
At trade close:
hold_baseline = sp20_sum_hold_baseline_over_trade(buf, env, 30,
current_t,
segment_hold_time)
alpha = R_event - hold_baseline # replaces Phase 2 placeholder
The summation walks backwards `min(30, segment_hold_time)` slots from
`(current_t - 1) % 30` in env i's row of the per-env circular buffer.
The trade-close bar's segment_complete branch does NOT write to the
buffer, so the most recent per-bar write is at current_t - 1.
Layout decision (plan errata Gap 9): per-env stride
`[N_envs × HOLD_BASELINE_BUFFER_SIZE = 30]` row-major. The kernel is
per-env-parallel; a single global ring would interleave bars across
envs and break trade-attribution semantic.
Trade-open tracking decision (plan errata Gap 10): NO new state slot
needed. The existing `segment_hold_time` (= saved_hold_time captured
before PS_HOLD_TIME reset at experience_kernels.cu:2618) plus
current_t suffice — walk backwards in the buffer.
Replaces SP18 D-leg `compute_sp18_hold_opportunity_cost` calls at
experience_kernels.cu:3672 and :3783. The device function in
trade_physics.cuh:655 is RETAINED — still called by
sp18_hold_opp_test_kernel.cu (oracle test surface). Per
`feedback_no_stubs`: only production callers migrate; the helper
stays for the test surface.
New device helpers (sp20_hold_baseline.cuh):
- sp20_compute_per_bar_aux_conf_k2(logits) — bit-identical to the
formula in sp20_stats_compute_kernel.cu Pass A.
- sp20_sum_hold_baseline_over_trade(buf, env, size, current_t,
segment_hold_time) — walks backwards with modulo wrap-around.
New buffer + reset infrastructure:
- GpuExperienceCollector.hold_baseline_buffer field
- HOLD_BASELINE_BUFFER_SIZE = 30 constant (re-exported)
- StateResetRegistry FoldReset entry + invariant test
- reset_named_state dispatch arm
Six new kernel args appended to experience_env_step signature:
aux_logits_per_env, hold_baseline_buffer, hold_buffer_size, aux_k,
hold_cost_scale_idx, hold_reward_ema_idx. All NULL-tolerant.
HOLD_REWARD_EMA forward-reference: ISV[516] is updated by
sp20_emas_compute_kernel reading per_bar_hold_reward from
sp20_aggregate_inputs_kernel which currently emits 0.0f as a Phase
3.2 forward-ref placeholder (line 292). The parallel `sp20-phase-2-fix`
agent owns wiring the real producer; Task 3.2's centering math
references the ISV slot (not a hardcoded 0), so when the parallel
branch lands, EMA starts updating and centering becomes load-bearing
automatically. No additional Task 3.2 work needed post-merge.
Tests (sp20_hold_baseline_test.rs, 4 GPU oracle tests):
1. aux_conf_k2_bounds_and_fixed_points — bounds, uniform/saturated/
spec-warmup/spec-confident/symmetry fixed points.
2. sum_hold_baseline_over_trade_indexing — basic indexing,
hold_time>size clamp, defensive guards, modulo wrap-around.
3. sum_hold_baseline_per_env_stride — env-stride correctness across
row-major buffer.
4. dual_emission_hold_vs_non_hold — full Path 1 + Path 2 contract,
mirrors plan §3.2 reference fixture.
Verification:
SQLX_OFFLINE=true cargo check -p ml --tests --features cuda # green
SQLX_OFFLINE=true cargo test -p ml --lib sp20 # 21/21 pass
SQLX_OFFLINE=true cargo build -p ml # cubin compile
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ml
10-model ML ensemble for the Foxhunt HFT system, built on Candle v0.9.1.
Models
- DQN (Rainbow) — deep Q-network with prioritized replay, dueling heads, noisy nets
- PPO — proximal policy optimization with GAE, LSTM policies, clip-higher
- TFT — temporal fusion transformer for multi-horizon forecasting
- Mamba2 — state space model for sequence prediction
- Liquid Networks — biologically inspired networks for non-stationary data
- TLOB — transformer-based limit order book analysis
- KAN — Kolmogorov-Arnold networks
- xLSTM — extended LSTM architecture
- TGGN — temporal graph neural network
- Diffusion — diffusion-based generative model
Key Modules
ensemble— model ensemble coordination and confidence aggregationhyperopt— PSO-based hyperparameter optimization with per-model adapterstrainers— unified training loops (DQN, PPO, supervised)inference—InferenceAdaptertrait for predictioncheckpoint— model checkpointing and restorationevaluation— walk-forward evaluation pipeline
Usage
use ml::dqn::DQN;
use ml::ppo::PpoTrainer;