jgrusewski 6d433784f4 feat(rl): R3 — GPU-resident EMA + advantage/return kernels
Closes defect #5 from the flawed Phase F+G arc (feedback_cpu_is_read_only
violation in step_with_lobsim's host advantage + EMA loops) by landing
the GPU primitives those loops will become in R6.

Three new kernels, each with a GPU-oracle gate test:

1. ema_update_on_done.cu — done-gated EMA producer.
   - Slot-parameterised (one kernel, 3 callers in R5 covering
     mean_abs_pnl_ema, q_divergence_ema, td_kurtosis_ema).
   - Shared-mem tree reduce, no atomicAdd (feedback_no_atomicadd).
   - Per pearl_first_observation_bootstrap: sentinel-zero ISV → first
     observation replaces directly. Defers bootstrap if mean_obs == 0
     to avoid writing a degenerate sentinel that would be re-bootstrapped
     next call.
   - Per pearl_wiener_alpha_floor_for_nonstationary: Wiener-α blend on
     subsequent calls; caller pre-floors α at 0.4.

2. ema_update_per_step.cu — per-step EMA producer (no done-gate).
   - Slot-parameterised (kl_pi_ema, entropy_observed_ema,
     advantage_var_ratio_ema, mean_trade_duration_ema in R5).
   - Same shared-mem tree reduce + bootstrap discipline as
     ema_update_on_done.

3. compute_advantage_return.cu — element-wise
   returns[b] = r + γ(1-done)·V(s_{t+1}); advantages[b] = returns − V(s_t).
   - Reads γ from ISV[400] (R1 bootstrap = 0.99).
   - Trivially parallel, one thread per batch entry; no atomics.

Rust launchers added to IntegratedTrainer:
- launch_ema_update_on_done(slot, alpha, obs_d, dones_d, b_size)
- launch_ema_update_per_step(slot, alpha, obs_d, b_size)
- launch_compute_advantage_return(rewards_d, dones_d, v_t_d, v_tp1_d,
                                  returns_d, advantages_d, b_size)

3 cubin includes, 3 module/function fields, loaders in new() between
the rl_reward_scale_controller load and the with_controllers_bootstrapped
call so the new fields are populated by struct construction.

GPU-oracle tests in tests/r3_ema_advantage.rs (per
feedback_no_cpu_test_fallbacks every oracle is either the kernel's
documented bootstrap behaviour or an analytical property of the
formula, not a CPU reference):

  R3.1: ema_update_on_done bootstrap path — sentinel-zero ISV + one
        observation k → ISV[slot] == k exactly. Negative invariant:
        hold-only step (dones all zero) preserves the EMA.
  R3.2: ema_update_per_step convergence — feed obs=5.0 for 50 steps
        with α=0.4 → ISV[slot] → 5.0 within 1e-4 (EMA of constant =
        constant).
  R3.3: compute_advantage_return formula — r=0, done=0, v_t=v_tp1=k,
        γ=0.99 → returns=γk=4.95, advantages=(γ−1)k=−0.05. Negative
        invariant: done=1 + r=0 zeros the future-value bootstrap
        (returns=0, advantages=−k).

Build cache-bust v26.

cargo check + cargo build --test r3_ema_advantage on ml-alpha green.
Pre-existing heads_bit_equiv.rs index-out-of-bounds failure persists
(unrelated; pre-Phase E).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 09:56:28 +02:00

Foxhunt

Production HFT trading system in Rust.

Architecture

The workspace contains 32 crates organized as follows:

Core Libraries (16)

Crate Purpose
trading_engine Order processing, FIX 4.4, IB TWS, SIMD, RDTSC timing
risk VaR, Kelly, circuit breakers, kill switches, compliance
risk-data Risk data types and shared structures
trading-data Trading data types
ml DQN Rainbow, PPO, TFT, Mamba2, ensemble inference
ml-data ML data types and feature definitions
data Market data ingestion and storage
backtesting Replay engine, strategy tester
adaptive-strategy Ensemble execution, microstructure analysis
common Shared types, resilience, error handling
storage S3 and local model storage
model_loader Model serialization and loading
market-data Market data feed handlers
database PostgreSQL access layer (SQLx)
config Configuration management
tli CLI commands and tooling

Services (8)

Service Purpose
backtesting_service gRPC backtesting service
broker_gateway_service FIX routing, broker connectivity
trading_service Core trading operations
ml_training_service Model training orchestration
data_acquisition_service Market data acquisition
trading_agent_service Autonomous trading agents
api_gateway gRPC API gateway with auth
web-gateway Axum REST + WebSocket gateway

Frontend

web-dashboard/ -- React 19 + TypeScript + Vite + TradingView charts.

Building

# Check compilation (no PostgreSQL required)
SQLX_OFFLINE=true cargo check --workspace

# Run tests for a specific crate
SQLX_OFFLINE=true cargo test -p <crate> --lib

# Clippy
SQLX_OFFLINE=true cargo clippy --workspace

ML Models

Four production model architectures on Candle v0.9.1 with CUDA:

  • DQN Rainbow -- Deep Q-Network with prioritized replay, dueling heads, noisy nets
  • PPO -- Proximal Policy Optimization with GAE and LSTM policies
  • TFT -- Temporal Fusion Transformer for multi-horizon forecasting
  • Mamba2 -- State space model for sequence prediction

Each model has a standalone trainer and a UnifiedTrainable adapter for the hyperopt pipeline.

Infrastructure

  • Git: Gitea at git.fxhnt.ai (Tailscale-only), Scaleway DEV1-S
  • Observability: OpenTelemetry OTLP (env OTEL_EXPORTER_OTLP_ENDPOINT)
  • Database: PostgreSQL with SQLx offline mode for CI

License

Proprietary. All rights reserved.

Description
No description provided
Readme 849 MiB
Languages
Rust 88.2%
Cuda 7.7%
Python 1.3%
Shell 1.1%
PLpgSQL 0.8%
Other 0.8%