jgrusewski 0a32a3bb89 feat(rl): R5 — wire 7 controllers per-step + target-net soft update
Closes defects #3 (controllers never launched) and #4 (target net
never soft-updated) from the flawed Phase F+G arc.

CONTROLLER SIGNATURE CHANGE (all 7 .cu files):
Scalar input arg → int input_slot. Each controller now reads its EMA
input from ISV[input_slot] directly inside the kernel, eliminating
the 7 DtoH-per-step host roundtrips a scalar-arg signature would have
required — per feedback_cpu_is_read_only (hot-path must be GPU-pure).

Bootstrap path unchanged: kernel reads its output slot, sees sentinel
zero, writes *_BOOTSTRAP, and returns BEFORE the input_slot read. So
R1's launch_isv_controller_3arg(controller_fn, alpha=0.4, input_slot)
works both at bootstrap (input read deferred via early return) and
at per-step (input slot has real EMA observation from R3 producers).

PER-STEP CONTROLLER LAUNCHER:
New IntegratedTrainer::launch_rl_controllers_per_step() fires all 7
controllers in sequence, each with its dedicated EMA input slot:
  ISV[400] γ              ← ISV[417] MEAN_TRADE_DURATION_EMA
  ISV[401] τ              ← ISV[418] Q_DIVERGENCE_EMA
  ISV[402] ε              ← ISV[419] KL_PI_EMA
  ISV[403] entropy_coef   ← ISV[420] ENTROPY_OBSERVED_EMA
  ISV[404] n_rollout_steps← ISV[421] ADVANTAGE_VAR_RATIO_EMA
  ISV[405] per_α          ← ISV[422] TD_KURTOSIS_EMA
  ISV[406] reward_scale   ← ISV[423] MEAN_ABS_PNL_EMA

R1's with_controllers_bootstrapped also updated to pass the input
slot indices (the bootstrap path still ignores them via early return).

TARGET-NET SOFT UPDATE (defect #4):
New cuda/dqn_target_soft_update.cu — element-wise
  target[i] = (1-τ)·target[i] + τ·current[i]
reading τ from ISV[401]. Trivially parallel, no atomicAdd. DqnHead
gains target_soft_update_fn + _target_soft_update_module fields +
soft_update_target(&isv_d) method that fires the kernel twice
(weights + biases). R6 calls this from step_with_lobsim after the
Q-head Adam update.

GATE TESTS (tests/r5_controllers_and_soft_update.rs):

G3: g3_per_step_controllers_move_isv_outputs_when_fed_real_emas
  - Verifies R1 bootstrap pre-conditions (all 7 output slots at
    documented bootstrap values; all 7 EMA-input slots at sentinel 0).
  - Populates each EMA-input slot with a distinct non-zero value via
    R3's ema_update_per_step bootstrap path (different values per slot
    so a wrong-slot wiring bug would produce out-of-range outputs).
  - Verifies the EMA producers wrote what we expected (sanity).
  - Fires launch_rl_controllers_per_step.
  - Asserts each output slot moved off its bootstrap value (catches
    "controller doesn't fire" / "reads wrong slot" / "dead kernel").

G4: g4_dqn_target_soft_update_implements_polyak_formula
  - Force-overwrite w_d with all-ones (breaks the w==target init
    symmetry so soft_update has something to blend).
  - Snapshot w_target (Xavier init values).
  - Fire dqn_head.soft_update_target with R1-bootstrapped τ=0.005.
  - For sample indices: assert target_after[i] equals
    (1-τ)·target_before[i] + τ·1.0 within 1e-6 (exact algebraic
    identity, not a CPU reference — kernel IS the kernel).
  - Negative invariant: at least one element changed.

Per feedback_no_cpu_test_fallbacks: G3 oracle is the invariant
"output != bootstrap after non-trivial input"; G4 oracle is the
algebraic identity (1-τ)·a + τ·b applied to the SAME numbers the
kernel saw — not a parallel CPU implementation.

Build cache-bust v28. cargo check + cargo build --tests on ml-alpha
green for all R-phase tests.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 10:19:13 +02:00

Foxhunt

Production HFT trading system in Rust.

Architecture

The workspace contains 32 crates organized as follows:

Core Libraries (16)

Crate Purpose
trading_engine Order processing, FIX 4.4, IB TWS, SIMD, RDTSC timing
risk VaR, Kelly, circuit breakers, kill switches, compliance
risk-data Risk data types and shared structures
trading-data Trading data types
ml DQN Rainbow, PPO, TFT, Mamba2, ensemble inference
ml-data ML data types and feature definitions
data Market data ingestion and storage
backtesting Replay engine, strategy tester
adaptive-strategy Ensemble execution, microstructure analysis
common Shared types, resilience, error handling
storage S3 and local model storage
model_loader Model serialization and loading
market-data Market data feed handlers
database PostgreSQL access layer (SQLx)
config Configuration management
tli CLI commands and tooling

Services (8)

Service Purpose
backtesting_service gRPC backtesting service
broker_gateway_service FIX routing, broker connectivity
trading_service Core trading operations
ml_training_service Model training orchestration
data_acquisition_service Market data acquisition
trading_agent_service Autonomous trading agents
api_gateway gRPC API gateway with auth
web-gateway Axum REST + WebSocket gateway

Frontend

web-dashboard/ -- React 19 + TypeScript + Vite + TradingView charts.

Building

# Check compilation (no PostgreSQL required)
SQLX_OFFLINE=true cargo check --workspace

# Run tests for a specific crate
SQLX_OFFLINE=true cargo test -p <crate> --lib

# Clippy
SQLX_OFFLINE=true cargo clippy --workspace

ML Models

Four production model architectures on Candle v0.9.1 with CUDA:

  • DQN Rainbow -- Deep Q-Network with prioritized replay, dueling heads, noisy nets
  • PPO -- Proximal Policy Optimization with GAE and LSTM policies
  • TFT -- Temporal Fusion Transformer for multi-horizon forecasting
  • Mamba2 -- State space model for sequence prediction

Each model has a standalone trainer and a UnifiedTrainable adapter for the hyperopt pipeline.

Infrastructure

  • Git: Gitea at git.fxhnt.ai (Tailscale-only), Scaleway DEV1-S
  • Observability: OpenTelemetry OTLP (env OTEL_EXPORTER_OTLP_ENDPOINT)
  • Database: PostgreSQL with SQLx offline mode for CI

License

Proprietary. All rights reserved.

Description
No description provided
Readme 849 MiB
Languages
Rust 88.2%
Cuda 7.7%
Python 1.3%
Shell 1.1%
PLpgSQL 0.8%
Other 0.8%