d3a35cc6e9
First of four producer kernels in the Earned Gradient Flow pearl chain.
Per-step computes the K=4↔K=2 mapped argmax mismatch between the Q-head's
4-way direction action (DIR_SHORT/DIR_HOLD/DIR_LONG/DIR_FLAT) and the
aux head's 2-way next-bar prediction (down/up), updates fast + slow
EMAs of the disagreement rate, and updates a Welford-style variance
EMA on the same signal in a single launch.
Mapping (per state_layout.cuh):
DIR_SHORT (=0) → aux down (=0) contributes
DIR_HOLD (=1) → masked no contribution
DIR_LONG (=2) → aux up (=1) contributes
DIR_FLAT (=3) → masked no contribution
Hold and Flat are masked because they represent "no NEW directional
commitment" (Hold = keep prior position; Flat = close all positions).
Penalising the aux head for not matching them would conflate
position-management actions with directional predictions.
Block tree-reduce on numerator + count separately, then divide and
update EMAs in a single thread (tid == 0). No atomicAdd per
feedback_no_atomicadd.md. Pure GPU compute per
feedback_no_cpu_compute_strict.md.
Pearl-A first-observation: when ISV[Q_DISAGREEMENT_SHORT_EMA=383] AND
ISV[Q_DISAGREEMENT_LONG_EMA=384] both equal sentinel 0.5f AND batch
has at least one valid contribution (total_cnt > 0), both EMAs are
replaced directly with the first observation per
pearl_first_observation_bootstrap.md. The 0.5f exact-match is safe
because 0.5 is exactly representable in IEEE 754 single precision
(mantissa = 1.0, exponent = -1). The Welford variance EMA at slot
389 drives the adaptive k_q sigmoid steepness consumed by the
alpha_grad_compute_kernel in B.4.
Slot indices are hardcoded inside the kernel via #define — must match
crates/ml/src/cuda_pipeline/sp14_isv_slots.rs (currently 383, 384, 389).
The kernel header documents the coupling explicitly.
Launch contract:
grid_dim = (1, 1, 1)
block_dim = (256, 1, 1)
shared_mem_bytes = 2 * 256 * sizeof(float) = 2048
A shared_mem_bytes = 0 launch reads garbage and corrupts the EMA — the
kernel header documents the launcher requirement; oracle tests pass
2048 explicitly.
Files:
- crates/ml/src/cuda_pipeline/q_disagreement_update_kernel.cu (NEW)
- crates/ml/build.rs — register cubin in kernels_with_common
- crates/ml/tests/sp14_oracle_tests.rs — append #[cfg(feature="cuda")]
mod gpu with cubin handle + 2 GPU oracle tests
- docs/dqn-wire-up-audit.md — SP14 B.3 section (Invariant 7)
Tests (both pass on RTX 3050 Ti):
- q_disagreement_k4_k2_mapping: 8-row batch with 2 agreements,
2 disagreements, 4 masked Hold/Flat → first-obs Pearl-A replaces
both EMAs with batch_mean = 0.5 (asserted within 1e-4)
- q_disagreement_all_hold_no_contribution: all-Hold edge case;
total_cnt = 0 so batch_mean = 0/1 = 0; sentinel-bootstrap guard
(total_cnt > 0) keeps EMA from collapsing to 0; current kernel
blends to 0.35, test bound [0.0, 0.5] tolerant of either current
blend or future skip-update refinement, asserts is_finite()
Wire-up status: producer kernel exists and is exercised only by the
oracle tests. The Rust launcher and graph-capture integration land in
B.7+ alongside the consumer (alpha_grad_compute in B.4 reads slots
383/384/389). This is one producer kernel of four (B.3 q_disagreement,
B.4 alpha_grad_compute, B.5 gradient_hack_detect, B.6 dir_concat_qaux);
known-orphan for the duration of the producer chain per
feedback_wire_everything_up.md (the same-commit wire-up rule is
relaxed for atomic chained-producer-consumer landings, with each
orphan documented in the audit doc; this orphan is acknowledged in
docs/dqn-wire-up-audit.md SP14 B.3 section).
Build + test verification:
- SQLX_OFFLINE=true CUDA_COMPUTE_CAP=86 cargo check -p ml --features cuda
→ clean (only the 18 pre-existing warnings)
- SQLX_OFFLINE=true cargo test -p ml --test sp14_oracle_tests
set_aux_weight → host-only A.2 still passes (no shared dependency)
- SQLX_OFFLINE=true CUDA_COMPUTE_CAP=86 cargo test -p ml
--test sp14_oracle_tests --features cuda q_disagreement
-- --ignored --nocapture → 2 PASS
Slot indices reflect the SP13-closeout +2 shift documented in
sp14_isv_slots.rs (the original plan's 381/382/387 became 383/384/389
because HOLD_RATE_TARGET=381 and HOLD_RATE_OBSERVED_EMA=382 landed
between when the plan was written and B.1 was implemented).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Foxhunt
Production HFT trading system in Rust.
Architecture
The workspace contains 32 crates organized as follows:
Core Libraries (16)
| Crate | Purpose |
|---|---|
trading_engine |
Order processing, FIX 4.4, IB TWS, SIMD, RDTSC timing |
risk |
VaR, Kelly, circuit breakers, kill switches, compliance |
risk-data |
Risk data types and shared structures |
trading-data |
Trading data types |
ml |
DQN Rainbow, PPO, TFT, Mamba2, ensemble inference |
ml-data |
ML data types and feature definitions |
data |
Market data ingestion and storage |
backtesting |
Replay engine, strategy tester |
adaptive-strategy |
Ensemble execution, microstructure analysis |
common |
Shared types, resilience, error handling |
storage |
S3 and local model storage |
model_loader |
Model serialization and loading |
market-data |
Market data feed handlers |
database |
PostgreSQL access layer (SQLx) |
config |
Configuration management |
tli |
CLI commands and tooling |
Services (8)
| Service | Purpose |
|---|---|
backtesting_service |
gRPC backtesting service |
broker_gateway_service |
FIX routing, broker connectivity |
trading_service |
Core trading operations |
ml_training_service |
Model training orchestration |
data_acquisition_service |
Market data acquisition |
trading_agent_service |
Autonomous trading agents |
api_gateway |
gRPC API gateway with auth |
web-gateway |
Axum REST + WebSocket gateway |
Frontend
web-dashboard/ -- React 19 + TypeScript + Vite + TradingView charts.
Building
# Check compilation (no PostgreSQL required)
SQLX_OFFLINE=true cargo check --workspace
# Run tests for a specific crate
SQLX_OFFLINE=true cargo test -p <crate> --lib
# Clippy
SQLX_OFFLINE=true cargo clippy --workspace
ML Models
Four production model architectures on Candle v0.9.1 with CUDA:
- DQN Rainbow -- Deep Q-Network with prioritized replay, dueling heads, noisy nets
- PPO -- Proximal Policy Optimization with GAE and LSTM policies
- TFT -- Temporal Fusion Transformer for multi-horizon forecasting
- Mamba2 -- State space model for sequence prediction
Each model has a standalone trainer and a UnifiedTrainable adapter for the hyperopt pipeline.
Infrastructure
- Git: Gitea at
git.fxhnt.ai(Tailscale-only), Scaleway DEV1-S - Observability: OpenTelemetry OTLP (env
OTEL_EXPORTER_OTLP_ENDPOINT) - Database: PostgreSQL with SQLx offline mode for CI
License
Proprietary. All rights reserved.