fa92cb8dbb
compute-sanitizer caught 4 OOB writes in compute_expected_q at threads
60..63 of block 0 (production: batch=64, total_actions=13). Stride
52 bytes/thread (= 13 floats), increasing 45 → 97 → 149 → 201 bytes
past a 4-byte-aligned neighbour allocation. The kernel writes
`q_variance[i*total_actions + a]` with stride total_actions=13 (4+3+3+3
factored layout) into a buffer that was still sized at b*12 from the
legacy 3+3+3+3 exposure layout. Same class of bug as `d625ca28e` (q_readback
12→total_actions): the direction branch grew from 3 → 4 actions but
several legacy 12-stride consumers were not migrated.
Per feedback_no_partial_refactor: q_var_buf_trainer's contract is a
shared one — every consumer migrates in lockstep. Five touch points:
• alloc q_var_buf_trainer: b*12 → b*total_actions
• compute_expected_q (writer): unchanged, already stride total_actions
• q_denoise_step kernel: add `var_stride` param, read var_q at stride
total_actions; FC layer dim D=12 unchanged (denoiser MLP weights stay)
• q_denoise_backward kernel: same — add `var_stride`, two read sites
• denoise_build_input kernel: add `var_stride` between D and schedule
• launch_loss_reduce: b_qvar = b*12 → b*total_actions for the third
c51_loss_reduce launch (ISV slot 3/4 mean reduction)
The denoiser's internal D=12 is preserved — its FC layers operate on the
12-slot exposure layout and only consume the first 12 variance entries
per sample. The 13th variance slot (last urgency action) is computed by
compute_expected_q but currently unused by the denoiser; epistemic_gate
already reads with stride total_actions and benefits from full per-action
variance now that the buffer is correctly sized.
This was a latent bug masked by allocator fortune — earlier mag_concat
corruption (`8bc6f1ccd`) was overwriting buffers first, so this OOB
landed in already-corrupted memory and produced no compute-sanitizer
report. With mag_concat fixed, the q_var stride mismatch became the
dominant remaining corruption source, surfacing as
`bias_grad_reduce_f32_p1: DriverError(CUDA_ERROR_LAUNCH_FAILED)` on the
next launch in production runs.
Sanitizer (RTX 3050 Ti, magnitude_distribution smoke):
compute_expected_q OOB count: 4 → 0
Note: the run still trips 16 OOB reads in `magma_sgemmEx_kernel` (cuBLAS
GEMM) under a different code path; those are pre-existing — they appear
only because q_var growing from b*12 to b*total_actions reshuffled
allocation addresses, exposing a separate latent cuBLAS leading-dim
mismatch that was previously aliasing into q_var's old footprint. That
finding is filed in the wire-up audit and is not in scope for this commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Foxhunt
Production HFT trading system in Rust.
Architecture
The workspace contains 32 crates organized as follows:
Core Libraries (16)
| Crate | Purpose |
|---|---|
trading_engine |
Order processing, FIX 4.4, IB TWS, SIMD, RDTSC timing |
risk |
VaR, Kelly, circuit breakers, kill switches, compliance |
risk-data |
Risk data types and shared structures |
trading-data |
Trading data types |
ml |
DQN Rainbow, PPO, TFT, Mamba2, ensemble inference |
ml-data |
ML data types and feature definitions |
data |
Market data ingestion and storage |
backtesting |
Replay engine, strategy tester |
adaptive-strategy |
Ensemble execution, microstructure analysis |
common |
Shared types, resilience, error handling |
storage |
S3 and local model storage |
model_loader |
Model serialization and loading |
market-data |
Market data feed handlers |
database |
PostgreSQL access layer (SQLx) |
config |
Configuration management |
tli |
CLI commands and tooling |
Services (8)
| Service | Purpose |
|---|---|
backtesting_service |
gRPC backtesting service |
broker_gateway_service |
FIX routing, broker connectivity |
trading_service |
Core trading operations |
ml_training_service |
Model training orchestration |
data_acquisition_service |
Market data acquisition |
trading_agent_service |
Autonomous trading agents |
api_gateway |
gRPC API gateway with auth |
web-gateway |
Axum REST + WebSocket gateway |
Frontend
web-dashboard/ -- React 19 + TypeScript + Vite + TradingView charts.
Building
# Check compilation (no PostgreSQL required)
SQLX_OFFLINE=true cargo check --workspace
# Run tests for a specific crate
SQLX_OFFLINE=true cargo test -p <crate> --lib
# Clippy
SQLX_OFFLINE=true cargo clippy --workspace
ML Models
Four production model architectures on Candle v0.9.1 with CUDA:
- DQN Rainbow -- Deep Q-Network with prioritized replay, dueling heads, noisy nets
- PPO -- Proximal Policy Optimization with GAE and LSTM policies
- TFT -- Temporal Fusion Transformer for multi-horizon forecasting
- Mamba2 -- State space model for sequence prediction
Each model has a standalone trainer and a UnifiedTrainable adapter for the hyperopt pipeline.
Infrastructure
- Git: Gitea at
git.fxhnt.ai(Tailscale-only), Scaleway DEV1-S - Observability: OpenTelemetry OTLP (env
OTEL_EXPORTER_OTLP_ENDPOINT) - Database: PostgreSQL with SQLx offline mode for CI
License
Proprietary. All rights reserved.