e3d0829680
Without this trigger, cargo treated CUDA_COMPUTE_CAP-driven cubin
compilation as cached output of a non-tracked env var. The cargo-target
PVC is shared across H100 (sm_90) and L40S (sm_89) training jobs, so
swapping --gpu-pool between archs left stale cubins from the previous
build cached for any future sm_X variation.
Symptom (T10 train-multi-seed-rn559 today):
Failed to create DQN: rmsnorm cubin load: DriverError(
CUDA_ERROR_NO_BINARY_FOR_GPU,
"no kernel image is available for execution on the device")
The training pod scheduled on L40S (sm_89), the binary cache served
sm_90 cubins from a prior H100 build, and the rmsnorm cubin had no
sm_89 entrypoint.
Fix is one line: tell cargo to invalidate when CUDA_COMPUTE_CAP changes.
First post-fix build will rebuild all cubins (cargo conservatively
reruns when a new rerun-if-env-changed trigger is added without prior
state). Subsequent builds rebuild only on actual env changes.
Generalises the same lesson as the recent SP7 host-branch-in-captured-graph
fix: env-var-conditional code that doesn't declare its dependencies
freezes at first observation regardless of runtime input.
Foxhunt
Production HFT trading system in Rust.
Architecture
The workspace contains 32 crates organized as follows:
Core Libraries (16)
| Crate | Purpose |
|---|---|
trading_engine |
Order processing, FIX 4.4, IB TWS, SIMD, RDTSC timing |
risk |
VaR, Kelly, circuit breakers, kill switches, compliance |
risk-data |
Risk data types and shared structures |
trading-data |
Trading data types |
ml |
DQN Rainbow, PPO, TFT, Mamba2, ensemble inference |
ml-data |
ML data types and feature definitions |
data |
Market data ingestion and storage |
backtesting |
Replay engine, strategy tester |
adaptive-strategy |
Ensemble execution, microstructure analysis |
common |
Shared types, resilience, error handling |
storage |
S3 and local model storage |
model_loader |
Model serialization and loading |
market-data |
Market data feed handlers |
database |
PostgreSQL access layer (SQLx) |
config |
Configuration management |
tli |
CLI commands and tooling |
Services (8)
| Service | Purpose |
|---|---|
backtesting_service |
gRPC backtesting service |
broker_gateway_service |
FIX routing, broker connectivity |
trading_service |
Core trading operations |
ml_training_service |
Model training orchestration |
data_acquisition_service |
Market data acquisition |
trading_agent_service |
Autonomous trading agents |
api_gateway |
gRPC API gateway with auth |
web-gateway |
Axum REST + WebSocket gateway |
Frontend
web-dashboard/ -- React 19 + TypeScript + Vite + TradingView charts.
Building
# Check compilation (no PostgreSQL required)
SQLX_OFFLINE=true cargo check --workspace
# Run tests for a specific crate
SQLX_OFFLINE=true cargo test -p <crate> --lib
# Clippy
SQLX_OFFLINE=true cargo clippy --workspace
ML Models
Four production model architectures on Candle v0.9.1 with CUDA:
- DQN Rainbow -- Deep Q-Network with prioritized replay, dueling heads, noisy nets
- PPO -- Proximal Policy Optimization with GAE and LSTM policies
- TFT -- Temporal Fusion Transformer for multi-horizon forecasting
- Mamba2 -- State space model for sequence prediction
Each model has a standalone trainer and a UnifiedTrainable adapter for the hyperopt pipeline.
Infrastructure
- Git: Gitea at
git.fxhnt.ai(Tailscale-only), Scaleway DEV1-S - Observability: OpenTelemetry OTLP (env
OTEL_EXPORTER_OTLP_ENDPOINT) - Database: PostgreSQL with SQLx offline mode for CI
License
Proprietary. All rights reserved.