31e0f219a5
Combines uncommitted iv-main + iv-ext aux paths + Fix A (Polyak EMA
extension covering VSN [95..119)) + Fix B (missed dispatch arm), and
strips rc2/rc3 band-aids that turned out to be unnecessary once the
upstream cause was fixed.
Root cause (focused HtoD/DtoD/pinned-memory audit):
target_ema_update only ran the EMA kernel over non_isv_params at
byte range covering [0..FIRST_ISV_TENSOR=77), skipping the 24 VSN
tensors at [95..119) added in 1B-ii. Online VSN trained from the
1B-iv backward chain; target VSN stayed at alloc_zeros for the
entire run. Bellman target used softmax(zeros) = uniform 1/6 mask
while online's mask drifted toward [market=0.13, portfolio=0.25].
The systematic Q-estimate divergence collapsed fold-2 magnitude
branch and pinned F0 best Sharpe at 21.14 across every prior
magnitude-scaling / state-isolation attempt.
Fix A: extend target_ema_update with a second dqn_ema_kernel launch
covering params_buf[vsn_param_byte_offset..vsn_param_total) so
target VSN tracks online VSN through the same Polyak EMA the rest
of the network uses. Same kernel signature, on-device, no DtoH.
Fix B: add the missed isv_vsn_aux_grad_scale dispatch arm in
training_loop.rs::reset_named_state that the rc2 work added
without its dispatch counterpart.
Cleanup (Phase B): strip rc2 ISV slot 113 + dqn_scale_f32_isv_scaled
+ aux_bottleneck_vsn_backward_dispatch indirection AND rc3 split-
Adam vsn_m_buf/vsn_v_buf — band-aids for the symptom Fix A actually
addresses. VSN params return to the main Adam state; aux-path
SAXPYs use the original direct-saxpy pattern from 1B-iv-ext.
Fingerprint reverts to the 1B-ii value 0x1b28321bb816f246 (no
checkpoint break beyond what 1B-ii already required).
Smoke (multi_fold_convergence --release-test, 3 folds × 5 epochs on
RTX 3050 Ti, 671.68s, all 3 dqn_fold{N}_best.safetensors written):
F0=93.4114 F1=73.0430 F2=73.0749
geom-mean = 79.31 vs 71.24 floor = +11.3%
F0 +36% over 1B-iii baseline; F2 +18%; F1 -14% (60-epoch L40S
run will validate equilibration; short-horizon asymmetry expected).
Constraints honoured: GPU-only (target EMA on-device), no atomicAdd,
no stubs, no // ok: band-aids, no tuned constants (Polyak EMA tau
shared with existing main-range launch), partial-refactor invariant
(dqn_ema_kernel signature unchanged — both launches use identical
args, different byte offsets/counts).
Lesson: 4 prior remedies (rc, rc2, rc3, rc4) and 1 diagnostic run
(E1) all chased downstream symptoms (gradient scale, Adam variance,
kernel sync). The upstream cause was a 1-line gap in Polyak target
sync that didn't include post-Plan-4 tensors. Same shape as the
2c.3a follow-up bottleneck Linear bug (commit f3e3ac347, 4 stale
runtime indices missed in GRN reshuffle): simple wire-up gaps in
shared infrastructure cause inscrutable downstream behavior. See
feedback_no_partial_refactor.md.
cargo check clean at 11 warnings (workspace baseline preserved).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Foxhunt
Production HFT trading system in Rust.
Architecture
The workspace contains 32 crates organized as follows:
Core Libraries (16)
| Crate | Purpose |
|---|---|
trading_engine |
Order processing, FIX 4.4, IB TWS, SIMD, RDTSC timing |
risk |
VaR, Kelly, circuit breakers, kill switches, compliance |
risk-data |
Risk data types and shared structures |
trading-data |
Trading data types |
ml |
DQN Rainbow, PPO, TFT, Mamba2, ensemble inference |
ml-data |
ML data types and feature definitions |
data |
Market data ingestion and storage |
backtesting |
Replay engine, strategy tester |
adaptive-strategy |
Ensemble execution, microstructure analysis |
common |
Shared types, resilience, error handling |
storage |
S3 and local model storage |
model_loader |
Model serialization and loading |
market-data |
Market data feed handlers |
database |
PostgreSQL access layer (SQLx) |
config |
Configuration management |
tli |
CLI commands and tooling |
Services (8)
| Service | Purpose |
|---|---|
backtesting_service |
gRPC backtesting service |
broker_gateway_service |
FIX routing, broker connectivity |
trading_service |
Core trading operations |
ml_training_service |
Model training orchestration |
data_acquisition_service |
Market data acquisition |
trading_agent_service |
Autonomous trading agents |
api_gateway |
gRPC API gateway with auth |
web-gateway |
Axum REST + WebSocket gateway |
Frontend
web-dashboard/ -- React 19 + TypeScript + Vite + TradingView charts.
Building
# Check compilation (no PostgreSQL required)
SQLX_OFFLINE=true cargo check --workspace
# Run tests for a specific crate
SQLX_OFFLINE=true cargo test -p <crate> --lib
# Clippy
SQLX_OFFLINE=true cargo clippy --workspace
ML Models
Four production model architectures on Candle v0.9.1 with CUDA:
- DQN Rainbow -- Deep Q-Network with prioritized replay, dueling heads, noisy nets
- PPO -- Proximal Policy Optimization with GAE and LSTM policies
- TFT -- Temporal Fusion Transformer for multi-horizon forecasting
- Mamba2 -- State space model for sequence prediction
Each model has a standalone trainer and a UnifiedTrainable adapter for the hyperopt pipeline.
Infrastructure
- Git: Gitea at
git.fxhnt.ai(Tailscale-only), Scaleway DEV1-S - Observability: OpenTelemetry OTLP (env
OTEL_EXPORTER_OTLP_ENDPOINT) - Database: PostgreSQL with SQLx offline mode for CI
License
Proprietary. All rights reserved.