ebc1b15023
Lib test suite was at 14 failures from accumulated layout/contract drift.
Fixed each by tracing root cause; lib suite now 1016 passing / 0 failing.
Failures fixed (test → root cause → fix):
1. sp14_isv_slots::sp20_isv_slots_reserved_510_to_520 — ISV_TOTAL_DIM
pin drifted from SP20-era 520 to current 538 via SP21/SP22 H6 bus
growth. Slot positions 510-519 still pinned. Fix: relax total-dim
check to >= 520; keep slot-position asserts tight.
2-3. gradient_budget::test_spectral_norm_all_heads_no_panic +
test_spectral_norm_constrains_operator_norm — test config used
cfg.state_dim=16 but trainer uses STATE_DIM=128 when bottleneck
off; also DuelingWeightBacking slices [2]/[3] used pre-GRN w_s2
shape, but post-GRN it's w_b_h_s1 [2*SH1, SH1] = 2048. Fix: add
s1_input_dim_for_test helper; correct slice sizes to GRN layout.
4-9. dqn::trainer::tests::test_feature_vector_to_state +
test_single_sample_batch + test_batched_action_selection +
test_batched_vs_sequential_action_selection_consistency +
test_batch_size_mismatch_larger/smaller_than_configured —
feature_vector_to_state wrapper falsely advertised "no OFI" by
signature but body required strict OFI. Contract violation —
broke 6 unit tests + hyperopt's public convert_to_state APIs.
Fix: wrapper now genuinely produces 45-dim no-OFI state; OFI-
strict callers use _with_ofi with Some(idx).
10. state_kl_monitor::observe_tracks_fire_rate_on_change — last_amp
initialized to 1.0 coincidentally equaled first observation value,
silently suppressing first fire. Test expected first observation
always fires (cold-start semantic). Fix: Option<f32> sentinel —
None means "no prior baseline" → first observe ALWAYS fires.
11. cuda_pipeline::tests::test_eval_action_select_thompson_picks_
proportionally — test launched experience_action_select kernel
with 1 missing arg (v_logits_dir from SP17 Commit C). Kernel
read garbage pointer → SIGSEGV. Also threshold 0.70 calibrated
for pre-SP17 raw-A distribution; post-SP17 sampler uses mean-zero
softmax(V + A_centered), Long still dominates ~4× but P(Long)
drops to ~0.66. Fix: add zero-filled v_logits_buf at correct
arg position; recalibrate threshold 0.70 → 0.60.
12. cuda_pipeline::tests::test_ppo_gpu_data_upload — flaked in parallel
test run with CUDA_ERROR_INVALID_CONTEXT from MappedF32Buffer's
cudaHostAlloc. cuda_stream() test helper used OnceLock to share
a CudaStream but didn't bind_to_thread on subsequent calls. CUDA
contexts are per-thread state. Fix: helper now calls bind_to_
thread() on every invocation (idempotent — mirrors trainer/
constructor.rs:111's pattern).
13. ppo::tests::test_reward_computation — test created hold action
with ExposureLevel::Flat but is_hold() matches only Hold (Flat
= close position = transaction with cost). So hold and buy got
same transaction-cost reduction; reward_hold > reward_buy assert
failed. Fix: use ExposureLevel::Hold for no-cost hold semantic.
14. training_profile::tests::test_production_profile_applies_all_
sections — pinned n_steps==5 and tau==0.005 (pre-TD(0)); production
flipped to n_steps=1, tau=0.01 per dqn-production.toml ("dense
micro-rewards cancel over n>1 bars; faster target tracking for
TD(0)"). Fix: update pins to current production values.
All 14 fixes target root causes — no #[ignore] masking. Verified:
- Lib-only run: cargo test -p ml --lib → 1016/0 (passed/failed)
- Stash-test pre-changes: 1001/15 (confirms 14 fixed atomically)
Remaining flake under `cargo test -p ml --tests`:
- ensemble::adapters::dqn::tests::test_dqn_checkpoint_round_trip —
CUDA-context-race-adjacent flake under parallel `--tests` mode;
passes in isolation and in `--lib` mode (single binary). Predates
this commit; deferred as separate triage.
Audit: docs/dqn-wire-up-audit.md "Fix sweep" section.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Foxhunt
Production HFT trading system in Rust.
Architecture
The workspace contains 32 crates organized as follows:
Core Libraries (16)
| Crate | Purpose |
|---|---|
trading_engine |
Order processing, FIX 4.4, IB TWS, SIMD, RDTSC timing |
risk |
VaR, Kelly, circuit breakers, kill switches, compliance |
risk-data |
Risk data types and shared structures |
trading-data |
Trading data types |
ml |
DQN Rainbow, PPO, TFT, Mamba2, ensemble inference |
ml-data |
ML data types and feature definitions |
data |
Market data ingestion and storage |
backtesting |
Replay engine, strategy tester |
adaptive-strategy |
Ensemble execution, microstructure analysis |
common |
Shared types, resilience, error handling |
storage |
S3 and local model storage |
model_loader |
Model serialization and loading |
market-data |
Market data feed handlers |
database |
PostgreSQL access layer (SQLx) |
config |
Configuration management |
tli |
CLI commands and tooling |
Services (8)
| Service | Purpose |
|---|---|
backtesting_service |
gRPC backtesting service |
broker_gateway_service |
FIX routing, broker connectivity |
trading_service |
Core trading operations |
ml_training_service |
Model training orchestration |
data_acquisition_service |
Market data acquisition |
trading_agent_service |
Autonomous trading agents |
api_gateway |
gRPC API gateway with auth |
web-gateway |
Axum REST + WebSocket gateway |
Frontend
web-dashboard/ -- React 19 + TypeScript + Vite + TradingView charts.
Building
# Check compilation (no PostgreSQL required)
SQLX_OFFLINE=true cargo check --workspace
# Run tests for a specific crate
SQLX_OFFLINE=true cargo test -p <crate> --lib
# Clippy
SQLX_OFFLINE=true cargo clippy --workspace
ML Models
Four production model architectures on Candle v0.9.1 with CUDA:
- DQN Rainbow -- Deep Q-Network with prioritized replay, dueling heads, noisy nets
- PPO -- Proximal Policy Optimization with GAE and LSTM policies
- TFT -- Temporal Fusion Transformer for multi-horizon forecasting
- Mamba2 -- State space model for sequence prediction
Each model has a standalone trainer and a UnifiedTrainable adapter for the hyperopt pipeline.
Infrastructure
- Git: Gitea at
git.fxhnt.ai(Tailscale-only), Scaleway DEV1-S - Observability: OpenTelemetry OTLP (env
OTEL_EXPORTER_OTLP_ENDPOINT) - Database: PostgreSQL with SQLx offline mode for CI
License
Proprietary. All rights reserved.