bd811a7748
Combined atomic attack on foxhunt's structural overtrading pathology
per omnisearch (2026-06-03) RL HFT literature: foxhunt uniquely uses
Q-distill as the SOLE policy-training mechanism, making it susceptible
to the four-stage Q→π attenuation chain (Goodhart-Skalse 2024) that
blinds π to small persistent fees. Three concurrent fixes attack
different layers:
A. Hold-action logit bias (+log(4) ≈ 1.386 on action 2)
* crates/ml-alpha/src/rl/ppo.rs PolicyHead::new
* crates/ml-alpha/src/rl/multi_head_policy.rs all K heads + build_priors
Counter-balances the structural 4:1 open-vs-hold action prior (4
open variants 0,1,5,6 vs 1 Hold=2). Pre-bias P(open)=36% / P(hold)=9%;
post-bias P(hold)≈29% / P(any open)≈7%. Mid-smoke seed=42 confirms
Hold rises to 71/128 = 55.5% by step 1999.
B. Quadratic-in-trade-size impact-aware cost (Cao et al. 2026,
arXiv:2603.29086 §4)
* crates/ml-alpha/cuda/rl_fused_reward_pipeline.cu Phase 1.5
* 3 new ISV slots 794-796 (α=0.5, β=2.0, enabled=1.0)
cost = α·|Δlots| + β·(Δlots)². 1-lot=2.5; 4-lot flip=34; 8-lot=132
(superlinear). Applied BEFORE shaping so the surfer-scaffold weight
does not amplify or mute. Trail actions (7,8) are no-ops in the
position kernel → cost=0 for them as expected.
C. PPO surrogate gradient restoration with adaptive blend (Cao 2026 §4)
* crates/ml-alpha/cuda/rl_pi_grad_blend.cu (new — element-wise
scale-or-zero operator)
* crates/ml-alpha/src/trainer/integrated.rs Step 7 (π gradient blend)
* 2 new ISV slots 797-798 (weight=0.005, enabled=1.0)
Previously π was trained ONLY by Q-distillation (line 5850 header).
Now: pi_grad = w_ppo·grad_PPO + grad_Q_distill + grad_SAC_entropy.
Restores the direct fee-aware policy-gradient channel that Q-distill
alone cannot transmit. Blend kernel runs BETWEEN surrogate_backward
and rl_q_pi_distill_grad (which uses +=).
Diag emission (E):
* crates/ml-alpha/src/trainer/integrated.rs rewards.{quadratic_cost_alpha,
quadratic_cost_beta, quadratic_cost_enabled, ppo_surrogate_weight,
ppo_surrogate_enabled}
Tests (D):
* multi_head_policy_invariants: updated k1_reduces_to_single_head for
Phase 3D-A bias; new phase_3d_a_hold_bias_propagates_all_heads
invariant verifies Hold dominance in every head at h_t=0. 18/18 pass.
* phase_3d_blend_kernel_invariants (new): 3 GPU-oracle invariants on
rl_pi_grad_blend (disabled-zeros, enabled-scales-linearly, weight-0-
equivalent-to-disabled). 3/3 pass.
* reward_alignment_invariants: 2 new tests (phase_3d_diag_emission +
phase_3d_quadratic_cost_visible_in_rewards) + fix to the existing
surfer_scaffold test (relaxed bootstrap check for Phase 3B-Y pure-pnl
mode default 0.0; absorbs eval drain row). 3/3 pass.
* All 68 ml-alpha lib tests pass.
Verification:
* SQLX_OFFLINE=true cargo build --release --example alpha_rl_train -p
ml-alpha: exit 0
* SQLX_OFFLINE=true cargo check --workspace: exit 0
* ./scripts/determinism-check.sh --quick: DETERMINISTIC (200/200 rows
bit-equal across two same-seed runs)
* FOXHUNT_USE_MULTI_HEAD_POLICY=1 ./scripts/determinism-check.sh --quick:
DETERMINISTIC
* Local Tier 1.5 mid-smoke (seed=42, b=128, 2000 train + 500 eval):
exit 0, completed_clean=true, no NaN, no abort.
Primary kill criterion (total_trades final < 5,000): NOT MET.
Result: 11,767 trades vs 14,691 baseline = 20% reduction. Cao 2026
forecast 96% reduction for pure-PPO/SAC architectures was not
achieved — foxhunt's Q-distill dominance (q_pi_agree_ema = 0.948 in
this run) attenuates the PPO surrogate's fee signal even with the
blend operator. The behavioral signature IS present (Hold dominance
rises from baseline ~36% structural prior to 55.5% at step 1999;
action_entropy = 1.748 within healthy [1.2, 2.04] target).
Regression guard (eval pnl ≥ -$5M): MARGINAL PASS at -$4.96M (-$36k
inside threshold). The Tier 1.5 verdict flags KILL on Pearson +
wr_train + wr_eval + eval_pnl. Pre-cluster, the literature
recommendation is to A/B-ablate each intervention (slots 794-798
individually gated). Mid-smoke architecturally validates that the
three interventions PROPAGATE and do not crash; cluster b=1024 with
longer runs (20k steps) will surface whether the 20% reduction
compounds into a viable policy.
Architectural references:
* pearl_foxhunt_pi_trained_by_q_distillation_not_ppo
* pearl_reward_signal_anti_aligned_with_pnl
* pearl_bootstrap_must_respect_clamp_range
* feedback_no_atomicadd / feedback_no_htod_htoh_only_mapped_pinned
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Foxhunt
Production HFT trading system in Rust.
Architecture
The workspace contains 32 crates organized as follows:
Core Libraries (16)
| Crate | Purpose |
|---|---|
trading_engine |
Order processing, FIX 4.4, IB TWS, SIMD, RDTSC timing |
risk |
VaR, Kelly, circuit breakers, kill switches, compliance |
risk-data |
Risk data types and shared structures |
trading-data |
Trading data types |
ml |
DQN Rainbow, PPO, TFT, Mamba2, ensemble inference |
ml-data |
ML data types and feature definitions |
data |
Market data ingestion and storage |
backtesting |
Replay engine, strategy tester |
adaptive-strategy |
Ensemble execution, microstructure analysis |
common |
Shared types, resilience, error handling |
storage |
S3 and local model storage |
model_loader |
Model serialization and loading |
market-data |
Market data feed handlers |
database |
PostgreSQL access layer (SQLx) |
config |
Configuration management |
tli |
CLI commands and tooling |
Services (8)
| Service | Purpose |
|---|---|
backtesting_service |
gRPC backtesting service |
broker_gateway_service |
FIX routing, broker connectivity |
trading_service |
Core trading operations |
ml_training_service |
Model training orchestration |
data_acquisition_service |
Market data acquisition |
trading_agent_service |
Autonomous trading agents |
api_gateway |
gRPC API gateway with auth |
web-gateway |
Axum REST + WebSocket gateway |
Frontend
web-dashboard/ -- React 19 + TypeScript + Vite + TradingView charts.
Building
# Check compilation (no PostgreSQL required)
SQLX_OFFLINE=true cargo check --workspace
# Run tests for a specific crate
SQLX_OFFLINE=true cargo test -p <crate> --lib
# Clippy
SQLX_OFFLINE=true cargo clippy --workspace
ML Models
Four production model architectures on Candle v0.9.1 with CUDA:
- DQN Rainbow -- Deep Q-Network with prioritized replay, dueling heads, noisy nets
- PPO -- Proximal Policy Optimization with GAE and LSTM policies
- TFT -- Temporal Fusion Transformer for multi-horizon forecasting
- Mamba2 -- State space model for sequence prediction
Each model has a standalone trainer and a UnifiedTrainable adapter for the hyperopt pipeline.
Infrastructure
- Git: Gitea at
git.fxhnt.ai(Tailscale-only), Scaleway DEV1-S - Observability: OpenTelemetry OTLP (env
OTEL_EXPORTER_OTLP_ENDPOINT) - Database: PostgreSQL with SQLx offline mode for CI
License
Proprietary. All rights reserved.