0463e44e0c
Three coupled structural fixes for the TrailLoosen-pathology identified
by the Phase 4-A3 ultrathink investigation. Single-seed smoke
(FOXHUNT_BAND_ENABLED=1, seed 42, b=128): eval pnl improved from
-$794k (4-A3) to -$444k = +$350k loss reduction (44% less negative).
Win rate 37.1% (was 31.7%).
## Fix 1: Trail-max ceiling (new ISV slot 814)
RL_TRAIL_MAX_INITIAL_R_RATIO_INDEX bootstrap 4.0. In rl_trail_mutate.cu,
clamp unit_trail_distance ≤ initial_trail × 4.0. Without this the
TrailLoosen action (× 1.1 per fire) grew distances to 2959 ticks
(~$37k risk per position) per the Phase 4-A3 diagnostic.
Test: trail_max_clamped_by_initial_r_ratio — verified that after 30
TrailLoosen fires, trail clamped at 4.0× (uncapped would have been
17.4×, a 4.4× reduction).
Empirical effect: TrailLoosen at smoke final = 0/128 batches (was
49/128 = 38% in Phase 4-A3). The arbitrage is mechanically eliminated.
## Fix 2: Mark-to-market reward (new ISV slots 815/816)
RL_MTM_REWARD_ENABLED_INDEX (bootstrap 1.0) + RL_MTM_REWARD_WEIGHT_INDEX
(bootstrap 1.0). New Phase 1.6 in rl_fused_reward_pipeline.cu:
r += w_mtm × (unrealized_now − unrealized_prev)
Per-step reward proportional to total wealth delta (realized +
unrealized). Penalizes holding losers in real-time. Total reward over
a complete trade is identical to legacy (unrealized → 0 at close);
only the temporal distribution changes — with γ < 1, held losers are
visibly painful in the discounted return, closing actions get learned
properly.
Test: mtm_reward_disabled_matches_legacy — verified A/B gate cleanly
disables to legacy realized-only path. unrealized_pnl read from
pos.vwap_entry + current mid via standard accounting.
Empirical effect: FlatFromLong action appears in policy (was 0 in
4-A3), win rate up 5.4pp.
## Fix 3: Entropy gradient formula (correctness)
rl_q_pi_distill_grad.cu:131 had spurious +1.0f:
BEFORE: grad_entropy = -alpha × pi_a × (log_pi_a + 1.0f + s_entropy)
AFTER: grad_entropy = -alpha × pi_a × (log_pi_a + s_entropy)
True ∂(-H)/∂logit_a = π(a)·(log π(a) + H) per textbook softmax-entropy
gradient. Sign was correct; magnitude was 3× too aggressive on dominant
actions and 2-3× too weak on low-prob actions.
Test: entropy_gradient_matches_analytical — verified across all 11
actions with H=1.4931, kernel matches analytical formula within 1e-5.
## Verification
* 3/3 phase_5_invariants tests PASS
* 11/11 band_invariants regression tests PASS
* FOXHUNT_USE_MULTI_HEAD_POLICY=0 ./scripts/determinism-check.sh
--quick: exit 0 (200 rows bit-equal)
* FOXHUNT_BAND_ENABLED=1 ./scripts/determinism-check.sh --quick:
exit 0 (200 rows bit-equal)
* Pre-commit hook: 0 atomicAdd, 0 raw memcpy_htod/dtoh
## Smoke trajectory (FOXHUNT_BAND_ENABLED=1, seed 42, b=128, 2000+500)
step entropy top_3_action_hist
0 0.000 [128, 0, 0] (init)
100 0.785 [Hold=117, Long-=3, Short-=5, TrailLoosen=1]
500 0.489 [Hold=122, Long-=1, FlatLong=2, TrailLoosen=0]
1000 0.359 [Hold=123, FlatLong=5, TrailLoosen=0]
1500 0.268 [Hold=124, FlatLong=3, TrailLoosen=1]
1999 0.215 [Hold=125, FlatLong=3, TrailLoosen=0]
eval (5000 steps frozen policy):
total_pnl_usd: -$444,225 (best of session, was -$794k in 4-A3)
win_rate: 0.371 (best of session, was 0.317)
n_trades: 197
profit_factor: 0.534
max_drawdown_usd: $462,212
sharpe_ann: -7.20
## Falsification gates
* G_mechanism (no NaN, exit 0): PASS
* G_no_regression (pnl ≥ -$5M): PASS by wide margin (-$444k)
* G_trail_bounded: PASS (0 TrailLoosen fires at final, ceiling working)
* G_action_diversity (entropy ≥ 1.0): FAIL (0.22 — new pure-Hold
conservatism failure mode emerges; the policy learns "don't open
trades" as the safest path with all the constraints in place)
## Analysis of the new failure mode
The Phase 5 fixes worked exactly as designed: trail-max eliminated
the runaway risk mechanism, MTM made held losers painful, entropy
formula now matches theory. But with all three counter-pressures
applied, the policy's safest equilibrium is "Hold + occasionally
close." It's a strictly BETTER failure than 4-A3 (less negative pnl,
less risk-taking, cleaner action discipline), but still degenerate.
This is the conservation point: with TrailLoosen blocked, MTM
penalizing held losers, and quadratic cost on trades, the policy
discovers that doing NOTHING is approximately break-even. Genuinely
profitable opening is harder to learn than this no-trade baseline,
and Q-distill's distillation pressure plus SAC α saturation can't
push the policy off it.
Next investigation: WHY can't the policy discover profitable opens?
Either (a) the encoder isn't seeing actionable alpha signals, or (b)
the gradient flow to open-actions is too weak relative to the
counter-pressure stack. This is a separate spec.
## Linked
* Phase 4-A3: 65c328d3f (band foundation)
* Phase 4 cluster verdict: alpha-rl-4hq7c (band works at scale;
same TrailLoosen pathology confirmed at cluster)
* Phase 4-A3 ultrathink diagnosis: identified the TrailLoosen
pathology + entropy formula bug + this fix design
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Foxhunt
Production HFT trading system in Rust.
Architecture
The workspace contains 32 crates organized as follows:
Core Libraries (16)
| Crate | Purpose |
|---|---|
trading_engine |
Order processing, FIX 4.4, IB TWS, SIMD, RDTSC timing |
risk |
VaR, Kelly, circuit breakers, kill switches, compliance |
risk-data |
Risk data types and shared structures |
trading-data |
Trading data types |
ml |
DQN Rainbow, PPO, TFT, Mamba2, ensemble inference |
ml-data |
ML data types and feature definitions |
data |
Market data ingestion and storage |
backtesting |
Replay engine, strategy tester |
adaptive-strategy |
Ensemble execution, microstructure analysis |
common |
Shared types, resilience, error handling |
storage |
S3 and local model storage |
model_loader |
Model serialization and loading |
market-data |
Market data feed handlers |
database |
PostgreSQL access layer (SQLx) |
config |
Configuration management |
tli |
CLI commands and tooling |
Services (8)
| Service | Purpose |
|---|---|
backtesting_service |
gRPC backtesting service |
broker_gateway_service |
FIX routing, broker connectivity |
trading_service |
Core trading operations |
ml_training_service |
Model training orchestration |
data_acquisition_service |
Market data acquisition |
trading_agent_service |
Autonomous trading agents |
api_gateway |
gRPC API gateway with auth |
web-gateway |
Axum REST + WebSocket gateway |
Frontend
web-dashboard/ -- React 19 + TypeScript + Vite + TradingView charts.
Building
# Check compilation (no PostgreSQL required)
SQLX_OFFLINE=true cargo check --workspace
# Run tests for a specific crate
SQLX_OFFLINE=true cargo test -p <crate> --lib
# Clippy
SQLX_OFFLINE=true cargo clippy --workspace
ML Models
Four production model architectures on Candle v0.9.1 with CUDA:
- DQN Rainbow -- Deep Q-Network with prioritized replay, dueling heads, noisy nets
- PPO -- Proximal Policy Optimization with GAE and LSTM policies
- TFT -- Temporal Fusion Transformer for multi-horizon forecasting
- Mamba2 -- State space model for sequence prediction
Each model has a standalone trainer and a UnifiedTrainable adapter for the hyperopt pipeline.
Infrastructure
- Git: Gitea at
git.fxhnt.ai(Tailscale-only), Scaleway DEV1-S - Observability: OpenTelemetry OTLP (env
OTEL_EXPORTER_OTLP_ENDPOINT) - Database: PostgreSQL with SQLx offline mode for CI
License
Proprietary. All rights reserved.