e074c91fb2
Cluster smoke `alpha-rl-nqd68` showed the signal-driven LR controller working mechanically but destabilising the π head: lr_pi swung MIN→MAX (1000×) over ~10k steps, then got stuck at MAX after the catastrophic Adam updates wrecked the policy weights. Aggregate l_pi max = 2.4e17, l_v max = 2,083,330 (no NaN abort, but useless for learning). Q head was fine (lr_q correctly stuck at MIN throughout, l_q dropped 34% vs fixed-LR). ## Two fixes ### 1. Per-step rate-of-change cap The original target formula `target = lr_prev × (TARGET/observed)` allows arbitrary swing magnitude. When `observed` is tiny (e.g. quiescent π grad-norm between trade closes), target = lr_prev × 1000, which the Wiener α=0.4 blend drags toward LR_MAX in a few steps. Once at MAX, the next real reward signal applies catastrophic Adam updates → policy explodes → grad-norm spikes 10⁵× → controller sees this and tries to shrink, but the damage is done. Adds `target_lr ∈ [lr_prev × 0.5, lr_prev × 2.0]` constraint post-formula, pre-clamp. The controller can now at most halve or double LR per step, taking ~10 steps to traverse the full [LR_MIN, LR_MAX] range. Downstream gradient signal has time to react before LR overshoots. Same pattern as `rl_rollout_steps_controller`'s `scale ∈ [0.5, 2.0]` cap (which was added for the same class of multiplicative-controller instability). ### 2. Per-head TARGET_GRAD_NORM The single `TARGET_GRAD_NORM = 1.0` anchor was wrong for π and V: those heads have far fewer parameters than Q (1,152 and 128 vs 24,192). A "well-tuned" grad-norm magnitude scales with √n_params (so per-parameter grad magnitude stays Adam-friendly ≈ 1e-2). Q head w_d: 9 × 21 × 128 = 24,192 params → √ ≈ 156 → target 1.5 π head w_d: 9 × 128 = 1,152 params → √ ≈ 34 → target 0.3 V head w_d: 128 = 128 params → √ ≈ 11 → target 0.1 Without this scaling, the controller was pushing π LR up because its grad-norm (typically 0.1-0.3) was always "below the 1.0 target" — interpreted as "model coasting, grow LR" when really the smaller grad-norm just reflected the smaller parameter count. `update_lr_with_signal` now takes `head_target_grad_norm` as a parameter. BCE and AUX heads (owned by perception, signal_slot=-1) get target=1.0 but it's unused because the early-return at `signal_slot < 0` short-circuits past the target derivation. ## Verified gates (local sm_86) G1 isv_bootstrap ✅ G3 controllers_emit ✅ G4 target_soft_update ✅ G6 r7d_per_wiring ✅ smoke ✅ all losses finite ## Expected effect on next 50k smoke * lr_q stays near MIN (already worked — Q grad-norm > target_Q typically) — unchanged. * lr_pi should NOT runaway to MAX — rate cap limits 1000× swing to at most 2× per step; per-head π target 0.3 puts the multiplicative ratio closer to 1.0 (no extreme target). * lr_v should also stabilise via the V-specific target 0.1. * Aggregate l_pi / l_v max values should drop from the 1e17 / 2e6 range to O(1). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Foxhunt
Production HFT trading system in Rust.
Architecture
The workspace contains 32 crates organized as follows:
Core Libraries (16)
| Crate | Purpose |
|---|---|
trading_engine |
Order processing, FIX 4.4, IB TWS, SIMD, RDTSC timing |
risk |
VaR, Kelly, circuit breakers, kill switches, compliance |
risk-data |
Risk data types and shared structures |
trading-data |
Trading data types |
ml |
DQN Rainbow, PPO, TFT, Mamba2, ensemble inference |
ml-data |
ML data types and feature definitions |
data |
Market data ingestion and storage |
backtesting |
Replay engine, strategy tester |
adaptive-strategy |
Ensemble execution, microstructure analysis |
common |
Shared types, resilience, error handling |
storage |
S3 and local model storage |
model_loader |
Model serialization and loading |
market-data |
Market data feed handlers |
database |
PostgreSQL access layer (SQLx) |
config |
Configuration management |
tli |
CLI commands and tooling |
Services (8)
| Service | Purpose |
|---|---|
backtesting_service |
gRPC backtesting service |
broker_gateway_service |
FIX routing, broker connectivity |
trading_service |
Core trading operations |
ml_training_service |
Model training orchestration |
data_acquisition_service |
Market data acquisition |
trading_agent_service |
Autonomous trading agents |
api_gateway |
gRPC API gateway with auth |
web-gateway |
Axum REST + WebSocket gateway |
Frontend
web-dashboard/ -- React 19 + TypeScript + Vite + TradingView charts.
Building
# Check compilation (no PostgreSQL required)
SQLX_OFFLINE=true cargo check --workspace
# Run tests for a specific crate
SQLX_OFFLINE=true cargo test -p <crate> --lib
# Clippy
SQLX_OFFLINE=true cargo clippy --workspace
ML Models
Four production model architectures on Candle v0.9.1 with CUDA:
- DQN Rainbow -- Deep Q-Network with prioritized replay, dueling heads, noisy nets
- PPO -- Proximal Policy Optimization with GAE and LSTM policies
- TFT -- Temporal Fusion Transformer for multi-horizon forecasting
- Mamba2 -- State space model for sequence prediction
Each model has a standalone trainer and a UnifiedTrainable adapter for the hyperopt pipeline.
Infrastructure
- Git: Gitea at
git.fxhnt.ai(Tailscale-only), Scaleway DEV1-S - Observability: OpenTelemetry OTLP (env
OTEL_EXPORTER_OTLP_ENDPOINT) - Database: PostgreSQL with SQLx offline mode for CI
License
Proprietary. All rights reserved.