383b1ad83c
The rl_lr_controller emitted a hardcoded `LR_BOOTSTRAP = 1e-3` for
every step regardless of training dynamics. The kernel accepted 5
`*_signal` scalar args but ignored them via `(void)signal;` — a
stub. This commit makes the LR genuinely signal-driven per
`pearl_controller_anchors_isv_driven` + `feedback_isv_for_adaptive_bounds`.
## Architecture
Per-head grad-norm EMA → LR target derivation:
observed_grad_norm = EMA(‖grad_w_head‖₂)
target_lr = lr_prev × (TARGET_GRAD_NORM / max(observed, ε))
Wiener-α blend (floor 0.4) + clamp to [LR_MIN, LR_MAX].
Multiplicative pattern — same shape as rl_target_tau / rl_ppo_clip
controllers. High observed gradient (model thrashing) shrinks LR
(calm updates); low observed gradient (model coasting) grows LR
(push more aggressive learning).
## Components
1. **rl_l2_norm.cu** (new) — single-buffer L2 norm `‖x‖₂` via
grid-stride loop + shared-mem tree reduce. Used for per-head
grad_w_*_d reductions.
2. **rl_lr_controller.cu** (rewrite) — kernel signature changes
from 5 scalar `*_signal` args to 5 `int *_signal_slot` args
(ISV slot indices). The kernel reads each signal from
`isv[slot]`, derives target multiplicatively, and applies the
cold-start gate + replace-directly pattern (same R9-audit fixes
that closed the dead-zones in the other multiplicative
controllers). BCE and AUX heads pass sentinel `-1` for their
signal slot (those heads are owned by the perception trainer);
the kernel falls back to LR_BOOTSTRAP for those.
3. **ISV slot extension** (`isv_slots.rs`):
* `RL_Q_GRAD_NORM_EMA_INDEX = 424`
* `RL_PI_GRAD_NORM_EMA_INDEX = 425`
* `RL_V_GRAD_NORM_EMA_INDEX = 426`
* `RL_SLOTS_END = 427` (was 424).
4. **Trainer wiring** (`integrated.rs`):
* New `rl_l2_norm` module + fn fields + load in `new()`.
* New `launch_l2_norm` helper (256-thread single-block reduce).
* After-encoder-backward block in `step_synthetic` gains 3
grad-norm + EMA launches (Q grad_w 24,192 floats, π grad_w
1,152 floats, V grad_w 128 floats) alongside the existing
entropy / td_kurtosis / kl_pi EMAs.
* `launch_rl_lr_controller` updated to pass i32 slot indices
instead of f32 scalars.
## What's NOT in this commit
* BCE and AUX LR signals — those heads' gradients live in the
perception trainer, not the RL trainer. A future commit can
wire `perception.bce_grad_w_d` → ISV slot if the BCE/AUX LRs
need to adapt for cross-trainer alignment.
* Production tuning of `TARGET_GRAD_NORM = 1.0`. Empirical from
the 50k smoke (Q grad_w L2 norm landed near 1 at LR=1e-3); the
smoke at this commit will confirm whether the LR controller
drives the grad-norm to this anchor.
## Verified gates (local sm_86)
G1 isv_bootstrap ✅ (per_α, γ, etc. — unchanged)
G3 controllers_emit ✅ (test pre-seeds inputs)
G4 target_soft_update ✅
G6 r7d_per_wiring ✅
smoke ✅ all losses finite
## Expected effect
Prior 50k run showed l_q oscillating in 2.7-4.4 range without
visible convergence at LR=1e-3 constant. With LR now adaptive,
the trainer should:
* Shrink Q LR when Q grad-norm spikes (large per-sample CE
after a big trade close).
* Grow Q LR when grad-norm stays small (steady-state coasting).
* Same logic for π and V.
Next cluster smoke at 50k steps will produce a diag.jsonl where
ISV[413..415] (lr_q, lr_pi, lr_v) AND ISV[424..426] (grad-norm
EMAs) both evolve over time — observable convergence dynamics
that previously didn't exist.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Foxhunt
Production HFT trading system in Rust.
Architecture
The workspace contains 32 crates organized as follows:
Core Libraries (16)
| Crate | Purpose |
|---|---|
trading_engine |
Order processing, FIX 4.4, IB TWS, SIMD, RDTSC timing |
risk |
VaR, Kelly, circuit breakers, kill switches, compliance |
risk-data |
Risk data types and shared structures |
trading-data |
Trading data types |
ml |
DQN Rainbow, PPO, TFT, Mamba2, ensemble inference |
ml-data |
ML data types and feature definitions |
data |
Market data ingestion and storage |
backtesting |
Replay engine, strategy tester |
adaptive-strategy |
Ensemble execution, microstructure analysis |
common |
Shared types, resilience, error handling |
storage |
S3 and local model storage |
model_loader |
Model serialization and loading |
market-data |
Market data feed handlers |
database |
PostgreSQL access layer (SQLx) |
config |
Configuration management |
tli |
CLI commands and tooling |
Services (8)
| Service | Purpose |
|---|---|
backtesting_service |
gRPC backtesting service |
broker_gateway_service |
FIX routing, broker connectivity |
trading_service |
Core trading operations |
ml_training_service |
Model training orchestration |
data_acquisition_service |
Market data acquisition |
trading_agent_service |
Autonomous trading agents |
api_gateway |
gRPC API gateway with auth |
web-gateway |
Axum REST + WebSocket gateway |
Frontend
web-dashboard/ -- React 19 + TypeScript + Vite + TradingView charts.
Building
# Check compilation (no PostgreSQL required)
SQLX_OFFLINE=true cargo check --workspace
# Run tests for a specific crate
SQLX_OFFLINE=true cargo test -p <crate> --lib
# Clippy
SQLX_OFFLINE=true cargo clippy --workspace
ML Models
Four production model architectures on Candle v0.9.1 with CUDA:
- DQN Rainbow -- Deep Q-Network with prioritized replay, dueling heads, noisy nets
- PPO -- Proximal Policy Optimization with GAE and LSTM policies
- TFT -- Temporal Fusion Transformer for multi-horizon forecasting
- Mamba2 -- State space model for sequence prediction
Each model has a standalone trainer and a UnifiedTrainable adapter for the hyperopt pipeline.
Infrastructure
- Git: Gitea at
git.fxhnt.ai(Tailscale-only), Scaleway DEV1-S - Observability: OpenTelemetry OTLP (env
OTEL_EXPORTER_OTLP_ENDPOINT) - Database: PostgreSQL with SQLx offline mode for CI
License
Proprietary. All rights reserved.