b93971726d
First behavior change since B-7 (preceding B-8/B-9/B-10 were
observability-only). Attenuates the distill gradient when softmax(Q/τ)
is near-uniform — when target_entropy → ln(N_ACTIONS), the distill term
is pure max-entropy regularization with no informational signal, so it
fights PPO instead of carrying Q's preferences into π.
Verified cluster cause @ alpha-rl-88f5c (B-10 smoke, c1dc84a34):
target_entropy = 2.3976 / ln(11) = 2.398 max → 99.98% of max with Q_range
13.79 and τ=20.18. Distill at λ=0.224 was applying ~uniform-target KL
regularization across 20k steps → eval pnl -$218M at full run
(alpha-rl-8gtk2).
Gate (per-block, smooth):
λ_eff = λ_base × max(0, 1 − h_target / ln(N_ACTIONS))^p
where λ_base is the existing KL-target Schulman controller's output
(slot 486, untouched) and p defaults to 2.0 (slot 744).
Bitwise disabled-mode: when RL_Q_DISTILL_INFO_GATE_ENABLED_INDEX = 0.0
the ternary forces gate_factor = 1.0 verbatim and s_lambda_eff = lambda
exactly. Same ISV-toggle pattern as B-7's reward-clamp control (slot 724).
Source delta:
- 3 new ISV slots: RL_Q_DISTILL_INFO_GATE_ENABLED_INDEX = 743,
RL_Q_DISTILL_INFO_GATE_SENSITIVITY_INDEX = 744,
RL_Q_DISTILL_LAMBDA_EFFECTIVE_INDEX = 745; RL_SLOTS_END 743 → 746.
- Bootstrap array 230 → 233 (defaults: ENABLED=1.0, SENSITIVITY=2.0,
LAMBDA_EFFECTIVE=0.0 sentinel; kernel block-0 overwrites every step).
- rl_q_pi_distill_grad.cu: shared s_lambda_eff, per-block target-entropy
reduction in the existing thread-0 s_pi_target block, gate compute,
__syncthreads broadcast; grad_distill uses s_lambda_eff. Block-0 emits
slot 745 alongside existing B-10 G2 emits.
- Diag: policy_diagnostic.q_distill_lambda_effective (672 leaves total).
- New invariant smoke: tests/q_distill_info_gate_invariants.rs (4 gates).
Local 200+50 b=16 fold-1 smoke validates all 4 invariants across 249 rows:
peak h_target/ln(N) = 1.000 (β cause confirmed at smoke scale too); gate
factor range [0, 5.99e-6]; 132 saturated rows (h_frac ≥ 0.999). l_q_b
healthy throughout.
NOT byte-identical to pre-B-11-β runs by design: gated grad_distill
changes π → action distribution → trade outcomes. V4 (spec §4) calls
this out explicitly. Cluster comparison is on aggregate signals (eval
pnl, l_pi trajectory), not exact reproducibility.
Falsification criteria (spec §2.G5) gate the next iteration:
- eval pnl > -$50M → β was dominant, B-11-β is the fix
- -$50M to -$100M → β contributed, write B-11-γ (advantage standardization)
- < -$100M → β not dominant alone; B-11-α + B-11-γ combined
Risk D explicit threshold (spec §8): if λ_base (slot 486) exceeds 2× its
pre-B-11-β baseline EMA sustained 5k steps, the KL-target controller is
in compensatory-ramp territory and B-11-β.1 gates the controller's input
on the gated KL.
Spec: docs/superpowers/specs/2026-06-01-b11-beta-q-distill-informativeness-gate.md
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Foxhunt
Production HFT trading system in Rust.
Architecture
The workspace contains 32 crates organized as follows:
Core Libraries (16)
| Crate | Purpose |
|---|---|
trading_engine |
Order processing, FIX 4.4, IB TWS, SIMD, RDTSC timing |
risk |
VaR, Kelly, circuit breakers, kill switches, compliance |
risk-data |
Risk data types and shared structures |
trading-data |
Trading data types |
ml |
DQN Rainbow, PPO, TFT, Mamba2, ensemble inference |
ml-data |
ML data types and feature definitions |
data |
Market data ingestion and storage |
backtesting |
Replay engine, strategy tester |
adaptive-strategy |
Ensemble execution, microstructure analysis |
common |
Shared types, resilience, error handling |
storage |
S3 and local model storage |
model_loader |
Model serialization and loading |
market-data |
Market data feed handlers |
database |
PostgreSQL access layer (SQLx) |
config |
Configuration management |
tli |
CLI commands and tooling |
Services (8)
| Service | Purpose |
|---|---|
backtesting_service |
gRPC backtesting service |
broker_gateway_service |
FIX routing, broker connectivity |
trading_service |
Core trading operations |
ml_training_service |
Model training orchestration |
data_acquisition_service |
Market data acquisition |
trading_agent_service |
Autonomous trading agents |
api_gateway |
gRPC API gateway with auth |
web-gateway |
Axum REST + WebSocket gateway |
Frontend
web-dashboard/ -- React 19 + TypeScript + Vite + TradingView charts.
Building
# Check compilation (no PostgreSQL required)
SQLX_OFFLINE=true cargo check --workspace
# Run tests for a specific crate
SQLX_OFFLINE=true cargo test -p <crate> --lib
# Clippy
SQLX_OFFLINE=true cargo clippy --workspace
ML Models
Four production model architectures on Candle v0.9.1 with CUDA:
- DQN Rainbow -- Deep Q-Network with prioritized replay, dueling heads, noisy nets
- PPO -- Proximal Policy Optimization with GAE and LSTM policies
- TFT -- Temporal Fusion Transformer for multi-horizon forecasting
- Mamba2 -- State space model for sequence prediction
Each model has a standalone trainer and a UnifiedTrainable adapter for the hyperopt pipeline.
Infrastructure
- Git: Gitea at
git.fxhnt.ai(Tailscale-only), Scaleway DEV1-S - Observability: OpenTelemetry OTLP (env
OTEL_EXPORTER_OTLP_ENDPOINT) - Database: PostgreSQL with SQLx offline mode for CI
License
Proprietary. All rights reserved.