bac8fd28ce
Completes the controller-input EMA wiring across all 7 RL
controllers. Previously 6 of 7 EMA input slots received no signal,
freezing those controllers at bootstrap for the entire training run.
Three new derivation kernels populate the last three:
* `rl_kl_approx_b` — Schulman-style per-step KL approximation
`mean(log π_old(a) − log π_new(a))` over batches at the sampled
action. Single-block tree reduce; inputs are `log_pi_old_d`
(recorded at action sample time) and `pi_log_prob_d` (= log
π_new at the same action, output of PPO surrogate forward).
Feeds `kl_pi_ema` (ISV[419]) consumed by rl_ppo_clip.
* `rl_l2_diff_norm` — `‖W_online − W_target‖₂` over the full
DQN weight tensor (24,192 floats = 9 actions × 21 atoms × 128
hidden). Single-block grid-stride loop (256 threads, ~95
iterations each); shared-mem tree-reduce produces a scalar.
Feeds `q_divergence_ema` (ISV[418]) consumed by rl_target_tau.
Launched immediately after `soft_update_target` so the divergence
reflects the post-update gap.
* `rl_step_counter_update` — per-batch trade-duration counter +
done-gated emit. Trainer-owned `steps_since_done_d: [i32; B]`
increments every step and resets on done; on done the counter
value (= event count the position was open) is written to
`trade_duration_emit_d` for `ema_update_on_done` to fold into
`mean_trade_duration_ema` (ISV[417]) consumed by rl_gamma.
Element-wise, one thread per batch index.
## Trainer wiring placement
* trade_duration counter + EMA emit: in `step_with_lobsim`
immediately after `extract_realized_pnl_delta` populates dones_d,
BEFORE the controllers fire — γ adapts THIS step from a real
duration observation.
* q_divergence reduce + EMA: in `step_with_lobsim` immediately
after `dqn_head.soft_update_target` runs, so the divergence
captures the post-update gap. One step lag for the τ controller
(controllers fired earlier in step_with_lobsim, before
step_synthetic).
* kl_pi reduce + EMA: in `step_synthetic` end-of-function block
alongside entropy + td_kurtosis updates. Deferred to AFTER the
encoder backward so the `&self.perception` borrow held by
`h_t_borrow` releases before the `&mut self` launches. One
step lag for the ε controller.
## All 7 controller inputs now wired
| ISV slot | Input EMA | Producer |
|----------|-----------|----------|
| 417 | mean_trade_duration | rl_step_counter_update → ema_update_on_done |
| 418 | q_divergence | rl_l2_diff_norm → ema_update_per_step |
| 419 | kl_pi | rl_kl_approx_b → ema_update_per_step |
| 420 | entropy_observed | ema_update_per_step (direct mean on entropy_d) |
| 421 | advantage_var_ratio | rl_var_over_abs_mean_b → ema_update_per_step |
| 422 | td_kurtosis | rl_kurtosis_b → ema_update_per_step |
| 423 | mean_abs_pnl | abs_copy + ema_update_on_done (existing) |
Combined with the cold-start gate + replace-directly-on-first-warm
fixes from the prior two commits, all 7 controllers will:
1. Hold at bootstrap until their input EMA receives signal
(cold-start gate prevents migration to clamps during sentinel
input period).
2. Replace prev → target directly on first non-zero observation
(no 60% bootstrap contamination in the first warm step).
3. Wiener-α blend (floored at 0.4) on subsequent steps.
## Phase-prefix comment cleanup
Per directive to stop phase prefixing in code, scrubbed "R9 audit",
"Phase A", "Phase B" markers from comments I added across the
multi-commit fix sequence. The remaining "Phase B: cross-batch
param-grad reducer" in build.rs is a pre-existing comment on the
perception trainer's `reduce_axis0` kernel, unrelated to this work.
## Verified gates (local sm_86)
G1 isv_bootstrap ✅
G3 controllers_emit ✅ (test pre-seeds inputs)
G4 target_soft_update ✅
G6 r7d_per_wiring ✅
R3, R4, smoke ✅
The next cluster smoke at this SHA will produce a diag.jsonl where
ALL 7 controller-input EMAs evolve over the 1000 steps, and ALL 7
controllers visibly adapt — the first time the integrated trainer
has every adaptive controller wired since the rebuild plan was
written.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Foxhunt
Production HFT trading system in Rust.
Architecture
The workspace contains 32 crates organized as follows:
Core Libraries (16)
| Crate | Purpose |
|---|---|
trading_engine |
Order processing, FIX 4.4, IB TWS, SIMD, RDTSC timing |
risk |
VaR, Kelly, circuit breakers, kill switches, compliance |
risk-data |
Risk data types and shared structures |
trading-data |
Trading data types |
ml |
DQN Rainbow, PPO, TFT, Mamba2, ensemble inference |
ml-data |
ML data types and feature definitions |
data |
Market data ingestion and storage |
backtesting |
Replay engine, strategy tester |
adaptive-strategy |
Ensemble execution, microstructure analysis |
common |
Shared types, resilience, error handling |
storage |
S3 and local model storage |
model_loader |
Model serialization and loading |
market-data |
Market data feed handlers |
database |
PostgreSQL access layer (SQLx) |
config |
Configuration management |
tli |
CLI commands and tooling |
Services (8)
| Service | Purpose |
|---|---|
backtesting_service |
gRPC backtesting service |
broker_gateway_service |
FIX routing, broker connectivity |
trading_service |
Core trading operations |
ml_training_service |
Model training orchestration |
data_acquisition_service |
Market data acquisition |
trading_agent_service |
Autonomous trading agents |
api_gateway |
gRPC API gateway with auth |
web-gateway |
Axum REST + WebSocket gateway |
Frontend
web-dashboard/ -- React 19 + TypeScript + Vite + TradingView charts.
Building
# Check compilation (no PostgreSQL required)
SQLX_OFFLINE=true cargo check --workspace
# Run tests for a specific crate
SQLX_OFFLINE=true cargo test -p <crate> --lib
# Clippy
SQLX_OFFLINE=true cargo clippy --workspace
ML Models
Four production model architectures on Candle v0.9.1 with CUDA:
- DQN Rainbow -- Deep Q-Network with prioritized replay, dueling heads, noisy nets
- PPO -- Proximal Policy Optimization with GAE and LSTM policies
- TFT -- Temporal Fusion Transformer for multi-horizon forecasting
- Mamba2 -- State space model for sequence prediction
Each model has a standalone trainer and a UnifiedTrainable adapter for the hyperopt pipeline.
Infrastructure
- Git: Gitea at
git.fxhnt.ai(Tailscale-only), Scaleway DEV1-S - Observability: OpenTelemetry OTLP (env
OTEL_EXPORTER_OTLP_ENDPOINT) - Database: PostgreSQL with SQLx offline mode for CI
License
Proprietary. All rights reserved.