648078ce20
Architectural finding during Phase 3c implementation attempt:
Original spec: α adds `W[a] * state_121[b]` to scalar Q values. Adam-
trained W via dloss/dQ gradient.
Actual codebase: C51 distributional Q-learning. Q is computed from a
probability distribution over fixed atom positions; C51 loss is KL
divergence over distributions, not MSE over scalars. Adding a constant
to logits is softmax-shift-invariant; adding to scalar expected_Q
happens post-loss-path. Either way, the KL loss is invariant under
scalar Q bias → W gradient = 0 → W never trains → α is mathematically
ineffective as originally specified.
Principled fix
──────────────
Per-(b, a) atom-position shift:
z_n_effective[b, a, n] = z_n_base + W[a] * state_121[b] for all n
Shifts atom positions state-dependently. C51 Bellman projection
re-projects target onto shifted positions → loss depends on
`W[a] * state_121[b]` smoothly → W gradient flows from projection
arithmetic.
Implementation cost grows from ~25-35 hr to ~40-60 hr because the
atom-shift threads through every kernel that uses atom positions
for the direction branch:
- compute_expected_q (action selection)
- c51_loss_kernel (Bellman projection)
- c51_grad_kernel (backward dW + dstate from projection arithmetic)
- mag_concat_qdir (magnitude-branch conditioning Q_dir compute)
- quantile_q_select / iqn_dual_head_kernel (investigate)
The Phase A `aux_to_q_dir_bias_kernel.cu` + `aux_to_q_dir_bias_backward_kernel.cu` become architecturally redundant under the revised
design. The dW + dstate gradient computations integrate into
c51_grad_kernel's existing projection backward. Phase A kernels
stay as compiled cubins (committed in 464bc5f7a); future cleanup
commit may delete them.
Initialization: structural prior `W_init = [-0.5, 0.0, +0.5, 0.0]`
reflects domain belief that aux conviction × position-sign biases
toward aligned trades. Adam refines from there.
state_121 ∈ [-1, +1] = recentered aux p_up. When +1 (aux up):
Q_short atoms shift DOWN by 0.5 → action selection avoids Short
Q_long atoms shift UP by 0.5 → action selection biases Long
Hold / Flat unchanged
Adam-vs-fixed-W: the atom-shift design subsumes the fixed-W case
as a config choice (set Adam LR for W = 0). Per user directive
"no shortcuts", the full version is the canonical Phase 3-final
spec.
β + SP11 controller + A2 eval-side + telemetry unchanged from
prior spec sections — those parts of Phase 3 are architecturally
sound. Only α needed correction.
Files touched
─────────────
- docs/plans/2026-05-12-sp22-h6-phase3-alpha-beta.md:
Replaced "α — post-encoder bypass-head" section with
"α — atom-position shift (architecturally revised 2026-05-13)".
Documents the C51 incompatibility, the per-(b, a) atom-shift
fix, the structural-prior initialization, the kernel inventory
for atom-shift threading, the Adam wireup, and the gradient
routing decision (option (i) unchanged).
- docs/dqn-wire-up-audit.md:
"Phase 3c — α architectural finding + spec revision (2026-05-13)"
entry documenting the math obstacle, the principled fix, the
Phase A kernel redundancy under the revised design, and the
scope-revision rationale.
Next session
────────────
Runbook (`docs/plans/2026-05-13-sp22-h6-phase3-alpha-beta-runbook.md`)
α tasks (B9 Steps 4-9, C1) describe the original scalar-bias
implementation and need rewriting for atom-shift threading before
execution can resume.
Refs
────
- pearl_audit_unboundedness_for_implicit_asymmetry (motivates
per-action W signs for the structural prior)
- C51 distributional Q math: Bellemare, Dabney, Munos 2017
- pearl_no_partial_refactor (atom-shift threading is atomic across
all consumer kernels)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Foxhunt
Production HFT trading system in Rust.
Architecture
The workspace contains 32 crates organized as follows:
Core Libraries (16)
| Crate | Purpose |
|---|---|
trading_engine |
Order processing, FIX 4.4, IB TWS, SIMD, RDTSC timing |
risk |
VaR, Kelly, circuit breakers, kill switches, compliance |
risk-data |
Risk data types and shared structures |
trading-data |
Trading data types |
ml |
DQN Rainbow, PPO, TFT, Mamba2, ensemble inference |
ml-data |
ML data types and feature definitions |
data |
Market data ingestion and storage |
backtesting |
Replay engine, strategy tester |
adaptive-strategy |
Ensemble execution, microstructure analysis |
common |
Shared types, resilience, error handling |
storage |
S3 and local model storage |
model_loader |
Model serialization and loading |
market-data |
Market data feed handlers |
database |
PostgreSQL access layer (SQLx) |
config |
Configuration management |
tli |
CLI commands and tooling |
Services (8)
| Service | Purpose |
|---|---|
backtesting_service |
gRPC backtesting service |
broker_gateway_service |
FIX routing, broker connectivity |
trading_service |
Core trading operations |
ml_training_service |
Model training orchestration |
data_acquisition_service |
Market data acquisition |
trading_agent_service |
Autonomous trading agents |
api_gateway |
gRPC API gateway with auth |
web-gateway |
Axum REST + WebSocket gateway |
Frontend
web-dashboard/ -- React 19 + TypeScript + Vite + TradingView charts.
Building
# Check compilation (no PostgreSQL required)
SQLX_OFFLINE=true cargo check --workspace
# Run tests for a specific crate
SQLX_OFFLINE=true cargo test -p <crate> --lib
# Clippy
SQLX_OFFLINE=true cargo clippy --workspace
ML Models
Four production model architectures on Candle v0.9.1 with CUDA:
- DQN Rainbow -- Deep Q-Network with prioritized replay, dueling heads, noisy nets
- PPO -- Proximal Policy Optimization with GAE and LSTM policies
- TFT -- Temporal Fusion Transformer for multi-horizon forecasting
- Mamba2 -- State space model for sequence prediction
Each model has a standalone trainer and a UnifiedTrainable adapter for the hyperopt pipeline.
Infrastructure
- Git: Gitea at
git.fxhnt.ai(Tailscale-only), Scaleway DEV1-S - Observability: OpenTelemetry OTLP (env
OTEL_EXPORTER_OTLP_ENDPOINT) - Database: PostgreSQL with SQLx offline mode for CI
License
Proprietary. All rights reserved.