jgrusewski 2fb7d7f57c fix(data): Fix 30 Stale-B — backtest_plan_kernel raw_close from prices_buf (post-MappedF32Buffer migration)
Closes Fix 29 audit row #13 — the last ⚠ Stale row from the Bug-1
contract drift triage. Pre-fix, `backtest_plan_state_isv` extracted
raw_close as `features[bar*feat_dim + 0]`. Post Bug-1 (commit
`5a5dd0fed`) `features[..+0]` is z-normed log-return, NOT raw_close.
The resulting `equity = cash + position*raw_close` and
`unrealized = position*(raw_close - entry_price)` formulas mixed
z-normed-log-return as a dollar price, corrupting val plan_isv
slots [PNL_VS_TARGET] (slot 1) and [PNL_VS_STOP] (slot 2). Other
plan_isv slots (progress, conviction, drift, regime, remaining)
don't depend on raw_close and were correct pre-fix.

Resolution: route raw_close from the upload-once `prices` buffer
(layout `[n*max_len*4]` raw OHLC). Close is at column index 3 — the
same index `backtest_env_kernel.cu` already reads from for portfolio
mark-to-market (line 64 of that kernel; OHLC layout is canonical
across the val backtest path). Reading from `prices` aligns the val
plan_isv path with env_step's source-of-truth, eliminating the
mixed-units pathology end-to-end.

Sites fixed:
  - crates/ml/src/cuda_pipeline/backtest_plan_kernel.cu:74-100
    Kernel signature: `const float* features` and `int feat_dim`
    parameters dropped, replaced by `const float* prices` (the
    [n*max_len*4] raw OHLC buffer). Raw_close read becomes
    `prices[(w*max_len + current_step)*4 + 3]`. Multi-line comment
    block documents the Bug-1 origin and the env_step parity
    reference. The `bool have_close` / `prices != nullptr` guard
    semantics preserved — callers without OHLC data fall back to
    `unrealized = 0` exactly as pre-fix.
  - crates/ml/src/cuda_pipeline/gpu_backtest_evaluator.rs:~1956
    Single launcher (`evaluate_dqn_graphed` chunk loop, the only
    invocation site of `plan_state_isv_kernel`) updated to pass
    `&self.prices_buf.dev_ptr` instead of `&self.features_buf.dev_ptr`
    and to drop the now-unused `feat_dim_i32` local + `.arg()` call.
    Inline comment explains the Bug-1 origin.
  - docs/dqn-wire-up-audit.md
    Stale-B row appended to Fix 30's table. The standalone Stale-B
    DEFERRED paragraph at the bottom replaced by the commit summary
    + the Fix 30 closure note (all 4 ⚠ Stale and 1  Ambiguous rows
    from Fix 29's deferred follow-ups now resolved).

Migration scope per `feedback_no_partial_refactor`: kernel signature
changed → every consumer migrates in the same commit. Single launcher;
verified via `grep -rn backtest_plan_state_isv` (only the kernel
definition + the gpu_backtest_evaluator launcher + load site appear).

Verification:
  - SQLX_OFFLINE=true cargo check -p ml --offline (43.68s) clean.
  - SQLX_OFFLINE=true cargo build -p ml --release --offline
    --features cuda (1m 30s) clean; cubin recompiled via nvcc.
  - Pre-commit DtoD-via-pinned guard passes (the prereq commit
    `4d966e62f` migrated `gpu_backtest_evaluator.rs`'s buffers to
    MappedF32Buffer, eliminating the 5 `_via_pinned` callers that
    had blocked any prior staging of this file).

Refs Fix 29 row #13. `feedback_no_partial_refactor` (single launcher
migrated alongside kernel signature change in one commit),
`feedback_no_functionality_removal` (PNL_VS_TARGET / PNL_VS_STOP
slots preserved — only their data source corrected; the audit's
"drop the slots" alternative explicitly rejected),
`feedback_no_hiding` (no fallback to z-normed reads remaining;
kernel either gets real raw_close or falls through to
`have_close=false` with `unrealized=0`, identical to pre-fix
smoke-test semantics where prices==NULL),
`feedback_no_cpu_compute_strict` n/a (zero new host-side compute),
`feedback_no_htod_htoh_only_mapped_pinned` already satisfied
(`prices_buf` is `MappedF32Buffer` post the prereq migration),
`feedback_trust_code_not_docs` (the kernel comment said
"raw_close from features buffer" for months — accurate-when-written
pre-Bug-1, stale-after-Bug-1; verify-against-code disambiguates).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 22:56:00 +02:00

Foxhunt

Production HFT trading system in Rust.

Architecture

The workspace contains 32 crates organized as follows:

Core Libraries (16)

Crate Purpose
trading_engine Order processing, FIX 4.4, IB TWS, SIMD, RDTSC timing
risk VaR, Kelly, circuit breakers, kill switches, compliance
risk-data Risk data types and shared structures
trading-data Trading data types
ml DQN Rainbow, PPO, TFT, Mamba2, ensemble inference
ml-data ML data types and feature definitions
data Market data ingestion and storage
backtesting Replay engine, strategy tester
adaptive-strategy Ensemble execution, microstructure analysis
common Shared types, resilience, error handling
storage S3 and local model storage
model_loader Model serialization and loading
market-data Market data feed handlers
database PostgreSQL access layer (SQLx)
config Configuration management
tli CLI commands and tooling

Services (8)

Service Purpose
backtesting_service gRPC backtesting service
broker_gateway_service FIX routing, broker connectivity
trading_service Core trading operations
ml_training_service Model training orchestration
data_acquisition_service Market data acquisition
trading_agent_service Autonomous trading agents
api_gateway gRPC API gateway with auth
web-gateway Axum REST + WebSocket gateway

Frontend

web-dashboard/ -- React 19 + TypeScript + Vite + TradingView charts.

Building

# Check compilation (no PostgreSQL required)
SQLX_OFFLINE=true cargo check --workspace

# Run tests for a specific crate
SQLX_OFFLINE=true cargo test -p <crate> --lib

# Clippy
SQLX_OFFLINE=true cargo clippy --workspace

ML Models

Four production model architectures on Candle v0.9.1 with CUDA:

  • DQN Rainbow -- Deep Q-Network with prioritized replay, dueling heads, noisy nets
  • PPO -- Proximal Policy Optimization with GAE and LSTM policies
  • TFT -- Temporal Fusion Transformer for multi-horizon forecasting
  • Mamba2 -- State space model for sequence prediction

Each model has a standalone trainer and a UnifiedTrainable adapter for the hyperopt pipeline.

Infrastructure

  • Git: Gitea at git.fxhnt.ai (Tailscale-only), Scaleway DEV1-S
  • Observability: OpenTelemetry OTLP (env OTEL_EXPORTER_OTLP_ENDPOINT)
  • Database: PostgreSQL with SQLx offline mode for CI

License

Proprietary. All rights reserved.

Description
No description provided
Readme 849 MiB
Languages
Rust 88.2%
Cuda 7.7%
Python 1.3%
Shell 1.1%
PLpgSQL 0.8%
Other 0.8%