jgrusewski c90553953c feat(sp21): T1.2+T1.4 — enrichment real metrics + backtracking signal-driven (atomic)
Closes the remaining SP21 Tier 1 hardcoded-constant items.

T1.2 — enrichment fed real metrics, not placeholders:
  - was: extract_eval_trades_from_metrics(_, 60000.0, 0.0, 0.5, ...)
         with hardcoded trade_count=60000, total_pnl=0.0, win_rate=0.5
  - now: reads from self.last_val_metrics: Option<[f32; 14]> populated
         by val backtest pass at metrics.rs:868. Layout [2]=win_rate,
         [4]=total_trades, [7]=total_pnl. Cold-start fallback (None)
         is (0.0, 0.0, 0.0) — preferable to fabricated 60000-trade
         signal that biased E2/gamma/ensemble from epoch 0.
  - Per feedback_no_todo_fixme + feedback_no_stubs.

T1.4 — backtracking thresholds signal-driven:
  - Three hardcoded thresholds in run_backtracking_epoch_end replaced
    with sigma = sqrt(ISV[VAL_SHARPE_VAR_EMA_INDEX=351]) derivatives:

    a) Save trigger (improvement_rate > 0.01) → > 0.5σ.
       The 0.01 fired on every epoch (any tiny change > 0.01);
       0.5σ requires a meaningful move (typical sigma O(1-10)).
    b) Plateau-detection frozen check (abs(delta) < 0.01) → < 0.5σ.
       The 0.01 ~never fired; 0.5σ correctly identifies stagnation.
    c) Route acceptance (>= min_improvement_rate=0.1) → >= 1.5σ.
       Stricter than save-trigger as designed.

  - BacktrackingState::min_improvement_rate field deleted — replaced
    by per-call signal-driven computation. Floor 0.5 covers cold-start
    before var_ema bootstraps from sentinel per
    pearl_blend_formulas_must_have_permanent_floor.
  - Per feedback_isv_for_adaptive_bounds + feedback_adaptive_not_tuned.

Affected files:
  - crates/ml/src/trainers/dqn/trainer/training_loop.rs:1510-1530
    (T1.2 enrichment) + :7458-7530 (T1.4 sigma + 3 threshold sites)
  - crates/ml/src/trainers/dqn/trainer/mod.rs:108,142
    (T1.4 min_improvement_rate field deletion)

Verification:
  - cargo check -p ml --tests: passes (warnings only)
  - cargo test -p ml --lib early_stopping: 8/8 pass

Cumulative SP21 Tier 1 status: T1.1a ✓, T1.1b ✓, T1.2 ✓, T1.4 ✓,
T2.3 ✓ — Tier 1 closed. Tier 2 (check_early_stopping(avg_q_value)
deletion + enrichment.rs constants soup) is next.

Plan reference: docs/plans/2026-05-10-sp21-train-eval-coherence-isv-defrost.md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 20:45:00 +02:00

Foxhunt

Production HFT trading system in Rust.

Architecture

The workspace contains 32 crates organized as follows:

Core Libraries (16)

Crate Purpose
trading_engine Order processing, FIX 4.4, IB TWS, SIMD, RDTSC timing
risk VaR, Kelly, circuit breakers, kill switches, compliance
risk-data Risk data types and shared structures
trading-data Trading data types
ml DQN Rainbow, PPO, TFT, Mamba2, ensemble inference
ml-data ML data types and feature definitions
data Market data ingestion and storage
backtesting Replay engine, strategy tester
adaptive-strategy Ensemble execution, microstructure analysis
common Shared types, resilience, error handling
storage S3 and local model storage
model_loader Model serialization and loading
market-data Market data feed handlers
database PostgreSQL access layer (SQLx)
config Configuration management
tli CLI commands and tooling

Services (8)

Service Purpose
backtesting_service gRPC backtesting service
broker_gateway_service FIX routing, broker connectivity
trading_service Core trading operations
ml_training_service Model training orchestration
data_acquisition_service Market data acquisition
trading_agent_service Autonomous trading agents
api_gateway gRPC API gateway with auth
web-gateway Axum REST + WebSocket gateway

Frontend

web-dashboard/ -- React 19 + TypeScript + Vite + TradingView charts.

Building

# Check compilation (no PostgreSQL required)
SQLX_OFFLINE=true cargo check --workspace

# Run tests for a specific crate
SQLX_OFFLINE=true cargo test -p <crate> --lib

# Clippy
SQLX_OFFLINE=true cargo clippy --workspace

ML Models

Four production model architectures on Candle v0.9.1 with CUDA:

  • DQN Rainbow -- Deep Q-Network with prioritized replay, dueling heads, noisy nets
  • PPO -- Proximal Policy Optimization with GAE and LSTM policies
  • TFT -- Temporal Fusion Transformer for multi-horizon forecasting
  • Mamba2 -- State space model for sequence prediction

Each model has a standalone trainer and a UnifiedTrainable adapter for the hyperopt pipeline.

Infrastructure

  • Git: Gitea at git.fxhnt.ai (Tailscale-only), Scaleway DEV1-S
  • Observability: OpenTelemetry OTLP (env OTEL_EXPORTER_OTLP_ENDPOINT)
  • Database: PostgreSQL with SQLx offline mode for CI

License

Proprietary. All rights reserved.

Description
No description provided
Readme 849 MiB
Languages
Rust 88.2%
Cuda 7.7%
Python 1.3%
Shell 1.1%
PLpgSQL 0.8%
Other 0.8%