jgrusewski 5186701982 feat(sp21): T2.2 Phase 8.4 — v6 loose ends (win_conc / curric_conc / hindsight_mag display) (atomic)
Three targeted fixes for v6 smoke (train-x4m96) findings that Phase 8.2
left on the table. v6 cycle-by-cycle showed `win_conc=0.0000` and
`curric_conc=0.0000` pinned across all 9 cycles, and `hindsight_mag`
printed `0.0000` due to format-width rounding.

Fix 1: compute_winner_concentration guard threshold 0.1 → 0.01 × pnl_std
  v6 had pnl_std ≈ 1e-5 and val-trade all_mean ≈ 1e-7..1e-6. At 0.1×
  the threshold was 1e-6, still above typical all_mean → short-circuit
  fired every cycle. At 0.01× (threshold 1e-7) healthy small-but-positive
  policies emit a non-zero signal; degenerate-strategy guard preserved.

Fix 2: compute_curriculum_concentration formula
  Was: 1 - entropy(weights) / log(n)   (entropy-deficit, normalized)
  Now: CV(weights) / sqrt(n − 1)        (normalized coefficient of variation)

  Entropy-deficit is ~weight_std² near uniform. v6's 8 contiguous segments
  had per-segment Sharpes in a tight band, so normalized weights stayed
  within ~0.5% of 1/n and deficit collapsed to <1e-4 (only cycle 9
  reached 0.0001). CV scales linearly with weight_std/weight_mean,
  dramatically more sensitive in the near-uniform regime. Cold-start
  (uniform) still returns 0; one-hot returns 1.

Fix 3: hindsight_mag display format {:.4} → {:.2e}
  pnl_std ≈ 1e-5 on volume bars → hindsight magnitudes are similar scale,
  printed as 0.0000 under {:.4} even when the count is non-zero. Matches
  pnl_std={:.2e} format already in the log.

Pearls honoured:
  - feedback_isv_for_adaptive_bounds: thresholds derived from pnl_std
  - pearl_first_observation_bootstrap: uniform/empty inputs → 0.0 sentinel
  - pearl_controller_anchors_isv_driven: CV anchor (one-hot = sqrt(n-1))
    is structural, not magic
  - feedback_no_quickfixes: entropy → CV is a structural reformulation

Verification:
  cargo check -p ml --features cuda  # clean

Expected v8 smoke (when dispatched):
  - win_conc rises to ~1.5..3.0 (top_decile_mean / all_mean ratio)
  - curric_conc enters 0.05..0.20 range as segments develop differential
    difficulty; grows toward ~0.4 with overfit divergence
  - hindsight_mag prints as 5e-6 or similar (real value, not zero)

If these fire, Phase 7 (alpha boost via E6/E7/E8) is finally exercised
end-to-end on volume bars.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 11:15:13 +02:00

Foxhunt

Production HFT trading system in Rust.

Architecture

The workspace contains 32 crates organized as follows:

Core Libraries (16)

Crate Purpose
trading_engine Order processing, FIX 4.4, IB TWS, SIMD, RDTSC timing
risk VaR, Kelly, circuit breakers, kill switches, compliance
risk-data Risk data types and shared structures
trading-data Trading data types
ml DQN Rainbow, PPO, TFT, Mamba2, ensemble inference
ml-data ML data types and feature definitions
data Market data ingestion and storage
backtesting Replay engine, strategy tester
adaptive-strategy Ensemble execution, microstructure analysis
common Shared types, resilience, error handling
storage S3 and local model storage
model_loader Model serialization and loading
market-data Market data feed handlers
database PostgreSQL access layer (SQLx)
config Configuration management
tli CLI commands and tooling

Services (8)

Service Purpose
backtesting_service gRPC backtesting service
broker_gateway_service FIX routing, broker connectivity
trading_service Core trading operations
ml_training_service Model training orchestration
data_acquisition_service Market data acquisition
trading_agent_service Autonomous trading agents
api_gateway gRPC API gateway with auth
web-gateway Axum REST + WebSocket gateway

Frontend

web-dashboard/ -- React 19 + TypeScript + Vite + TradingView charts.

Building

# Check compilation (no PostgreSQL required)
SQLX_OFFLINE=true cargo check --workspace

# Run tests for a specific crate
SQLX_OFFLINE=true cargo test -p <crate> --lib

# Clippy
SQLX_OFFLINE=true cargo clippy --workspace

ML Models

Four production model architectures on Candle v0.9.1 with CUDA:

  • DQN Rainbow -- Deep Q-Network with prioritized replay, dueling heads, noisy nets
  • PPO -- Proximal Policy Optimization with GAE and LSTM policies
  • TFT -- Temporal Fusion Transformer for multi-horizon forecasting
  • Mamba2 -- State space model for sequence prediction

Each model has a standalone trainer and a UnifiedTrainable adapter for the hyperopt pipeline.

Infrastructure

  • Git: Gitea at git.fxhnt.ai (Tailscale-only), Scaleway DEV1-S
  • Observability: OpenTelemetry OTLP (env OTEL_EXPORTER_OTLP_ENDPOINT)
  • Database: PostgreSQL with SQLx offline mode for CI

License

Proprietary. All rights reserved.

Description
No description provided
Readme 849 MiB
Languages
Rust 88.2%
Cuda 7.7%
Python 1.3%
Shell 1.1%
PLpgSQL 0.8%
Other 0.8%