jgrusewski 8a25b330fc fix(sp9): targets as Invariant-1 anchors + divergence sentinel handling
Two targeted fixes to the SP9 Kelly cold-start warmup floor (Fix 37,
commit `48a8b9ee7`) for quirks observed in smoke-test-wrwkz on a
5-epoch L40S smoke. Atomic per `feedback_no_partial_refactor`.

**Quirk 1 — EMA target saturation:** ep1 HEALTH_DIAG showed
`stat_count_tgt=419 div_tgt=105002 temp_tgt=1.0 conf [stat=1.000
bhv=0.333 tmp=1.000] floor=0.0`. Pearl A sentinel-bootstrap on the
3 EMA target updaters (`kelly_*_target_ema_kernel.cu`) caused the
first observation to replace the sentinel directly, so
`target = first_obs`, `current/target = 1.0`, `confidence = 1.0`,
`floor = base × (1 − 1.0) = 0` — defeating the cold-start mechanism.

The principled fix per `feedback_isv_for_adaptive_bounds.md`:
"sufficient" is defined in absolute terms via Invariant-1 numerical
anchors. The 3 target ISV slots become constructor-written constants
(written ONCE in trainer constructor, identical pattern to existing
`config.cql_alpha → ISV[CQL_ALPHA_INDEX=48]` from Plan 1 Task 12).

  - ISV[KELLY_SAMPLE_COUNT_TARGET_INDEX=333] = 100.0 (trades)
  - ISV[KELLY_DIVERGENCE_TARGET_INDEX=334]   = 2.0   (ratio)
  - ISV[KELLY_TEMPORAL_TARGET_INDEX=335]     = 5.0   (epochs)

**Quirk 2 — divergence ratio explosion:** ep1 showed
`divergence=70005`. Algebraically `intent_f / max(eval_f, 1e-6) =
0.07 / 1e-6 = 70 000` because `ISV[EVAL_DIST_F_INDEX=338]` was still
at Pearl A sentinel 0 before the first val window populated it.
Algebraically the warmup-floor kernel still derived
`behavioral_conf = 0` (correct semantic), but the synthetic 70 000
in HEALTH_DIAG masked the real signal flow.

Fix: explicit sentinel detection in
`intent_eval_divergence_compute_kernel.cu`:

  if (eval_f < SENTINEL_THRESHOLD=1e-5)
    divergence = SENTINEL_DIVERGENCE=1e6  // forces behavioral_conf=0
  else
    divergence = intent_f / eval_f

Same `behavioral_conf = 0` outcome but with explicit provenance —
HEALTH_DIAG now reads `divergence=1e6` until eval_f matures.

**Atomic structure:**

- DELETE 3 EMA target updater kernels (`.cu` files)
- DELETE 3 entries from `crates/ml/build.rs::kernels_with_common`
- DELETE 3 cubin static byte arrays + 3 struct fields + 3 cubin
  loaders + 3 fields in trainer construction tuple in
  `gpu_dqn_trainer.rs`
- DELETE 3 launches in `launch_sp9_kelly_warmup_floor` (chain
  shrinks 5 → 2: q_var_mag_ema + main warmup-floor)
- DELETE 3 scratch slots; SP5_SCRATCH_TOTAL 268 → 265;
  SCRATCH_SP9_EVAL_DIST_BASE slides 265 → 262
- ADD constructor-write of 3 Invariant-1 anchors (100.0, 2.0, 5.0)
  immediately before layout-fingerprint write
- UPDATE 3 dispatch arms in `reset_named_state` to rewrite the
  Invariant-1 anchors at fold boundary (NOT sentinel 0) — these
  are constants, not stateful EMAs
- UPDATE 3 registry descriptions to reflect Invariant-1 anchor
  semantic + smoke-test-wrwkz evidence
- UPDATE `intent_eval_divergence_compute_kernel.cu` with
  sentinel-detect branch
- ISV slot layout UNCHANGED (3 target slots @ ISV[333..336)
  remain); Wiener-buffer linear span (`SP5_PRODUCER_COUNT=165`)
  UNCHANGED — 3 wiener triples for deleted producers become
  reserved-unused like Pearl 6's [525..543) carve-out
- audit doc Fix 37.1 entry with full provenance + smoke evidence

**Verification:**

- `SQLX_OFFLINE=true cargo check -p ml` — clean (only pre-existing
  18 warnings; no new errors or warnings).
- `SQLX_OFFLINE=true cargo test -p ml --lib state_reset` — 4/4 pass
  including `every_fold_and_soft_reset_entry_has_dispatch_arm`.
- `SQLX_OFFLINE=true cargo test -p ml --lib sp5_isv_slots` — 9/9
  pass including `sp9_slots_contiguous_above_sp8_block`.

**Expected smoke signature post-fix:**

- floor: starts ≈ base_floor × 1.0, decays as confidence accumulates
- divergence: 1e6 until first val window, small (≤ 5) afterward
- conf: stat=count/100, bhv=0 until eval_dist matures, tmp=ep/5
- combined_conf reaches 1.0 only when at least one axis genuinely
  matures (typically temporal at epoch 5 in fold 0)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 21:36:11 +02:00

Foxhunt

Production HFT trading system in Rust.

Architecture

The workspace contains 32 crates organized as follows:

Core Libraries (16)

Crate Purpose
trading_engine Order processing, FIX 4.4, IB TWS, SIMD, RDTSC timing
risk VaR, Kelly, circuit breakers, kill switches, compliance
risk-data Risk data types and shared structures
trading-data Trading data types
ml DQN Rainbow, PPO, TFT, Mamba2, ensemble inference
ml-data ML data types and feature definitions
data Market data ingestion and storage
backtesting Replay engine, strategy tester
adaptive-strategy Ensemble execution, microstructure analysis
common Shared types, resilience, error handling
storage S3 and local model storage
model_loader Model serialization and loading
market-data Market data feed handlers
database PostgreSQL access layer (SQLx)
config Configuration management
tli CLI commands and tooling

Services (8)

Service Purpose
backtesting_service gRPC backtesting service
broker_gateway_service FIX routing, broker connectivity
trading_service Core trading operations
ml_training_service Model training orchestration
data_acquisition_service Market data acquisition
trading_agent_service Autonomous trading agents
api_gateway gRPC API gateway with auth
web-gateway Axum REST + WebSocket gateway

Frontend

web-dashboard/ -- React 19 + TypeScript + Vite + TradingView charts.

Building

# Check compilation (no PostgreSQL required)
SQLX_OFFLINE=true cargo check --workspace

# Run tests for a specific crate
SQLX_OFFLINE=true cargo test -p <crate> --lib

# Clippy
SQLX_OFFLINE=true cargo clippy --workspace

ML Models

Four production model architectures on Candle v0.9.1 with CUDA:

  • DQN Rainbow -- Deep Q-Network with prioritized replay, dueling heads, noisy nets
  • PPO -- Proximal Policy Optimization with GAE and LSTM policies
  • TFT -- Temporal Fusion Transformer for multi-horizon forecasting
  • Mamba2 -- State space model for sequence prediction

Each model has a standalone trainer and a UnifiedTrainable adapter for the hyperopt pipeline.

Infrastructure

  • Git: Gitea at git.fxhnt.ai (Tailscale-only), Scaleway DEV1-S
  • Observability: OpenTelemetry OTLP (env OTEL_EXPORTER_OTLP_ENDPOINT)
  • Database: PostgreSQL with SQLx offline mode for CI

License

Proprietary. All rights reserved.

Description
No description provided
Readme 849 MiB
Languages
Rust 88.2%
Cuda 7.7%
Python 1.3%
Shell 1.1%
PLpgSQL 0.8%
Other 0.8%