Phase 1 — Fix broken models (P0): - Diffusion: wire optimizer_step to actually apply gradients (was no-op) - TLOB: connect forward pass to projection layers (was Tensor::zeros) - Mamba2: F64→F32 migration across 5 files (~30x faster on L40S tensor cores) Phase 2 — Eliminate hot-path GPU sync stalls: - Mamba2: keep dt on GPU in discretize_ssm (4 functions, no CPU round-trip) - TFT: gate attention weight logging to eval only (8 syncs/forward eliminated) - Mamba2: defer loss scalar after backward (pipeline stall removed) - Mamba2: delete dead gradient clipping (4N wasted GPU syncs removed) Phase 3 — Enable BF16 for supervised models: - Flip mixed_precision defaults to true in 4 config locations - Fix cuda_layer_norm to support BF16/F16 via F32 intermediate Phase 4 — Raise hyperopt bounds for datacenter GPUs: - 7 adapters with VRAM-aware tiers (TFT, Liquid, TGGN, KAN, xLSTM, Diffusion, TLOB) — L40S gets full hidden_dim range - Fix L40S tier boundary (was excluded at <48000, now >=40000) Phase 5 — Update memory estimates: - 10 param_count estimates updated (DQN 200K→12M, TFT 2M→50M, etc.) - Fix power-of-two rounding (was wasting up to 49% of budget) - Correct MODEL_OVERHEAD_MB in DQN/PPO/TFT adapters Phase 6 — Fix per-epoch CPU bottlenecks: - PPO: deduplicate double advantage normalization (correctness fix) - PPO: GPU tensor reward normalization + explained variance - Fuse per-parameter grad norm to single GPU sync (xLSTM, KAN, TGGN) Phase 7 — Data pipeline: - GpuBufferPool: use from_slice (eliminate staging buffer copy) Phase 8 — Correctness: - TFT: remove broken .detach() in forward_checkpointed (restore gradients) - Update stale RTX 3050 Ti doc references 33 files changed, 2451 tests pass, 0 clippy warnings. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
foxhunt-e2e
End-to-end testing framework covering service orchestration, gRPC clients, database integration, ML pipeline, and complete trading workflows.
Key Types
E2ETestFramework-- core orchestratorServiceManager-- automated service startup/shutdownDatabaseTestHarness-- transaction-isolated DB testingMLPipelineTestHarness-- mock ML model inference
Test Categories
- Service startup, shutdown, recovery
- gRPC client connections, streaming, error handling
- ML inference, training, ensemble predictions
- Trading workflows, order lifecycle, risk, emergency stop
Usage
SQLX_OFFLINE=true cargo test -p foxhunt-e2e --lib