jgrusewski 26b51a4f99 feat(ml): real validation, transaction costs, and data fixes for DQN/PPO pipeline ...

Replace stub validation functions with real model inference (DQN greedy,
PPO act()) so early stopping optimizes actual trading performance instead
of market volatility. Add transaction costs (commission + bid-ask spread)
to reward computation across train/hyperopt/evaluate examples.

Key changes:
- Symbol filtering (--symbol ES.FUT) prevents mixing futures contracts
- BTreeMap timestamp dedup handles overlapping .FUT contract bars
- Return clamping (--max-bar-return) filters contract roll boundaries
- Warmup offset alignment fixes feature-to-bar index mismatch
- Kelly sizing: 3 stubs replaced with real data-driven implementations
- Adam optimizer: BUG #14 diagnostic logging demoted to trace
- TFT: varmap_mut() accessor for checkpoint loading
- PPO hyperopt: with_costs() builder for tx cost configuration

DQN eval (ES.FUT, 2 folds): Sharpe=11.36, MaxDD=7.42%, WinRate=33.2%

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

2026-02-24 00:43:11 +01:00

..

asset_selection

fix(ml): address code review findings

2026-02-23 15:17:58 +01:00

backtesting

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

benchmark

refactor: consolidate DQNConfig to single canonical definition

2026-02-22 20:49:22 +01:00

bin

fix(ml): delete mock data loader from train_tft binary

2026-02-23 00:03:12 +01:00

checkpoint

feat(ml): SHA-256 checksum validation for model checkpoint integrity

2026-02-23 10:23:51 +01:00

common

refactor(ml): implement CircuitBreakerTrait for ML circuit breaker

2026-02-22 23:39:26 +01:00

config

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

cuda_common

🚀 PRODUCTION READY: Foxhunt HFT Trading System v1.0

2025-09-24 23:47:21 +02:00

data_loaders

feat(ml): implement microstructure features, TFT resource monitoring, document disabled tests

2026-02-23 01:03:35 +01:00

data_pipeline

fix(ml): address code review findings

2026-02-23 15:17:58 +01:00

data_validation

feat(ml): complete all algorithm gaps — TFT, Mamba2, PPO, CPCV, FDR

2026-02-20 19:19:16 +01:00

deployment

feat(trading_service): wire risk gRPC to real RiskEngine and kill switch

2026-02-21 23:28:49 +01:00

diffusion

feat(ml): add Diffusion model (DDPM/DDIM) for price path generation

2026-02-23 09:32:44 +01:00

dqn

fix(ml): fix PPO hyperopt data split and DQN diagnostic panic

2026-02-23 21:37:20 +01:00

ensemble

fix(ml): DST-aware sessions, normalize ensemble weights, filter non-finite predictions

2026-02-23 23:03:20 +01:00

evaluation

feat(trading_service): wire risk gRPC to real RiskEngine and kill switch

2026-02-21 23:28:49 +01:00

features

feat(ml): implement microstructure features, TFT resource monitoring, document disabled tests

2026-02-23 01:03:35 +01:00

flash_attention

refactor(ml): remove 797 lines of commented-out code and disabled imports

2026-02-20 19:43:47 +01:00

gpu

fix(ml): wire activation_multiplier, cache GPU detection, fix stale comments

2026-02-23 11:49:22 +01:00

gpu_benchmarks

refactor(ml): delete legacy SimpleNeuralNetwork and TrainingPipeline dead code

2026-02-23 00:07:59 +01:00

hyperopt

feat(ml): real validation, transaction costs, and data fixes for DQN/PPO pipeline

2026-02-24 00:43:11 +01:00

integration

feat(ml): add Diffusion model (DDPM/DDIM) for price path generation

2026-02-23 09:32:44 +01:00

kan

feat(ml): add KAN architecture + TLOB/KAN trainable/hyperopt adapters

2026-02-23 01:32:25 +01:00

labeling

feat(ml): factored 45-action masking, confidence EMA, RAII memory docs

2026-02-23 01:03:44 +01:00

liquid

refactor(ml): use canonical DeviceConfig from gpu module in Liquid CfC

2026-02-23 11:25:31 +01:00

mamba

fix(data, ml): replace expect() with recoverable error handling in streaming and Clone

2026-02-21 23:14:27 +01:00

memory_optimization

fix(ml): replace hardcoded t-distribution with statrs, compute quantization savings, parse safetensors header

2026-02-23 01:03:26 +01:00

metrics

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

microstructure

feat(ml): implement microstructure features, TFT resource monitoring, document disabled tests

2026-02-23 01:03:35 +01:00

model_registry

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

observability

merge: pull liquid CfC v2 and codebase deduplication from main into production-hardening

2026-02-23 10:09:25 +01:00

optimizers

feat(ml): real validation, transaction costs, and data fixes for DQN/PPO pipeline

2026-02-24 00:43:11 +01:00

paper_trading

feat(paper_trading): add PaperBroker and PnLTracker for simulated trading

2026-02-21 14:26:00 +01:00

ppo

feat(ml): NaN gradient detection with auto-halt during training

2026-02-23 11:26:49 +01:00

regime

feat(trading_service): wire risk gRPC to real RiskEngine and kill switch

2026-02-21 23:28:49 +01:00

registry

feat(ml,trading): add gate optimizer, model registry, and P&L attribution

2026-02-23 13:39:22 +01:00

risk

feat(ml): real validation, transaction costs, and data fixes for DQN/PPO pipeline

2026-02-24 00:43:11 +01:00

safety

feat(ml): factored 45-action masking, confidence EMA, RAII memory docs

2026-02-23 01:03:44 +01:00

security

fix(warnings): Eliminate 136 warnings across workspace via 11 parallel agents

2025-11-03 10:15:09 +01:00

stress_testing

chore(ml): remove orphan source files and zero-importer modules

2026-02-20 13:38:53 +01:00

tests

feat(deployment): Complete Runpod GPU deployment infrastructure

2025-10-24 01:11:43 +02:00

tft

feat(ml): real validation, transaction costs, and data fixes for DQN/PPO pipeline

2026-02-24 00:43:11 +01:00

tgnn

feat(ml): add UnifiedTrainable adapter for TGGN

2026-02-23 00:48:57 +01:00

tlob

feat(ml): add KAN architecture + TLOB/KAN trainable/hyperopt adapters

2026-02-23 01:32:25 +01:00

trainers

fix(ml): support zstd-compressed DBN files and recursive data directories

2026-02-23 20:49:30 +01:00

training

feat(ml): wire unified data loader to feature extractor

2026-02-21 21:22:34 +01:00

transformers

feat(ml): AttentionMask causal masking, document multi-asset DQN and conversion layer

2026-02-23 01:03:53 +01:00

types

refactor(ml): consolidate 13 duplicate OHLCVBar definitions into single canonical type

2026-02-20 18:14:42 +01:00

universe

cleanup(ml): remove 31 disabled imports and commented-out module blocks

2026-02-20 18:23:41 +01:00

validation

fix(ml): replace std::process::exit with Result propagation in temporal_guard tests

2026-02-23 09:46:05 +01:00

xlstm

feat(ml): add xLSTM architecture (sLSTM + mLSTM blocks, network, trainable, hyperopt)

2026-02-23 09:13:35 +01:00

.gitignore

🚀 PRODUCTION READY: Foxhunt HFT Trading System v1.0

2025-09-24 23:47:21 +02:00

batch_processing.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

benchmarks.rs

feat(trading_service): wire risk gRPC to real RiskEngine and kill switch

2026-02-21 23:28:49 +01:00

bridge.rs

🔧 Emergency Fix: Resolve catastrophic _i32 suffix corruption (463→0 errors)

2025-10-10 23:05:26 +02:00

cuda_compat.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

error.rs

refactor(ml): remove 797 lines of commented-out code and disabled imports

2026-02-20 19:43:47 +01:00

examples.rs

refactor: consolidate DQNConfig to single canonical definition

2026-02-22 20:49:22 +01:00

feature_cache.rs

feat: WAVE 23 Complete - Early Stopping + Feature Caching (99% speedup)

2025-11-24 10:03:15 +01:00

gradient_accumulation.rs

feat(ml): NaN gradient detection with auto-halt during training

2026-02-23 11:26:49 +01:00

gradient_utils.rs

feat(ml): NaN gradient detection with auto-halt during training

2026-02-23 11:30:01 +01:00

inference_validator.rs

🎯 Wave 159: Fix ML Training Infrastructure (22 Parallel Agents)

2025-10-14 09:06:37 +02:00

inference.rs

fix(ml): GPU OOM detection with automatic CPU fallback in inference engine

2026-02-23 10:06:40 +01:00

lib.rs

feat(ml): add walk-forward evaluation framework with normalization

2026-02-23 18:47:13 +01:00

model_factory.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

model_loader_integration.rs

fix(ml): remove panic vectors from NonZeroUsize::new and Debug mutex lock

2026-02-21 23:13:28 +01:00

model_registry.rs

feat(trading_agent): correlation matrix support in Markowitz allocation

2026-02-23 10:33:14 +01:00

model.rs

refactor(ml): remove 797 lines of commented-out code and disabled imports

2026-02-20 19:43:47 +01:00

models_demo.rs

feat(cleanup): Complete 30-agent codebase cleanup wave - 100% production ready

2025-10-23 09:16:58 +02:00

performance.rs

refactor(ml): remove 797 lines of commented-out code and disabled imports

2026-02-20 19:43:47 +01:00

portfolio_transformer.rs

fix(dqn): Wave D Production Readiness - 100% test pass rate

2025-11-05 02:21:58 +01:00

preprocessing.rs

feat: WAVE 20-22 - DQN 51-Feature + Kelly Integration Campaign Complete

2025-11-23 19:33:35 +01:00

production.rs

refactor(ml): remove 797 lines of commented-out code and disabled imports

2026-02-20 19:43:47 +01:00

qat_metrics_exporter.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

random_model.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

real_data_loader.rs

refactor(ml): consolidate 13 duplicate OHLCVBar definitions into single canonical type

2026-02-20 18:14:42 +01:00

regime_detection.rs

🔧 Major compilation fixes across entire workspace - Significant progress achieved

2025-09-27 20:56:22 +02:00

tensor_ops.rs

🔧 Tonic 0.14 Upgrade: Auto-generated and build system changes

2025-10-03 07:34:26 +02:00

training_pipeline.rs

refactor(ml): remove 797 lines of commented-out code and disabled imports

2026-02-20 19:43:47 +01:00

training.rs

refactor(ml): delete legacy SimpleNeuralNetwork and TrainingPipeline dead code

2026-02-23 00:07:59 +01:00

traits.rs

🔧 Wave 33: Test Compilation Improvements - 57 errors remaining

2025-10-01 21:24:28 +02:00

walk_forward.rs

feat(ml): add walk-forward evaluation framework with normalization

2026-02-23 18:47:13 +01:00