jgrusewski 26b51a4f99 feat(ml): real validation, transaction costs, and data fixes for DQN/PPO pipeline ...

Replace stub validation functions with real model inference (DQN greedy,
PPO act()) so early stopping optimizes actual trading performance instead
of market volatility. Add transaction costs (commission + bid-ask spread)
to reward computation across train/hyperopt/evaluate examples.

Key changes:
- Symbol filtering (--symbol ES.FUT) prevents mixing futures contracts
- BTreeMap timestamp dedup handles overlapping .FUT contract bars
- Return clamping (--max-bar-return) filters contract roll boundaries
- Warmup offset alignment fixes feature-to-bar index mismatch
- Kelly sizing: 3 stubs replaced with real data-driven implementations
- Adam optimizer: BUG #14 diagnostic logging demoted to trace
- TFT: varmap_mut() accessor for checkpoint loading
- PPO hyperopt: with_costs() builder for tx cost configuration

DQN eval (ES.FUT, 2 folds): Sharpe=11.36, MaxDD=7.42%, WinRate=33.2%

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

2026-02-24 00:43:11 +01:00

..

ab_test_demonstration.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

adaptive_ml_backtest.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

analyze_dqn_checkpoints.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

backtest_dqn_replay.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

backtest_dqn.rs

refactor(ml): consolidate 13 duplicate OHLCVBar definitions into single canonical type

2026-02-20 18:14:42 +01:00

backtest_ensemble.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

benchmark_cuda_speedup.rs

feat(ppo): gradient accumulation, clip-higher, and WorkingPPO→PPO rename

2026-02-21 01:00:54 +01:00

benchmark_future_decoder.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

benchmark_ppo_optimization.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

benchmark_streaming_vs_batch.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

benchmark_weight_caching.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

cache_dqn_features.rs

feat: WAVE 23 Complete - Early Stopping + Feature Caching (99% speedup)

2025-11-24 10:03:15 +01:00

check_feature_count.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

check_performance_regression.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

check_tft_weight_init.rs

feat(ml): MAMBA-2 P0 fixes + hyperparameter optimization (13 params)

2025-10-28 14:11:18 +01:00

comprehensive_model_backtest.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

convert_6e_dbn_to_parquet.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

convert_6e_parquet_simple.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

create_small_parquet_files.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

cross_validation_backtest.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

cuda_test.rs

🔧 Tonic 0.14 Upgrade: Auto-generated and build system changes

2025-10-03 07:34:26 +02:00

diagnose_factored_network.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

diagnose_q_value_diversity.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

download_baseline.rs

fix(ml): add stype_in=Parent for Databento .FUT symbol downloads

2026-02-23 20:02:38 +01:00

download_l2_data.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

download_l2_test.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

download_training_data.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

ensemble_uncertainty_demo.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

ensemble_visualization.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

evaluate_baseline.rs

feat(ml): real validation, transaction costs, and data fixes for DQN/PPO pipeline

2026-02-24 00:43:11 +01:00

evaluate_dqn_component5_usage_example.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

evaluate_dqn_component5.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

evaluate_dqn_load_function.rs

refactor: consolidate DQNConfig to single canonical definition

2026-02-22 20:49:22 +01:00

evaluate_dqn_main_orchestrator.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

evaluate_dqn.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

evaluate_ppo.rs

feat(ppo): gradient accumulation, clip-higher, and WorkingPPO→PPO rename

2026-02-21 01:00:54 +01:00

evaluate_production_checkpoint.rs

refactor(ml): consolidate 13 duplicate OHLCVBar definitions into single canonical type

2026-02-20 18:14:42 +01:00

feature_importance_analysis.rs

refactor(ml): consolidate 13 duplicate OHLCVBar definitions into single canonical type

2026-02-20 18:14:42 +01:00

generate_backtest_report.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

generate_calibration_dataset.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

gpu_memory_benchmark.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

gpu_memory_monitor.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

gpu_training_benchmark.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

hyperopt_baseline.rs

feat(ml): real validation, transaction costs, and data fixes for DQN/PPO pipeline

2026-02-24 00:43:11 +01:00

hyperopt_continuous_ppo_demo.rs

Wave 11: Rainbow DQN integration + 23/23 tests passing

2025-11-18 13:53:59 +01:00

hyperopt_dqn_demo.rs

feat: WAVE 23 Complete - Early Stopping + Feature Caching (99% speedup)

2025-11-24 10:03:15 +01:00

hyperopt_early_stopping_demo.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

hyperopt_mamba2_demo.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

hyperopt_ppo_demo.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

hyperopt_tft_demo.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

inference_benchmark.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

inspect_dbn_raw.rs

Wave 16S-V11: Bug #8 fix + P2-A/B implementation

2025-11-12 23:05:51 +01:00

inspect_safetensors.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

load_parquet_data_function.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

MAMBA2_DBN_INTEGRATION.md

🚀 Wave 160 Phase 2: ML Training Infrastructure + TLOB Investigation

2025-10-14 10:42:56 +02:00

measure_dqn_memory.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

model_diversity_analysis.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

model_registry_api.rs

feat(ml): MAMBA-2 P0 fixes + hyperparameter optimization (13 params)

2025-10-28 14:11:18 +01:00

ofi_features_demo.rs

feat: Wave 1 - Update HIGH RISK files (225→54 features)

2025-11-23 00:41:22 +01:00

optimize_all_models.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

optimize_barriers.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

optimize_batch_sizes.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

optimize_ensemble_weights.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

optimize_mamba2_egobox.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

optimize_mamba2_standalone.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

ppo_separate_lr_demo.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

profile_model_memory.rs

feat(ppo): gradient accumulation, clip-higher, and WorkingPPO→PPO rename

2026-02-21 01:00:54 +01:00

profile_tft_int8_memory.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

profile_training_memory_180d.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

quantize_tft_varmap.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

quantized_checkpoint_demo.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

quick_checkpoint_analysis.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

quick_performance_benchmark.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

README_MAMBA2_PARQUET.md

feat(wave1-2): Complete multi-model training architecture + TLI commands

2025-10-22 20:50:43 +02:00

real_time_inference_benchmark.rs

feat(ppo): gradient accumulation, clip-higher, and WorkingPPO→PPO rename

2026-02-21 01:00:54 +01:00

register_trained_models.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

retrain_all_models.rs

feat(ml): wire PPO/TFT/Mamba2 training in retrain_all_models

2026-02-21 19:28:57 +01:00

simple_dqn_eval.rs

refactor(ml): consolidate 13 duplicate OHLCVBar definitions into single canonical type

2026-02-20 18:14:42 +01:00

six_model_ensemble.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

stress_test_dqn.rs

feat: Wave 16 - Complete DQN advanced risk management integration

2025-11-13 19:14:20 +01:00

test_adamw_optimizer.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

test_adaptive_regime_detection.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

test_cuda_basic.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

test_dbn_loading.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

test_dbn_prices.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

test_dqn_init.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

test_ensemble.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

test_factored_q_values.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

test_future_decoder.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

test_gpu_hardware.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

test_gradient_checkpointing.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

test_lstm_ppo.rs

feat(ppo): gradient accumulation, clip-higher, and WorkingPPO→PPO rename

2026-02-21 01:00:54 +01:00

test_memory_optimization.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

test_new_factored_network.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

test_parquet_schema.rs

feat(wave12): Complete ML warning fixes and add Parquet training infrastructure

2025-10-21 08:54:26 +02:00

test_polyak_averaging.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

test_ppo_fix.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

test_symmetric_quantization.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

test_tft_fp32_varmap.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

tft_int8_calibration_simple.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

tft_int8_calibration.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

train_baseline.rs

feat(ml): real validation, transaction costs, and data fixes for DQN/PPO pipeline

2026-02-24 00:43:11 +01:00

train_continuous_ppo_parquet.rs

fix(ml): fix PPO flow policy shape dimensions and clean up tests

2026-02-20 23:52:12 +01:00

train_dqn_es_fut.rs

feat(hyperopt): Complete DQN hyperopt analysis and PSO optimizer fix

2025-11-02 21:49:07 +01:00

train_dqn_production.rs

feat(ml): add inference demo to train_dqn_production example

2026-02-20 21:32:58 +01:00

train_dqn.rs

fix(ml): fix PPO flow policy shape dimensions and clean up tests

2026-02-20 23:52:12 +01:00

train_liquid_dbn.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

train_mamba2_dbn.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

train_mamba2_parquet.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

train_mamba2.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

train_ppo_es_fut.rs

feat(hyperopt): Complete DQN hyperopt analysis and PSO optimizer fix

2025-11-02 21:49:07 +01:00

train_ppo_extended.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

train_ppo_parquet.rs

fix(ml): fix PPO flow policy shape dimensions and clean up tests

2026-02-20 23:52:12 +01:00

train_ppo.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

train_rainbow.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

train_tft_dbn.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

train_tft_parquet.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

train_tft_qat.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

train_tft.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

train_tlob.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

tune_hyperparameters.rs

fix(dqn): Integrate Bug #1-3 fixes from Wave B agents - Production ready

2025-11-04 23:54:18 +01:00

validate_54_features.rs

feat: Wave 8 - Integrate MBP-10 OFI features into DQN trainer

2025-11-23 14:24:50 +01:00

validate_225_features_databento.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

validate_225_features_runtime.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

validate_checkpoints.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

validate_databento_files.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

validate_dqn_225_features.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

validate_dqn_225_simple.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

validate_dqn_hyperopt_fixes.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

validate_dqn_real_training.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

validate_dqn_simple.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

validate_features_1_50.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

validate_features_151_200.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

validate_per_channel_quantization.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

validate_ppo_checkpoints.rs

feat(ppo): gradient accumulation, clip-higher, and WorkingPPO→PPO rename

2026-02-21 01:00:54 +01:00

validate_quantile_loss.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

validate_regime_features.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

validate_tft_hyperopt.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

validate_tft_int8_accuracy.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

validate_wave_c_features_51_150.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

verify_225_feature_values.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

verify_225_features_wave9.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

verify_action_mapping.rs

Wave 15: Complete FactoredAction migration + production monitoring

2025-11-11 23:48:02 +01:00

verify_dbn_loader_zero_free.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

verify_feature_dims.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

verify_grn_weight_init.rs

feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents

2025-10-19 09:10:55 +02:00

verify_mamba2_dimensions.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

wave_c_backtest.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

wave_comparison_full_features.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

wave_comparison_simple.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00

wave_d_backtest.rs

feat: Wave 2 - Update MEDIUM RISK files (225→54 features)

2025-11-23 00:57:17 +01:00