gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Files
01e5277e1c3d8b1b0b3af2f00b2325640a7d512d
foxhunt/ml/examples
History
jgrusewski 01e5277e1c fix(dqn): Fix 4 critical bugs + align hyperopt with production + implement HFT constraints 
This commit addresses critical bugs discovered during Wave 11 DQN hyperopt campaign
and implements HFT-specific constraint logic to guide optimization toward active trading.

## Bug Fixes

### Bug 1: epsilon_greedy_action placeholder (ml/src/trainers/dqn.rs:1646)
**Symptom**: Greedy action selection always returned BUY (action 0)
**Cause**: Placeholder `Ok(0)` never replaced with argmax(Q-values)
**Fix**: Implemented proper Q-network forward pass + argmax selection
**Impact**: Greedy action selection now correctly selects action with highest Q-value

### Bug 2: Epsilon-greedy during evaluation (ml/src/trainers/dqn.rs:492-540)
**Symptom**: Validation metrics contaminated with 5-30% random exploration
**Cause**: compute_validation_loss used epsilon-greedy instead of pure greedy
**Fix**: Added set_epsilon(0.0) before validation, restore original epsilon after
**Impact**: Evaluation now uses deterministic policy (Q-value argmax only)

### Bug 3: Epsilon decay per-step (ml/src/dqn/dqn.rs:618)
**Symptom**: Epsilon collapsed to floor (0.05) after only 2.1% of training
**Cause**: update_epsilon() called every training step (21,750×) instead of per epoch (5×)
**Math**: ε = 0.3 × 0.995^21750 ≈ 0.000001 → clamped to 0.05 floor at step 460
**Expected**: ε = 0.3 × 0.995^5 = 0.292 after 5 epochs
**Fix**: Removed epsilon decay from train_step, moved to epoch loop in trainer
**Impact**: Restored proper exploration schedule, action diversity now healthy

### Bug 4: Hyperopt-production parameter misalignment
**Symptom**: Hyperopt results not transferable to production (7 parameters diverged)
**Cause**: Parameters drifted over multiple development waves
**Critical**: hold_penalty_weight 0.01 vs 2.0 (200× difference)
**Fix**: Aligned all parameters with production values:
  - hold_penalty: -0.01 → -0.001 (production standard)
  - hold_penalty_weight: 0.01 → 2.0 (user-discovered optimal)
  - q_value_floor: 0.01 → 0.5 (early stopping threshold)
  - gradient_clip_norm: dynamic → fixed 10.0 (Wave 11 Bug #1 fix)
  - movement_threshold: optimized → fixed 0.02 (2% standard)
  - epsilon_start: 1.0 → 0.3 (production standard)
  - epsilon_decay: optimized → fixed 0.995 (production standard)

## HFT Constraint Logic (ml/src/hyperopt/adapters/dqn.rs)

**Motivation**: HFT trend-following requires active BUY/SELL decisions, not passive HOLD

### Parameter Space Changes
- **Before**: 4D (learning_rate, batch_size, gamma, buffer_size)
- **After**: 5D (added hold_penalty_weight: 0.5-5.0)
- **Removed**: movement_threshold (fixed 0.02), epsilon_decay (fixed 0.995)

### HFT Constraints (3 rules)
1. **Minimum penalty**: hold_penalty_weight ≥ 0.5 (force active trading)
2. **Training stability**: Low LR + very high penalty rejected (prevents instability)
3. **Buffer capacity**: Small buffer + high penalty rejected (prevents forgetting)

### Multi-Objective Enhancement
- **P&L**: 40% weight (primary objective)
- **HFT activity**: 30% weight (NEW - rewards BUY/SELL ratio, penalizes passive HOLD)
- **Stability**: 20% weight (low Q-value variance)
- **Completion**: 10% weight (early stopping penalty)

## Validation Results

**5-Epoch Test** (cargo run --release -p ml --example train_dqn --features cuda):
- Final epsilon: 0.2926 (matches expected 0.292)
- Action distribution: BUY 40%, SELL 10%, HOLD 50% (healthy diversity)
- Previous: 96.4% HOLD due to epsilon decay bug
- Q-values show continuous variation (argmax working correctly)

**Unit Tests**: 7/7 HFT constraint tests pass

## Files Modified
- ml/src/hyperopt/adapters/dqn.rs (268 lines changed)
  - Added hold_penalty_weight to search space
  - Implemented HFT constraints + enhanced multi-objective
  - Aligned all production parameters
  - Added 3 constraint unit tests
- ml/src/dqn/dqn.rs (12 lines changed)
  - Removed epsilon decay from train_step
  - Made update_epsilon public for trainer access
  - Added set_epsilon method
- ml/src/trainers/dqn.rs (54 lines changed)
  - Fixed epsilon_greedy_action argmax implementation
  - Added epsilon=0 during evaluation
  - Moved epsilon decay to epoch loop
- ml/examples/hyperopt_dqn_demo.rs (3 lines removed)
  - Removed epsilon_decay from parameter display
- ml/src/benchmark/dqn_benchmark.rs (1 line changed)
  - Aligned gradient_clip_norm with production (10.0)

## Breaking Changes
None - all changes internal to DQN hyperopt pipeline

## Next Steps
1. ✅ Validation complete (5-epoch test passed)
2. ⏳ Run hyperopt with HFT constraints (3-trial dry-run or 100-trial production)
3. ⏳ Deploy best parameters to production

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-06 22:31:00 +01:00
..
ab_test_demonstration.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
adaptive_ml_backtest.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
analyze_dqn_checkpoints.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
backtest_dqn_replay.rs
feat(hyperopt): Complete DQN hyperopt analysis and PSO optimizer fix
2025-11-02 21:49:07 +01:00
backtest_ensemble.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
benchmark_cuda_speedup.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
benchmark_future_decoder.rs
feat(wave1-2): Complete multi-model training architecture + TLI commands
2025-10-22 20:50:43 +02:00
benchmark_ppo_optimization.rs
feat(ml): Final Stabilization Wave - 100% FP32 test pass rate, QAT infrastructure
2025-10-25 15:36:57 +02:00
benchmark_streaming_vs_batch.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
benchmark_weight_caching.rs
feat(wave1-2): Complete multi-model training architecture + TLI commands
2025-10-22 20:50:43 +02:00
check_feature_count.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
check_performance_regression.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
check_tft_weight_init.rs
feat(ml): MAMBA-2 P0 fixes + hyperparameter optimization (13 params)
2025-10-28 14:11:18 +01:00
comprehensive_model_backtest.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
convert_6e_dbn_to_parquet.rs
feat(wave12): Complete ML warning fixes and add Parquet training infrastructure
2025-10-21 08:54:26 +02:00
convert_6e_parquet_simple.rs
feat(wave12): Complete ML warning fixes and add Parquet training infrastructure
2025-10-21 08:54:26 +02:00
create_small_parquet_files.rs
feat(wave1-2): Complete multi-model training architecture + TLI commands
2025-10-22 20:50:43 +02:00
cross_validation_backtest.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
cuda_test.rs
🔧 Tonic 0.14 Upgrade: Auto-generated and build system changes
2025-10-03 07:34:26 +02:00
download_l2_data.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
download_l2_test.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
download_training_data.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
ensemble_visualization.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
evaluate_dqn_component5_usage_example.rs
feat(hyperopt): Complete DQN hyperopt analysis and PSO optimizer fix
2025-11-02 21:49:07 +01:00
evaluate_dqn_component5.rs
feat(hyperopt): Complete DQN hyperopt analysis and PSO optimizer fix
2025-11-02 21:49:07 +01:00
evaluate_dqn_load_function.rs
feat(hyperopt): Complete DQN hyperopt analysis and PSO optimizer fix
2025-11-02 21:49:07 +01:00
evaluate_dqn_main_orchestrator.rs
feat(dqn): Wave 10 - Architectural improvements and bug fixes
2025-11-06 00:38:23 +01:00
evaluate_dqn.rs
feat(hyperopt): Complete DQN hyperopt analysis and PSO optimizer fix
2025-11-02 21:49:07 +01:00
evaluate_ppo.rs
feat(hyperopt): Complete DQN hyperopt analysis and PSO optimizer fix
2025-11-02 21:49:07 +01:00
feature_importance_analysis.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
generate_backtest_report.rs
fix(dqn): Integrate Bug #1-3 fixes from Wave B agents - Production ready
2025-11-04 23:54:18 +01:00
generate_calibration_dataset.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
gpu_memory_benchmark.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
gpu_memory_monitor.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
gpu_training_benchmark.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
hyperopt_dqn_demo.rs
fix(dqn): Fix 4 critical bugs + align hyperopt with production + implement HFT constraints
2025-11-06 22:31:00 +01:00
hyperopt_early_stopping_demo.rs
fix(ci): Fix GitLab CI YAML syntax and PPOConfig compilation errors
2025-10-31 00:20:00 +01:00
hyperopt_mamba2_demo.rs
feat(hyperopt): Complete DQN hyperopt analysis and PSO optimizer fix
2025-11-02 21:49:07 +01:00
hyperopt_ppo_demo.rs
refactor(ml): Remove default hyperparameters and add canonical configs
2025-11-02 11:12:14 +01:00
hyperopt_tft_demo.rs
refactor(ml): Remove default hyperparameters and add canonical configs
2025-11-02 11:12:14 +01:00
inference_benchmark.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
inspect_safetensors.rs
feat(hyperopt): Complete DQN hyperopt analysis and PSO optimizer fix
2025-11-02 21:49:07 +01:00
load_parquet_data_function.rs
feat(hyperopt): Complete DQN hyperopt analysis and PSO optimizer fix
2025-11-02 21:49:07 +01:00
MAMBA2_DBN_INTEGRATION.md
🚀 Wave 160 Phase 2: ML Training Infrastructure + TLOB Investigation
2025-10-14 10:42:56 +02:00
measure_dqn_memory.rs
feat(dqn): Wave 10 - Architectural improvements and bug fixes
2025-11-06 00:38:23 +01:00
model_diversity_analysis.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
model_registry_api.rs
feat(ml): MAMBA-2 P0 fixes + hyperparameter optimization (13 params)
2025-10-28 14:11:18 +01:00
optimize_all_models.rs
feat(ml): MAMBA-2 P0 fixes + hyperparameter optimization (13 params)
2025-10-28 14:11:18 +01:00
optimize_barriers.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
optimize_batch_sizes.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
optimize_ensemble_weights.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
optimize_mamba2_egobox.rs
feat(ml): MAMBA-2 P0 fixes + hyperparameter optimization (13 params)
2025-10-28 14:11:18 +01:00
optimize_mamba2_standalone.rs
feat(ml): MAMBA-2 P0 fixes + hyperparameter optimization (13 params)
2025-10-28 14:11:18 +01:00
ppo_separate_lr_demo.rs
feat(hyperopt): Complete DQN hyperopt analysis and PSO optimizer fix
2025-11-02 21:49:07 +01:00
profile_model_memory.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
profile_tft_int8_memory.rs
feat(wave1-2): Complete multi-model training architecture + TLI commands
2025-10-22 20:50:43 +02:00
profile_training_memory_180d.rs
feat(wave1-2): Complete multi-model training architecture + TLI commands
2025-10-22 20:50:43 +02:00
quantize_tft_varmap.rs
feat(wave1-2): Complete multi-model training architecture + TLI commands
2025-10-22 20:50:43 +02:00
quantized_checkpoint_demo.rs
feat(wave1-2): Complete multi-model training architecture + TLI commands
2025-10-22 20:50:43 +02:00
quick_checkpoint_analysis.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
quick_performance_benchmark.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
README_MAMBA2_PARQUET.md
feat(wave1-2): Complete multi-model training architecture + TLI commands
2025-10-22 20:50:43 +02:00
real_time_inference_benchmark.rs
feat(dqn): Wave 10 - Architectural improvements and bug fixes
2025-11-06 00:38:23 +01:00
register_trained_models.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
retrain_all_models.rs
fix(dqn): Integrate Bug #1-3 fixes from Wave B agents - Production ready
2025-11-04 23:54:18 +01:00
six_model_ensemble.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
test_adamw_optimizer.rs
feat(ml): MAMBA-2 P0 fixes + hyperparameter optimization (13 params)
2025-10-28 14:11:18 +01:00
test_adaptive_regime_detection.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
test_cuda_basic.rs
feat(wave1-2): Complete multi-model training architecture + TLI commands
2025-10-22 20:50:43 +02:00
test_dbn_loading.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
test_dbn_prices.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
test_ensemble.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
test_future_decoder.rs
feat(wave1-2): Complete multi-model training architecture + TLI commands
2025-10-22 20:50:43 +02:00
test_gpu_hardware.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
test_gradient_checkpointing.rs
feat(ml): 25-agent cleanup wave - QAT fixes + clippy + tests (Agents 1-25)
2025-10-23 10:43:52 +02:00
test_memory_optimization.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
test_parquet_schema.rs
feat(wave12): Complete ML warning fixes and add Parquet training infrastructure
2025-10-21 08:54:26 +02:00
test_ppo_fix.rs
fix(dqn): Integrate Bug #1-3 fixes from Wave B agents - Production ready
2025-11-04 23:54:18 +01:00
test_symmetric_quantization.rs
feat(wave1-2): Complete multi-model training architecture + TLI commands
2025-10-22 20:50:43 +02:00
test_tft_fp32_varmap.rs
feat(wave1-2): Complete multi-model training architecture + TLI commands
2025-10-22 20:50:43 +02:00
tft_int8_calibration_simple.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
tft_int8_calibration.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
train_dqn_es_fut.rs
feat(hyperopt): Complete DQN hyperopt analysis and PSO optimizer fix
2025-11-02 21:49:07 +01:00
train_dqn_production.rs
fix(dqn): Integrate Bug #1-3 fixes from Wave B agents - Production ready
2025-11-04 23:54:18 +01:00
train_dqn.rs
Wave 11: Fix 3 critical DQN bugs - All fixes implemented by 5 parallel agents
2025-11-06 01:17:53 +01:00
train_liquid_dbn.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
train_mamba2_dbn.rs
feat(ml): Complete hyperopt infrastructure + documentation
2025-10-29 19:52:21 +01:00
train_mamba2_parquet.rs
feat(ml): Complete hyperopt infrastructure + documentation
2025-10-29 19:52:21 +01:00
train_mamba2.rs
feat(ml): MAMBA-2 P0 fixes + hyperparameter optimization (13 params)
2025-10-28 14:11:18 +01:00
train_ppo_es_fut.rs
feat(hyperopt): Complete DQN hyperopt analysis and PSO optimizer fix
2025-11-02 21:49:07 +01:00
train_ppo_extended.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
train_ppo_parquet.rs
refactor(ml): Remove default hyperparameters and add canonical configs
2025-11-02 11:12:14 +01:00
train_ppo.rs
feat(ml): MAMBA-2 device fix + PPO batch size optimization + CUDA 12.9 migration
2025-10-26 11:14:33 +01:00
train_tft_dbn.rs
feat(ml): Implement Quantization-Aware Training (QAT) for TFT model
2025-10-21 21:13:11 +02:00
train_tft_parquet.rs
fix(ml): Add max_validation_batches to prevent validation OOM
2025-10-26 20:58:49 +01:00
train_tft_qat.rs
feat(ml): Implement Quantization-Aware Training (QAT) for TFT model
2025-10-21 21:13:11 +02:00
train_tft.rs
feat(ml): Implement Quantization-Aware Training (QAT) for TFT model
2025-10-21 21:13:11 +02:00
train_tlob.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
tune_hyperparameters.rs
fix(dqn): Integrate Bug #1-3 fixes from Wave B agents - Production ready
2025-11-04 23:54:18 +01:00
validate_225_features_databento.rs
feat(wave12): Complete ML warning fixes and add Parquet training infrastructure
2025-10-21 08:54:26 +02:00
validate_225_features_runtime.rs
feat(wave9-11): Complete 225-feature integration and service migration
2025-10-20 21:54:39 +02:00
validate_checkpoints.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
validate_databento_files.rs
feat(wave12): Complete ML warning fixes and add Parquet training infrastructure
2025-10-21 08:54:26 +02:00
validate_dqn_225_features.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
validate_dqn_225_simple.rs
feat(dqn): Wave 10 - Architectural improvements and bug fixes
2025-11-06 00:38:23 +01:00
validate_dqn_hyperopt_fixes.rs
feat(hyperopt): Fix all 29 critical issues - production certified
2025-10-28 16:11:01 +01:00
validate_dqn_real_training.rs
feat(hyperopt): Complete DQN hyperopt analysis and PSO optimizer fix
2025-11-02 21:49:07 +01:00
validate_dqn_simple.rs
feat(hyperopt): Complete DQN hyperopt analysis and PSO optimizer fix
2025-11-02 21:49:07 +01:00
validate_features_1_50.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
validate_features_151_200.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
validate_per_channel_quantization.rs
feat(wave1-2): Complete multi-model training architecture + TLI commands
2025-10-22 20:50:43 +02:00
validate_ppo_checkpoints.rs
fix(ci): Fix GitLab CI YAML syntax and PPOConfig compilation errors
2025-10-31 00:20:00 +01:00
validate_quantile_loss.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
validate_regime_features.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
validate_tft_hyperopt.rs
feat(ml): Complete hyperopt infrastructure + documentation
2025-10-29 19:52:21 +01:00
validate_tft_int8_accuracy.rs
feat(wave1-2): Complete multi-model training architecture + TLI commands
2025-10-22 20:50:43 +02:00
validate_wave_c_features_51_150.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
verify_225_feature_values.rs
feat(wave9-11): Complete 225-feature integration and service migration
2025-10-20 21:54:39 +02:00
verify_225_features_wave9.rs
feat(wave9-11): Complete 225-feature integration and service migration
2025-10-20 21:54:39 +02:00
verify_dbn_loader_zero_free.rs
feat(wave9-11): Complete 225-feature integration and service migration
2025-10-20 21:54:39 +02:00
verify_feature_dims.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
verify_grn_weight_init.rs
feat(wave-d): Complete Wave D Phase 6 with 240+ parallel agents
2025-10-19 09:10:55 +02:00
verify_mamba2_dimensions.rs
feat(wave9-11): Complete 225-feature integration and service migration
2025-10-20 21:54:39 +02:00
wave_c_backtest.rs
feat(wave9-11): Complete 225-feature integration and service migration
2025-10-20 21:54:39 +02:00
wave_comparison_full_features.rs
feat(wave9-11): Complete 225-feature integration and service migration
2025-10-20 21:54:39 +02:00
wave_comparison_simple.rs
feat(wave9-11): Complete 225-feature integration and service migration
2025-10-20 21:54:39 +02:00
wave_d_backtest.rs
feat(wave9-11): Complete 225-feature integration and service migration
2025-10-20 21:54:39 +02:00