# Wave 9 Agent 5: Feature Extraction Test Infrastructure Map **Mission**: Map all tests that validate feature extraction to ensure Wave D changes don't break existing functionality. **Date**: 2025-10-20 **Status**: ✅ COMPLETE **Test Files Identified**: 45+ test files with 150+ tests --- ## Executive Summary This report provides a comprehensive map of all feature extraction tests across the Foxhunt codebase. These tests MUST continue passing after any changes to feature extraction logic during Wave 9 integration work. ### Critical Test Expectations | Test Category | Feature Count | Key Validation | |---|---|---| | **Wave D Tests** | 225 features | Indices 201-224 are Wave D regime features | | **Wave C Tests** | 201 features | Baseline feature set (pre-Wave D) | | **Legacy Tests** | 256 features | Old format (being migrated) | | **Model Input Tests** | 225 features | All 4 ML models accept 225-dim input | | **Performance Tests** | N/A | <1ms/bar extraction target | --- ## 1. Core Feature Extraction Tests (ml/tests/) ### 1.1 Wave D Integration Tests (225 Features) **PRIMARY TEST: `integration_wave_d_features.rs`** - **Purpose**: End-to-end validation of 225-feature pipeline - **Key Tests**: - `test_wave_d_configuration_complete()` - Validates FeatureConfig::wave_d() reports exactly 225 features - `test_wave_c_vs_wave_d_feature_diff()` - Validates Wave C (201) vs Wave D (225) difference = 24 features - `test_wave_d_feature_extraction_simulated()` - Extracts all 225 features from simulated data - `test_regime_features_update_on_breaks()` - Validates CUSUM features (201-210) respond to structural breaks - `test_feature_extraction_performance()` - Validates <1ms per bar target **Feature Index Expectations**: ```rust // From integration_wave_d_features.rs:164-171 if let Some((start, end)) = indices.wave_d_regime { assert_eq!(end - start, 24, "Wave D should add exactly 24 features"); assert_eq!(start, 201, "Wave D features should start at index 201"); assert_eq!(end, 225, "Wave D features should end at index 225"); } ``` **Wave D Feature Breakdown** (indices 201-224): - **CUSUM Statistics** (201-210): 10 features - 201: cusum_s_plus_normalized - 202: cusum_s_minus_normalized - 203: cusum_break_indicator (0/1 flag) - 204: cusum_direction (+1/-1) - 205: cusum_time_since_break - 206: cusum_frequency - 207: cusum_positive_count - 208: cusum_negative_count - 209: cusum_intensity - 210: cusum_drift_ratio - **ADX & Directional** (211-215): 5 features - 211: adx (0-100 range) - 212: plus_di - 213: minus_di - 214: dx - 215: trend_classification (-1/0/1) - **Transition Probabilities** (216-220): 5 features - 216: regime_stability [0, 1] - 217: most_likely_next_regime (0/1/2) - 218: regime_entropy - 219: regime_expected_duration - 220: regime_change_probability [0, 1] - **Adaptive Strategies** (221-224): 4 features - 221: position_multiplier [0.5, 1.5] - 222: stop_loss_multiplier [1.0, 3.0] - 223: regime_conditioned_sharpe - 224: risk_budget_utilization [0, 1] ### 1.2 Real Data Validation Tests (225 Features) **Wave D E2E Tests (Real DBN Data)**: 1. `wave_d_e2e_es_fut_225_features_test.rs` - ES.FUT validation (500 bars) - Test: `test_es_fut_225_feature_extraction()` - Validates: (500 bars × 225 features) dimensions - Data: `test_data/real/databento/ES.FUT_ohlcv-1m_2024-01-02.dbn` 2. `wave_d_e2e_zn_fut_225_features_test.rs` - ZN.FUT validation - Test: `test_zn_fut_225_feature_extraction()` - Validates: DBN loader configured for 225 features - Data: `test_data/real/databento/ZN.FUT_ohlcv-1m_*.dbn` 3. `wave_d_e2e_6e_fut_225_features_test.rs` - 6E.FUT validation - Test: `test_6e_fut_225_feature_extraction()` - Data: `test_data/real/databento/6E.FUT_ohlcv-1m_2024-01-02_to_2024-01-31.dbn` 4. `wave_d_e2e_nq_fut_225_features_test.rs` - NQ.FUT validation - Test: `test_nq_fut_225_features_full_pipeline()` - Data: `test_data/real/databento/NQ.FUT_ohlcv-1m_2024-01-02.dbn` ### 1.3 ML Model Input Format Tests (225 Features) **FILE: `wave_d_ml_model_input_test.rs`** - **Purpose**: Validate all 4 ML models accept 225-feature input - **Key Tests**: - `test_mamba2_input_format_225_features()` - [batch=32, seq_len=100, features=225] - `test_dqn_input_format_225_features()` - [batch=64, state_dim=225] - `test_ppo_input_format_225_features()` - observation_space=Box(225,) - `test_tft_input_format_225_features()` - 24 static + 201 time-varying = 225 total - `test_all_models_accept_225_features()` - Integration test for all 4 models - `test_dbn_loader_225_features()` - DbnSequenceLoader produces 225-feature tensors **Critical Constant**: ```rust const WAVE_D_FEATURE_COUNT: usize = 225; const WAVE_C_FEATURE_COUNT: usize = 201; ``` ### 1.4 Feature Extraction Performance Tests **FILE: `performance_regression_tests.rs`** - Test: `test_feature_extraction_time_regression()` - Target: `feature_extraction_time_ms = 5.2ms` baseline - Alert if: >10% regression (>5.7ms) **FILE: `wave_d_profiling_test.rs`** - Test: `test_feature_count_validation()` - Validates: Exactly 225 features per bar - Line 640: `assert_eq!(features.len(), 225, "Expected exactly 225 features");` ### 1.5 Normalization & Edge Case Tests **FILE: `wave_d_normalization_integration_test.rs`** - Test: `test_cusum_normalization()` - Validates: CUSUM features (201-210) are normalized - Assert: `assert_eq!(cusum_features.len(), 10, "CUSUM should produce 10 features");` - Test: `test_adx_normalization()` - Validates: ADX features (211-215) are normalized - Assert: `assert_eq!(adx_features.len(), 5, "ADX should produce 5 features");` **FILE: `wave_d_edge_cases_test.rs`** - Test: `test_integration_all_extractors_with_nan_inputs()` - Test: `test_integration_all_extractors_with_extreme_values()` - Test: `test_integration_cold_start_all_extractors()` - Test: `test_integration_zero_volatility_all_extractors()` ### 1.6 Legacy 256-Feature Tests (Being Migrated) **FILE: `dbn_256_feature_validation.rs`** - **Status**: Legacy format (pre-Wave D) - Tests: - `test_es_fut_256_features()` - Line 328: `assert_eq!(report.feature_stats.len(), 256);` - `test_6e_fut_256_features()` - Line 362: `assert_eq!(report.feature_stats.len(), 256);` - `test_zn_fut_256_features()` - Line 415: `assert_eq!(report.feature_stats.len(), 256);` - `test_nq_fut_256_features()` - Similar validation **FILE: `test_extract_256_dim_features.rs`** - Test: `test_extract_256_dim_features()` - Line 44: `assert_eq!(features[0].len(), 225,` (FIXED to 225) - Test: `test_feature_dimensions()` - Line 96: `assert_eq!(feature_vec.len(), 225, "Wrong feature dimension");` --- ## 2. SharedMLStrategy Tests (common/tests/) **FILE: `test_sharedml_225_features.rs`** - **Purpose**: Validate SharedMLStrategy extracts 225 features - **Key Tests**: - `test_sharedml_extracts_225_features()` - Line 40: `assert_eq!(features.len(), 225,` - `test_feature_extraction_wave_d_breakdown()` - Validates Wave A (0-25), Wave B (26-35), Wave C (36-200), Wave D (201-224) **Critical Assertion** (Line 101-105): ```rust assert_eq!( features.len(), 225, "Expected 225 total features (26 Wave A + 10 Wave B + 165 Wave C + 24 Wave D)" ); ``` **FILE: `shared_ml_strategy_integration_test.rs`** - Test: `test_shared_ml_extract_features()` - Validates: Feature extraction via SharedMLStrategy interface **FILE: `ml_strategy_integration_tests.rs`** - Test: `test_feature_extraction_integration()` - Validates: ML strategy feature extraction pipeline --- ## 3. Regime-Specific Feature Tests (ml/tests/) ### 3.1 CUSUM Feature Tests **FILE: `regime_cusum_features_test.rs`** - Test: `test_cusum_features_indices_201_210()` - Assert: `assert_eq!(result.len(), 10, "Should return exactly 10 features");` - Validates: Features 201-210 (CUSUM statistics) ### 3.2 ADX Feature Tests **FILE: `adx_features_test.rs`** - Test: `test_adx_features_extraction()` - Validates: Features 211-215 (ADX & Directional) - Assert: ADX in range [0, 100] **FILE: `regime_adx_features_test.rs`** - Test: `test_adx_initial_state()` - Line 107: `assert_eq!(features.bar_count(), 0);` - Test: `test_adx_update_after_30_bars()` - Line 142: `assert_eq!(features.bar_count(), 30);` ### 3.3 Transition Probability Tests **FILE: `transition_probability_features_test.rs`** - Test: `test_all_five_features_together()` - Line 237: `assert_eq!(result.len(), 5, "Should return exactly 5 features");` - Validates: Features 216-220 (Regime transitions) **Individual Feature Tests**: - `test_stability_feature_216()` - Regime stability [0, 1] - `test_most_likely_next_regime_feature_217()` - Next regime (0/1/2) - `test_shannon_entropy_feature_218()` - Regime entropy - `test_expected_duration_feature_219()` - Expected duration - `test_change_probability_feature_220()` - Change probability [0, 1] ### 3.4 Adaptive Strategy Feature Tests **FILE: `adaptive_es_fut_crisis_scenario_test.rs`** - Test: `test_adaptive_features_finite_and_bounded()` - Validates: Features 221-224 (Adaptive strategies) --- ## 4. Data Loader Tests ### 4.1 DBN Sequence Loader Tests **FILE: `test_dbn_sequence_256_features.rs`** - Test: `test_feature_dimension_256()` - Line 128: `assert_eq!(nan_count, 0, "Found {} NaN values in features", nan_count);` - Test: `test_extract_features_dimension()` **FILE: `dbn_feature_config_test.rs`** - Test: `test_wave_a_26_features()` - Line 17: `assert_eq!(loader.feature_config.feature_count(), 26);` - Test: `test_wave_b_36_features()` - Line 30: `assert_eq!(loader.feature_config.feature_count(), 36);` - Test: `test_wave_c_65plus_features()` - Line 43: `assert!(loader.d_model >= 65, "Wave C should have 65+ features");` - Test: `test_feature_config_counts()` - Line 69-72: ```rust assert_eq!(wave_a.feature_count(), 26, "Wave A should have 26 features"); assert_eq!(wave_b.feature_count(), 36, "Wave B should have 36 features"); ``` ### 4.2 Feature Cache Tests **FILE: `test_feature_cache_service.rs`** - Test: `test_feature_extraction_validation()` - Line 153: `assert_eq!(features.len(), 50);` - Test: `test_feature_matrix_validation()` - Line 228: `assert_eq!(matrix.feature_dim, 15);` **FILE: `feature_cache_tests.rs`** - Test: `test_extract_256_dim_features()` - Test: `test_feature_dimensions()` - Test: `test_parquet_read_features()` --- ## 5. Service Integration Tests ### 5.1 ML Training Service Tests **FILE: `services/ml_training_service/tests/data_loader_integration.rs`** - Test: `test_feature_extraction_dbn()` - Validates: Feature extraction from DBN files ### 5.2 Trading Service Tests **FILE: `services/trading_service/tests/feature_extraction_test.rs`** - Test: `test_trading_service_feature_extraction()` - Validates: Trading service can extract features **FILE: `services/trading_service/tests/ml_paper_trading_e2e_test.rs`** - Test: `test_ml_paper_trading_feature_pipeline()` - Validates: End-to-end feature extraction in paper trading ### 5.3 Backtesting Service Tests **FILE: `services/backtesting_service/tests/ml_strategy_backtest_test.rs`** - Test: `test_backtest_feature_extraction()` - Validates: Feature extraction during backtests --- ## 6. Multi-Symbol Consistency Tests **FILE: `multi_symbol_tests.rs`** - Test: `test_feature_consistency_across_symbols()` - Line 164 - Validates: ES.FUT, NQ.FUT, 6E.FUT, ZN.FUT all produce consistent feature dimensions - Assert: `assert!(has_finite, "{} should have finite features", symbol);` (Line 217) **FILE: `wave_d_multi_symbol_concurrent_test.rs`** - Test: `test_feature_consistency_across_threads()` - Line 475 - Validates: Concurrent feature extraction produces identical results - Assert: `assert_eq!(seq.features_extracted.len(), con.features_extracted.len());` (Line 219) --- ## 7. Streaming & Real-Time Tests **FILE: `wave_d_realtime_streaming_test.rs`** - Constant: `const FEATURE_COUNT: usize = 225;` (Line 53) - Test: `test_realtime_225_feature_streaming()` - Validates: 225 features extracted before next bar arrives **FILE: `streaming_pipeline_edge_cases.rs`** - Test: `test_zero_feature_dimension()` - Line 637: `assert!(result.is_err(), "Zero feature dimension should be rejected");` --- ## 8. Calibration & Quantization Tests **FILE: `calibration_dataset_test.rs`** - Test: `test_calibration_feature_count()` - Line 290 - Assert: `assert!(dataset.feature_count > 0, "Should have features");` (Lines 131, 404, 456) **FILE: `tft_int8_calibration_dataset_test.rs`** - Test: `test_extract_256_dim_features()` - Line 46 - Assert: `assert_eq!(feature_vec.len(), 60 * 256, "Feature vector size mismatch");` (Line 65) - Assert: `assert!(feature_vec.iter().all(|x| x.is_finite()), "Features contain NaN or Inf");` (Line 74) --- ## 9. Meta-Labeling & TFT Tests **FILE: `meta_labeling_primary_test.rs`** - Test: `test_feature_extraction_integration()` - Line 160 - Assert: `assert_eq!(feature_vectors[0].len(), 225);` (Line 169) - Assert: `assert_eq!(expected, 225);` (Line 334) **FILE: `tft_tests.rs`** - Test: `test_variable_selection_feature_importance()` - Line 215 - Assert: `assert_eq!(top_features.len(), 3);` (Line 234) --- ## 10. Microstructure Feature Tests **FILE: `microstructure_tests.rs`** - Test: `test_microstructure_integration_256_features()` - Line 305 - Assert: `assert_eq!(features[0].len(), 225);` (Line 327) - Assert: `assert_eq!(features.len(), 50); // 100 bars - 50 warmup` (Line 326) **FILE: `ml_readiness_validation_tests.rs`** - Test: `test_feature_extraction()` - Line 65 - Assert: `assert_eq!(features.prices.len(), bars.len(), "Feature count mismatch");` (Line 72) --- ## Critical Test Expectations Summary ### Feature Count Assertions (MUST PASS) | Test File | Line | Assertion | Feature Count | |---|---|---|---| | integration_wave_d_features.rs | 82 | `assert_eq!(config.feature_count(), WAVE_D_FEATURE_COUNT, ...)` | 225 | | integration_wave_d_features.rs | 171 | `assert_eq!(end, 225, "Wave D features should end at index 225")` | 225 | | test_sharedml_225_features.rs | 40 | `assert_eq!(features.len(), 225, ...)` | 225 | | wave_d_ml_model_input_test.rs | 91 | `assert_eq!(dims[2], WAVE_D_FEATURE_COUNT, ...)` | 225 | | wave_d_profiling_test.rs | 640 | `assert_eq!(features.len(), 225, "Expected exactly 225 features")` | 225 | | meta_labeling_primary_test.rs | 169 | `assert_eq!(feature_vectors[0].len(), 225)` | 225 | | microstructure_tests.rs | 327 | `assert_eq!(features[0].len(), 225)` | 225 | | test_extract_256_dim_features.rs | 96 | `assert_eq!(feature_vec.len(), 225, "Wrong feature dimension")` | 225 | ### Wave D Feature Index Ranges (MUST VALIDATE) | Feature Group | Index Range | Feature Count | Test Validation | |---|---|---|---| | CUSUM Statistics | 201-210 | 10 | `regime_cusum_features_test.rs:36` | | ADX & Directional | 211-215 | 5 | `adx_features_test.rs:96` | | Transition Probabilities | 216-220 | 5 | `transition_probability_features_test.rs:237` | | Adaptive Strategies | 221-224 | 4 | `integration_wave_d_features.rs:222` | ### Performance Targets (MUST MEET) | Metric | Target | Test File | Line | |---|---|---|---| | Feature extraction time | <1ms per bar | integration_wave_d_features.rs | 385 | | Feature extraction time (baseline) | 5.2ms | performance_regression_tests.rs | 87 | | Feature extraction time (alert) | <5.7ms (10% regression) | performance_regression_tests.rs | 292 | ### Data Quality Checks (MUST PASS) | Check | Test File | Line | |---|---|---| | No NaN values | integration_wave_d_features.rs | 436 | | No Inf values | integration_wave_d_features.rs | 437 | | All finite values | dbn_256_feature_validation.rs | 565 | | Feature ranges [-5, +5] | integration_wave_d_features.rs | 466 | | ADX range [0, 100] | adx_features_test.rs | 96 | --- ## Test Execution Commands ### Run All Wave D Feature Tests ```bash cargo test -p ml --test integration_wave_d_features cargo test -p ml --test wave_d_ml_model_input_test cargo test -p ml --test wave_d_e2e_es_fut_225_features_test cargo test -p ml --test wave_d_e2e_zn_fut_225_features_test cargo test -p ml --test wave_d_e2e_6e_fut_225_features_test cargo test -p ml --test wave_d_e2e_nq_fut_225_features_test ``` ### Run SharedML Tests ```bash cargo test -p common --test test_sharedml_225_features cargo test -p common --test shared_ml_strategy_integration_test ``` ### Run Regime Feature Tests ```bash cargo test -p ml --test regime_cusum_features_test cargo test -p ml --test adx_features_test cargo test -p ml --test transition_probability_features_test ``` ### Run Performance Regression Tests ```bash cargo test -p ml --test performance_regression_tests cargo test -p ml --test wave_d_profiling_test ``` ### Run Full Test Suite (All Feature Tests) ```bash cargo test --workspace -- feature_extraction cargo test --workspace -- 225_features cargo test --workspace -- wave_d ``` --- ## Impact Analysis: Changes to Feature Extraction ### High-Risk Changes (Will Break Many Tests) 1. **Changing feature count** (201 → 225 or 225 → X) - Breaks: 30+ tests with hardcoded `assert_eq!(features.len(), 225)` - Fix: Update `WAVE_D_FEATURE_COUNT` constant + all assertions 2. **Changing feature indices** (e.g., moving CUSUM from 201-210 to 210-219) - Breaks: All Wave D feature validation tests - Fix: Update `FeatureConfig::feature_indices()` + all index assertions 3. **Changing feature value ranges** (e.g., ADX from [0, 100] to [-1, 1]) - Breaks: All normalization tests - Fix: Update range assertions in validation tests ### Medium-Risk Changes (Will Break Some Tests) 1. **Adding new Wave D features** (225 → 230) - Breaks: Feature count assertions (30+ tests) - Fix: Update `WAVE_D_FEATURE_COUNT` constant 2. **Changing feature normalization** (e.g., z-score to min-max) - Breaks: Normalization validation tests (10+ tests) - Fix: Update expected value ranges 3. **Changing warmup period** (currently 50 bars) - Breaks: Feature vector count assertions - Fix: Update `expected_vectors = total_bars - warmup` logic ### Low-Risk Changes (Should Not Break Tests) 1. **Performance optimizations** (as long as output is identical) - Should pass: All feature extraction tests - May fail: Performance regression tests (if slower) 2. **Refactoring extraction code** (no behavioral changes) - Should pass: All tests (if truly behavior-preserving) 3. **Adding new tests** (no changes to existing code) - Should pass: All existing tests --- ## Regression Prevention Checklist Before merging any feature extraction changes, verify: - [ ] All 225-feature tests pass (30+ tests) - [ ] All Wave D E2E tests pass (4 symbols: ES.FUT, NQ.FUT, 6E.FUT, ZN.FUT) - [ ] All ML model input tests pass (4 models: MAMBA-2, DQN, PPO, TFT) - [ ] SharedMLStrategy tests pass (2+ tests) - [ ] All regime feature tests pass (CUSUM, ADX, Transitions, Adaptive) - [ ] Performance regression tests pass (<10% slowdown) - [ ] No NaN/Inf values in extracted features - [ ] Feature dimensions match expected ranges - [ ] Multi-symbol consistency tests pass --- ## Test Files Reference (45 Files) ### ml/tests/ (35 files) 1. integration_wave_d_features.rs ⭐ PRIMARY 2. wave_d_ml_model_input_test.rs ⭐ CRITICAL 3. wave_d_e2e_es_fut_225_features_test.rs ⭐ E2E 4. wave_d_e2e_zn_fut_225_features_test.rs ⭐ E2E 5. wave_d_e2e_6e_fut_225_features_test.rs ⭐ E2E 6. wave_d_e2e_nq_fut_225_features_test.rs ⭐ E2E 7. wave_d_profiling_test.rs 8. wave_d_realtime_streaming_test.rs 9. wave_d_normalization_integration_test.rs 10. wave_d_edge_cases_test.rs 11. wave_d_multi_symbol_concurrent_test.rs 12. wave_d_latency_profiling_test.rs 13. regime_cusum_features_test.rs 14. adx_features_test.rs 15. regime_adx_features_test.rs 16. transition_probability_features_test.rs 17. adaptive_es_fut_crisis_scenario_test.rs 18. dbn_256_feature_validation.rs (LEGACY) 19. test_extract_256_dim_features.rs (LEGACY) 20. test_dbn_sequence_256_features.rs 21. dbn_feature_config_test.rs 22. test_feature_cache_service.rs 23. feature_cache_tests.rs 24. meta_labeling_primary_test.rs 25. ml_readiness_validation_tests.rs 26. performance_regression_tests.rs 27. multi_symbol_tests.rs 28. microstructure_tests.rs 29. streaming_pipeline_edge_cases.rs 30. calibration_dataset_test.rs 31. tft_int8_calibration_dataset_test.rs 32. tft_tests.rs 33. ensemble_integration_tests.rs 34. e2e_ensemble_integration.rs 35. model_validation_comprehensive.rs ### common/tests/ (3 files) 1. test_sharedml_225_features.rs ⭐ PRIMARY 2. shared_ml_strategy_integration_test.rs 3. ml_strategy_integration_tests.rs ### services/*/tests/ (5 files) 1. services/ml_training_service/tests/data_loader_integration.rs 2. services/trading_service/tests/feature_extraction_test.rs 3. services/trading_service/tests/ml_paper_trading_e2e_test.rs 4. services/backtesting_service/tests/ml_strategy_backtest_test.rs 5. data/tests/pipeline_integration.rs ### adaptive-strategy/tests/ (1 file) 1. regime_transition_tests.rs ### E2E tests/ (1 file) 1. tests/e2e/tests/ml_model_integration_tests.rs --- ## Recommendations for Wave 9 Integration ### 1. Run Test Suite Before Changes ```bash cargo test --workspace -- feature_extraction > baseline_results.txt cargo test --workspace -- 225_features >> baseline_results.txt cargo test --workspace -- wave_d >> baseline_results.txt ``` ### 2. After Changes, Run Regression Tests ```bash cargo test --workspace -- feature_extraction > new_results.txt diff baseline_results.txt new_results.txt ``` ### 3. Focus on Critical Tests First - Run `integration_wave_d_features.rs` (PRIMARY test) - Run `test_sharedml_225_features.rs` (SharedML validation) - Run `wave_d_ml_model_input_test.rs` (ML model compatibility) - Run all 4 E2E tests (ES.FUT, NQ.FUT, 6E.FUT, ZN.FUT) ### 4. Monitor Performance Metrics - Track feature extraction time (target: <1ms/bar) - Monitor memory usage (target: <8KB/symbol) - Validate throughput (target: 1000+ bars/sec) ### 5. Validate Data Quality - Zero NaN values (strict requirement) - Zero Inf values (strict requirement) - Feature ranges within expected bounds - All features are finite --- ## Status: ✅ READY FOR WAVE 9 INTEGRATION This comprehensive test map provides: - **45+ test files** covering feature extraction - **150+ individual tests** validating 225-feature pipeline - **30+ critical assertions** on feature count (225) - **4 E2E tests** with real DBN data (ES.FUT, NQ.FUT, 6E.FUT, ZN.FUT) - **Clear regression prevention checklist** - **Performance targets** and validation commands All tests are documented and ready for Wave 9 integration work. --- **End of Report**