foxhunt/WAVE9_AGENT5_FEATURE_EXTRACTION_TEST_MAP.md

Wave 9 Agent 5: Feature Extraction Test Infrastructure Map

Mission: Map all tests that validate feature extraction to ensure Wave D changes don't break existing functionality.

Date: 2025-10-20
Status: ✅ COMPLETE
Test Files Identified: 45+ test files with 150+ tests

---

Executive Summary

This report provides a comprehensive map of all feature extraction tests across the Foxhunt codebase. These tests MUST continue passing after any changes to feature extraction logic during Wave 9 integration work.

Critical Test Expectations

Test Category	Feature Count	Key Validation
Wave D Tests	225 features	Indices 201-224 are Wave D regime features
Wave C Tests	201 features	Baseline feature set (pre-Wave D)
Legacy Tests	256 features	Old format (being migrated)
Model Input Tests	225 features	All 4 ML models accept 225-dim input
Performance Tests	N/A	<1ms/bar extraction target

---

1. Core Feature Extraction Tests (ml/tests/)

1.1 Wave D Integration Tests (225 Features)

PRIMARY TEST: integration_wave_d_features.rs

• Purpose: End-to-end validation of 225-feature pipeline
• Key Tests:
  • test_wave_d_configuration_complete() - Validates FeatureConfig::wave_d() reports exactly 225 features
  • test_wave_c_vs_wave_d_feature_diff() - Validates Wave C (201) vs Wave D (225) difference = 24 features
  • test_wave_d_feature_extraction_simulated() - Extracts all 225 features from simulated data
  • test_regime_features_update_on_breaks() - Validates CUSUM features (201-210) respond to structural breaks
  • test_feature_extraction_performance() - Validates <1ms per bar target

Feature Index Expectations:

// From integration_wave_d_features.rs:164-171
if let Some((start, end)) = indices.wave_d_regime {
    assert_eq!(end - start, 24, "Wave D should add exactly 24 features");
    assert_eq!(start, 201, "Wave D features should start at index 201");
    assert_eq!(end, 225, "Wave D features should end at index 225");
}

Wave D Feature Breakdown (indices 201-224):

• CUSUM Statistics (201-210): 10 features

  • 201: cusum_s_plus_normalized
  • 202: cusum_s_minus_normalized
  • 203: cusum_break_indicator (0/1 flag)
  • 204: cusum_direction (+1/-1)
  • 205: cusum_time_since_break
  • 206: cusum_frequency
  • 207: cusum_positive_count
  • 208: cusum_negative_count
  • 209: cusum_intensity
  • 210: cusum_drift_ratio

• ADX & Directional (211-215): 5 features

  • 211: adx (0-100 range)
  • 212: plus_di
  • 213: minus_di
  • 214: dx
  • 215: trend_classification (-1/0/1)

• Transition Probabilities (216-220): 5 features

  • 216: regime_stability [0, 1]
  • 217: most_likely_next_regime (0/1/2)
  • 218: regime_entropy
  • 219: regime_expected_duration
  • 220: regime_change_probability [0, 1]

• Adaptive Strategies (221-224): 4 features

  • 221: position_multiplier [0.5, 1.5]
  • 222: stop_loss_multiplier [1.0, 3.0]
  • 223: regime_conditioned_sharpe
  • 224: risk_budget_utilization [0, 1]

1.2 Real Data Validation Tests (225 Features)

Wave D E2E Tests (Real DBN Data):

1. wave_d_e2e_es_fut_225_features_test.rs - ES.FUT validation (500 bars)

   • Test: test_es_fut_225_feature_extraction()
   • Validates: (500 bars × 225 features) dimensions
   • Data: test_data/real/databento/ES.FUT_ohlcv-1m_2024-01-02.dbn

2. wave_d_e2e_zn_fut_225_features_test.rs - ZN.FUT validation

   • Test: test_zn_fut_225_feature_extraction()
   • Validates: DBN loader configured for 225 features
   • Data: test_data/real/databento/ZN.FUT_ohlcv-1m_*.dbn

3. wave_d_e2e_6e_fut_225_features_test.rs - 6E.FUT validation

   • Test: test_6e_fut_225_feature_extraction()
   • Data: test_data/real/databento/6E.FUT_ohlcv-1m_2024-01-02_to_2024-01-31.dbn

4. wave_d_e2e_nq_fut_225_features_test.rs - NQ.FUT validation

   • Test: test_nq_fut_225_features_full_pipeline()
   • Data: test_data/real/databento/NQ.FUT_ohlcv-1m_2024-01-02.dbn

1.3 ML Model Input Format Tests (225 Features)

FILE: wave_d_ml_model_input_test.rs

• Purpose: Validate all 4 ML models accept 225-feature input
• Key Tests:
  • test_mamba2_input_format_225_features() - [batch=32, seq_len=100, features=225]
  • test_dqn_input_format_225_features() - [batch=64, state_dim=225]
  • test_ppo_input_format_225_features() - observation_space=Box(225,)
  • test_tft_input_format_225_features() - 24 static + 201 time-varying = 225 total
  • test_all_models_accept_225_features() - Integration test for all 4 models
  • test_dbn_loader_225_features() - DbnSequenceLoader produces 225-feature tensors

Critical Constant:

const WAVE_D_FEATURE_COUNT: usize = 225;
const WAVE_C_FEATURE_COUNT: usize = 201;

1.4 Feature Extraction Performance Tests

FILE: performance_regression_tests.rs

• Test: test_feature_extraction_time_regression()
• Target: feature_extraction_time_ms = 5.2ms baseline
• Alert if: >10% regression (>5.7ms)

FILE: wave_d_profiling_test.rs

• Test: test_feature_count_validation()
• Validates: Exactly 225 features per bar
• Line 640: assert_eq!(features.len(), 225, "Expected exactly 225 features");

1.5 Normalization & Edge Case Tests

FILE: wave_d_normalization_integration_test.rs

• Test: test_cusum_normalization()

  • Validates: CUSUM features (201-210) are normalized
  • Assert: assert_eq!(cusum_features.len(), 10, "CUSUM should produce 10 features");

• Test: test_adx_normalization()

  • Validates: ADX features (211-215) are normalized
  • Assert: assert_eq!(adx_features.len(), 5, "ADX should produce 5 features");

FILE: wave_d_edge_cases_test.rs

• Test: test_integration_all_extractors_with_nan_inputs()
• Test: test_integration_all_extractors_with_extreme_values()
• Test: test_integration_cold_start_all_extractors()
• Test: test_integration_zero_volatility_all_extractors()

1.6 Legacy 256-Feature Tests (Being Migrated)

FILE: dbn_256_feature_validation.rs

• Status: Legacy format (pre-Wave D)
• Tests:
  • test_es_fut_256_features() - Line 328: assert_eq!(report.feature_stats.len(), 256);
  • test_6e_fut_256_features() - Line 362: assert_eq!(report.feature_stats.len(), 256);
  • test_zn_fut_256_features() - Line 415: assert_eq!(report.feature_stats.len(), 256);
  • test_nq_fut_256_features() - Similar validation

FILE: test_extract_256_dim_features.rs

• Test: test_extract_256_dim_features() - Line 44: assert_eq!(features[0].len(), 225, (FIXED to 225)
• Test: test_feature_dimensions() - Line 96: assert_eq!(feature_vec.len(), 225, "Wrong feature dimension");

---

2. SharedMLStrategy Tests (common/tests/)

FILE: test_sharedml_225_features.rs

• Purpose: Validate SharedMLStrategy extracts 225 features
• Key Tests:
  • test_sharedml_extracts_225_features() - Line 40: assert_eq!(features.len(), 225,
  • test_feature_extraction_wave_d_breakdown() - Validates Wave A (0-25), Wave B (26-35), Wave C (36-200), Wave D (201-224)

Critical Assertion (Line 101-105):

assert_eq!(
    features.len(),
    225,
    "Expected 225 total features (26 Wave A + 10 Wave B + 165 Wave C + 24 Wave D)"
);

FILE: shared_ml_strategy_integration_test.rs

• Test: test_shared_ml_extract_features()
• Validates: Feature extraction via SharedMLStrategy interface

FILE: ml_strategy_integration_tests.rs

• Test: test_feature_extraction_integration()
• Validates: ML strategy feature extraction pipeline

---

3. Regime-Specific Feature Tests (ml/tests/)

3.1 CUSUM Feature Tests

FILE: regime_cusum_features_test.rs

• Test: test_cusum_features_indices_201_210()
• Assert: assert_eq!(result.len(), 10, "Should return exactly 10 features");
• Validates: Features 201-210 (CUSUM statistics)

3.2 ADX Feature Tests

FILE: adx_features_test.rs

• Test: test_adx_features_extraction()
• Validates: Features 211-215 (ADX & Directional)
• Assert: ADX in range [0, 100]

FILE: regime_adx_features_test.rs

• Test: test_adx_initial_state() - Line 107: assert_eq!(features.bar_count(), 0);
• Test: test_adx_update_after_30_bars() - Line 142: assert_eq!(features.bar_count(), 30);

3.3 Transition Probability Tests

FILE: transition_probability_features_test.rs

• Test: test_all_five_features_together() - Line 237: assert_eq!(result.len(), 5, "Should return exactly 5 features");
• Validates: Features 216-220 (Regime transitions)

Individual Feature Tests:

• test_stability_feature_216() - Regime stability [0, 1]
• test_most_likely_next_regime_feature_217() - Next regime (0/1/2)
• test_shannon_entropy_feature_218() - Regime entropy
• test_expected_duration_feature_219() - Expected duration
• test_change_probability_feature_220() - Change probability [0, 1]

3.4 Adaptive Strategy Feature Tests

FILE: adaptive_es_fut_crisis_scenario_test.rs

• Test: test_adaptive_features_finite_and_bounded()
• Validates: Features 221-224 (Adaptive strategies)

---

4. Data Loader Tests

4.1 DBN Sequence Loader Tests

FILE: test_dbn_sequence_256_features.rs

• Test: test_feature_dimension_256() - Line 128: assert_eq!(nan_count, 0, "Found {} NaN values in features", nan_count);
• Test: test_extract_features_dimension()

FILE: dbn_feature_config_test.rs

• Test: test_wave_a_26_features() - Line 17: assert_eq!(loader.feature_config.feature_count(), 26);
• Test: test_wave_b_36_features() - Line 30: assert_eq!(loader.feature_config.feature_count(), 36);
• Test: test_wave_c_65plus_features() - Line 43: assert!(loader.d_model >= 65, "Wave C should have 65+ features");
• Test: test_feature_config_counts() - Line 69-72:

  assert_eq!(wave_a.feature_count(), 26, "Wave A should have 26 features");
  assert_eq!(wave_b.feature_count(), 36, "Wave B should have 36 features");
  

4.2 Feature Cache Tests

FILE: test_feature_cache_service.rs

• Test: test_feature_extraction_validation() - Line 153: assert_eq!(features.len(), 50);
• Test: test_feature_matrix_validation() - Line 228: assert_eq!(matrix.feature_dim, 15);

FILE: feature_cache_tests.rs

• Test: test_extract_256_dim_features()
• Test: test_feature_dimensions()
• Test: test_parquet_read_features()

---

5. Service Integration Tests

5.1 ML Training Service Tests

FILE: services/ml_training_service/tests/data_loader_integration.rs

• Test: test_feature_extraction_dbn()
• Validates: Feature extraction from DBN files

5.2 Trading Service Tests

FILE: services/trading_service/tests/feature_extraction_test.rs

• Test: test_trading_service_feature_extraction()
• Validates: Trading service can extract features

FILE: services/trading_service/tests/ml_paper_trading_e2e_test.rs

• Test: test_ml_paper_trading_feature_pipeline()
• Validates: End-to-end feature extraction in paper trading

5.3 Backtesting Service Tests

FILE: services/backtesting_service/tests/ml_strategy_backtest_test.rs

• Test: test_backtest_feature_extraction()
• Validates: Feature extraction during backtests

---

6. Multi-Symbol Consistency Tests

FILE: multi_symbol_tests.rs

• Test: test_feature_consistency_across_symbols() - Line 164
  • Validates: ES.FUT, NQ.FUT, 6E.FUT, ZN.FUT all produce consistent feature dimensions
  • Assert: assert!(has_finite, "{} should have finite features", symbol); (Line 217)

FILE: wave_d_multi_symbol_concurrent_test.rs

• Test: test_feature_consistency_across_threads() - Line 475
  • Validates: Concurrent feature extraction produces identical results
  • Assert: assert_eq!(seq.features_extracted.len(), con.features_extracted.len()); (Line 219)

---

7. Streaming & Real-Time Tests

FILE: wave_d_realtime_streaming_test.rs

• Constant: const FEATURE_COUNT: usize = 225; (Line 53)
• Test: test_realtime_225_feature_streaming()
• Validates: 225 features extracted before next bar arrives

FILE: streaming_pipeline_edge_cases.rs

• Test: test_zero_feature_dimension() - Line 637: assert!(result.is_err(), "Zero feature dimension should be rejected");

---

8. Calibration & Quantization Tests

FILE: calibration_dataset_test.rs

• Test: test_calibration_feature_count() - Line 290
• Assert: assert!(dataset.feature_count > 0, "Should have features"); (Lines 131, 404, 456)

FILE: tft_int8_calibration_dataset_test.rs

• Test: test_extract_256_dim_features() - Line 46
• Assert: assert_eq!(feature_vec.len(), 60 * 256, "Feature vector size mismatch"); (Line 65)
• Assert: assert!(feature_vec.iter().all(|x| x.is_finite()), "Features contain NaN or Inf"); (Line 74)

---

9. Meta-Labeling & TFT Tests

FILE: meta_labeling_primary_test.rs

• Test: test_feature_extraction_integration() - Line 160
• Assert: assert_eq!(feature_vectors[0].len(), 225); (Line 169)
• Assert: assert_eq!(expected, 225); (Line 334)

FILE: tft_tests.rs

• Test: test_variable_selection_feature_importance() - Line 215
• Assert: assert_eq!(top_features.len(), 3); (Line 234)

---

10. Microstructure Feature Tests

FILE: microstructure_tests.rs

• Test: test_microstructure_integration_256_features() - Line 305
• Assert: assert_eq!(features[0].len(), 225); (Line 327)
• Assert: assert_eq!(features.len(), 50); // 100 bars - 50 warmup (Line 326)

FILE: ml_readiness_validation_tests.rs

• Test: test_feature_extraction() - Line 65
• Assert: assert_eq!(features.prices.len(), bars.len(), "Feature count mismatch"); (Line 72)

---

Critical Test Expectations Summary

Feature Count Assertions (MUST PASS)

Test File	Line	Assertion	Feature Count
integration_wave_d_features.rs	82	assert_eq!(config.feature_count(), WAVE_D_FEATURE_COUNT, ...)	225
integration_wave_d_features.rs	171	assert_eq!(end, 225, "Wave D features should end at index 225")	225
test_sharedml_225_features.rs	40	assert_eq!(features.len(), 225, ...)	225
wave_d_ml_model_input_test.rs	91	assert_eq!(dims[2], WAVE_D_FEATURE_COUNT, ...)	225
wave_d_profiling_test.rs	640	assert_eq!(features.len(), 225, "Expected exactly 225 features")	225
meta_labeling_primary_test.rs	169	assert_eq!(feature_vectors[0].len(), 225)	225
microstructure_tests.rs	327	assert_eq!(features[0].len(), 225)	225
test_extract_256_dim_features.rs	96	assert_eq!(feature_vec.len(), 225, "Wrong feature dimension")	225

Wave D Feature Index Ranges (MUST VALIDATE)

Feature Group	Index Range	Feature Count	Test Validation
CUSUM Statistics	201-210	10	regime_cusum_features_test.rs:36
ADX & Directional	211-215	5	adx_features_test.rs:96
Transition Probabilities	216-220	5	transition_probability_features_test.rs:237
Adaptive Strategies	221-224	4	integration_wave_d_features.rs:222

Performance Targets (MUST MEET)

Metric	Target	Test File	Line
Feature extraction time	<1ms per bar	integration_wave_d_features.rs	385
Feature extraction time (baseline)	5.2ms	performance_regression_tests.rs	87
Feature extraction time (alert)	<5.7ms (10% regression)	performance_regression_tests.rs	292

Data Quality Checks (MUST PASS)

Check	Test File	Line
No NaN values	integration_wave_d_features.rs	436
No Inf values	integration_wave_d_features.rs	437
All finite values	dbn_256_feature_validation.rs	565
Feature ranges [-5, +5]	integration_wave_d_features.rs	466
ADX range [0, 100]	adx_features_test.rs	96

---

Test Execution Commands

Run All Wave D Feature Tests

cargo test -p ml --test integration_wave_d_features
cargo test -p ml --test wave_d_ml_model_input_test
cargo test -p ml --test wave_d_e2e_es_fut_225_features_test
cargo test -p ml --test wave_d_e2e_zn_fut_225_features_test
cargo test -p ml --test wave_d_e2e_6e_fut_225_features_test
cargo test -p ml --test wave_d_e2e_nq_fut_225_features_test

Run SharedML Tests

cargo test -p common --test test_sharedml_225_features
cargo test -p common --test shared_ml_strategy_integration_test

Run Regime Feature Tests

cargo test -p ml --test regime_cusum_features_test
cargo test -p ml --test adx_features_test
cargo test -p ml --test transition_probability_features_test

Run Performance Regression Tests

cargo test -p ml --test performance_regression_tests
cargo test -p ml --test wave_d_profiling_test

Run Full Test Suite (All Feature Tests)

cargo test --workspace -- feature_extraction
cargo test --workspace -- 225_features
cargo test --workspace -- wave_d

---

Impact Analysis: Changes to Feature Extraction

High-Risk Changes (Will Break Many Tests)

1. Changing feature count (201 → 225 or 225 → X)

   • Breaks: 30+ tests with hardcoded assert_eq!(features.len(), 225)
   • Fix: Update WAVE_D_FEATURE_COUNT constant + all assertions

2. Changing feature indices (e.g., moving CUSUM from 201-210 to 210-219)

   • Breaks: All Wave D feature validation tests
   • Fix: Update FeatureConfig::feature_indices() + all index assertions

3. Changing feature value ranges (e.g., ADX from [0, 100] to [-1, 1])

   • Breaks: All normalization tests
   • Fix: Update range assertions in validation tests

Medium-Risk Changes (Will Break Some Tests)

1. Adding new Wave D features (225 → 230)

   • Breaks: Feature count assertions (30+ tests)
   • Fix: Update WAVE_D_FEATURE_COUNT constant

2. Changing feature normalization (e.g., z-score to min-max)

   • Breaks: Normalization validation tests (10+ tests)
   • Fix: Update expected value ranges

3. Changing warmup period (currently 50 bars)

   • Breaks: Feature vector count assertions
   • Fix: Update expected_vectors = total_bars - warmup logic

Low-Risk Changes (Should Not Break Tests)

1. Performance optimizations (as long as output is identical)

   • Should pass: All feature extraction tests
   • May fail: Performance regression tests (if slower)

2. Refactoring extraction code (no behavioral changes)

   • Should pass: All tests (if truly behavior-preserving)

3. Adding new tests (no changes to existing code)

   • Should pass: All existing tests

---

Regression Prevention Checklist

Before merging any feature extraction changes, verify:

• All 225-feature tests pass (30+ tests)
• All Wave D E2E tests pass (4 symbols: ES.FUT, NQ.FUT, 6E.FUT, ZN.FUT)
• All ML model input tests pass (4 models: MAMBA-2, DQN, PPO, TFT)
• SharedMLStrategy tests pass (2+ tests)
• All regime feature tests pass (CUSUM, ADX, Transitions, Adaptive)
• Performance regression tests pass (<10% slowdown)
• No NaN/Inf values in extracted features
• Feature dimensions match expected ranges
• Multi-symbol consistency tests pass

---

Test Files Reference (45 Files)

ml/tests/ (35 files)

1. integration_wave_d_features.rs ⭐ PRIMARY
2. wave_d_ml_model_input_test.rs ⭐ CRITICAL
3. wave_d_e2e_es_fut_225_features_test.rs ⭐ E2E
4. wave_d_e2e_zn_fut_225_features_test.rs ⭐ E2E
5. wave_d_e2e_6e_fut_225_features_test.rs ⭐ E2E
6. wave_d_e2e_nq_fut_225_features_test.rs ⭐ E2E
7. wave_d_profiling_test.rs
8. wave_d_realtime_streaming_test.rs
9. wave_d_normalization_integration_test.rs
10. wave_d_edge_cases_test.rs
11. wave_d_multi_symbol_concurrent_test.rs
12. wave_d_latency_profiling_test.rs
13. regime_cusum_features_test.rs
14. adx_features_test.rs
15. regime_adx_features_test.rs
16. transition_probability_features_test.rs
17. adaptive_es_fut_crisis_scenario_test.rs
18. dbn_256_feature_validation.rs (LEGACY)
19. test_extract_256_dim_features.rs (LEGACY)
20. test_dbn_sequence_256_features.rs
21. dbn_feature_config_test.rs
22. test_feature_cache_service.rs
23. feature_cache_tests.rs
24. meta_labeling_primary_test.rs
25. ml_readiness_validation_tests.rs
26. performance_regression_tests.rs
27. multi_symbol_tests.rs
28. microstructure_tests.rs
29. streaming_pipeline_edge_cases.rs
30. calibration_dataset_test.rs
31. tft_int8_calibration_dataset_test.rs
32. tft_tests.rs
33. ensemble_integration_tests.rs
34. e2e_ensemble_integration.rs
35. model_validation_comprehensive.rs

common/tests/ (3 files)

1. test_sharedml_225_features.rs ⭐ PRIMARY
2. shared_ml_strategy_integration_test.rs
3. ml_strategy_integration_tests.rs

services/*/tests/ (5 files)

1. services/ml_training_service/tests/data_loader_integration.rs
2. services/trading_service/tests/feature_extraction_test.rs
3. services/trading_service/tests/ml_paper_trading_e2e_test.rs
4. services/backtesting_service/tests/ml_strategy_backtest_test.rs
5. data/tests/pipeline_integration.rs

adaptive-strategy/tests/ (1 file)

1. regime_transition_tests.rs

E2E tests/ (1 file)

1. tests/e2e/tests/ml_model_integration_tests.rs

---

Recommendations for Wave 9 Integration

1. Run Test Suite Before Changes

cargo test --workspace -- feature_extraction > baseline_results.txt
cargo test --workspace -- 225_features >> baseline_results.txt
cargo test --workspace -- wave_d >> baseline_results.txt

2. After Changes, Run Regression Tests

cargo test --workspace -- feature_extraction > new_results.txt
diff baseline_results.txt new_results.txt

3. Focus on Critical Tests First

• Run integration_wave_d_features.rs (PRIMARY test)
• Run test_sharedml_225_features.rs (SharedML validation)
• Run wave_d_ml_model_input_test.rs (ML model compatibility)
• Run all 4 E2E tests (ES.FUT, NQ.FUT, 6E.FUT, ZN.FUT)

4. Monitor Performance Metrics

• Track feature extraction time (target: <1ms/bar)
• Monitor memory usage (target: <8KB/symbol)
• Validate throughput (target: 1000+ bars/sec)

5. Validate Data Quality

• Zero NaN values (strict requirement)
• Zero Inf values (strict requirement)
• Feature ranges within expected bounds
• All features are finite

---

Status: ✅ READY FOR WAVE 9 INTEGRATION

This comprehensive test map provides:

• 45+ test files covering feature extraction
• 150+ individual tests validating 225-feature pipeline
• 30+ critical assertions on feature count (225)
• 4 E2E tests with real DBN data (ES.FUT, NQ.FUT, 6E.FUT, ZN.FUT)
• Clear regression prevention checklist
• Performance targets and validation commands

All tests are documented and ready for Wave 9 integration work.

---

End of Report