foxhunt/WAVE_D_UTILITIES_QUICK_REFERENCE.txt

WAVE D REGIME DETECTION: REUSABLE UTILITIES - QUICK SUMMARY
============================================================

INVESTIGATION FINDINGS: 50+ PRODUCTION-READY FUNCTIONS ACROSS 14 MODULES

===============================================================================
CRITICAL UTILITIES AVAILABLE FOR WAVE D IMPLEMENTATION
===============================================================================

1. AUTOCORRELATION IMPLEMENTATIONS
   - Location: /home/jgrusewski/Work/foxhunt/ml/src/features/statistical_features.rs
   - Function: compute_autocorrelation(bars, period) -> f64
   - Use: Detect mean reversion regimes (lag-1 ACF)
   - Performance: <50μs

2. VOLATILITY CALCULATIONS (3 Estimators)
   - Location: /home/jgrusewski/Work/foxhunt/ml/src/features/price_features.rs
   - Functions:
     * compute_parkinson_volatility(bar) -> f64    [Range-based]
     * compute_garman_klass_volatility(bar) -> f64 [OHLC-based]
     * compute_yang_zhang_volatility(bars) -> f64  [Gap + Intraday]
   - Use: Volatility regime classification (High/Low/Extreme)
   - Performance: <100μs for all 3

3. ROLLING STATISTICS (O(1) Amortized)
   - Location: /home/jgrusewski/Work/foxhunt/ml/src/features/statistical_features.rs
   - Functions:
     * compute_rolling_mean(bars, period) -> f64
     * compute_rolling_std(bars, period) -> f64
     * compute_rolling_min(bars, period) -> f64
     * compute_rolling_max(bars, period) -> f64
   - Key Classes: MonotonicDeque (min/max), WelfordState (variance)
   - Performance: <100μs

4. EWMA ADAPTIVE THRESHOLDING
   - Location: /home/jgrusewski/Work/foxhunt/ml/src/features/ewma.rs
   - Classes:
     * EWMACalculator: Single EWMA with α = 2/(span+1)
     * AdaptiveThreshold: Dual EWMA (mean + variance)
   - Use: Detect structural breaks in mean/variance
   - Performance: O(1) per update, 24 bytes memory

5. CORRELATION & COVARIANCE
   - Location: /home/jgrusewski/Work/foxhunt/ml/src/features/ (multiple files)
   - compute_autocorrelation() in statistical_features.rs
   - compute_volume_price_correlation() in volume_features.rs
   - compute_range_volume_correlation() in volume_features.rs
   - compute_correlation(x, y) -> f64 [Pearson, generic]
   - Use: Detect correlation breaks (crisis/recovery regimes)

6. FEATURE NORMALIZATION
   - Location: /home/jgrusewski/Work/foxhunt/ml/src/features/normalization.rs
   - Classes:
     * RollingZScore: Z-score [-1, 1]
     * RollingPercentileRank: Percentile [0, 1]
     * LogZScoreNormalizer: Log + Z-score for skewed data
   - Use: Normalize regime features for ML models

7. MICROSTRUCTURE INDICATORS
   - Location: /home/jgrusewski/Work/foxhunt/ml/src/features/microstructure_features.rs
   - Functions:
     * Roll Measure spread estimator
     * Corwin-Schultz spread estimator
     * Amihud illiquidity metric (crisis detector)
     * Buy/Sell imbalance
     * Kyle's Lambda (market impact)
     * Variance ratio (mean reversion detector)
   - Use: Liquidity regimes (Normal/Illiquid/Crisis)
   - Performance: <200μs for all

8. PRICE-BASED STATISTICAL FEATURES
   - Location: /home/jgrusewski/Work/foxhunt/ml/src/features/price_features.rs
   - Functions:
     * compute_hurst_exponent(bars, period) -> f64
     * compute_rolling_skewness(bars, period) -> f64
     * compute_rolling_kurtosis(bars, period) -> f64
   - Use: Hurst → trending/ranging, Skew → Bull/Bear, Kurt → Tail risk

9. REGIME DETECTION FRAMEWORK (Existing Infrastructure)
   - Location: /home/jgrusewski/Work/foxhunt/adaptive-strategy/src/regime/mod.rs
   - Types:
     * enum MarketRegime { Normal, Trending, Bull, Bear, Crisis, ... }
     * trait RegimeDetectionModel { detect_regime(...), train(...) }
     * RegimeTransitionTracker: Tracks regime history + transition matrix
     * RegimePerformanceTracker: Regime-specific performance metrics
   - Use: Regime orchestration, transition tracking

10. VOLUME INDICATORS
    - Location: /home/jgrusewski/Work/foxhunt/ml/src/features/volume_features.rs
    - Functions:
      * compute_vwap(bars) -> f64
      * compute_obv(bars) -> f64
      * compute_obv_momentum(period) -> f64
    - Use: Volume-based regime indicators

11. TECHNICAL INDICATORS (Already Available)
    - Location: /home/jgrusewski/Work/foxhunt/common/src/ml_strategy.rs
    - Available: RSI, MACD, Bollinger Bands, ATR, ADX
    - Use: Ensemble features for regime classification

12. VaR CALCULATOR (Risk Module)
    - Location: /home/jgrusewski/Work/foxhunt/risk/src/var_calculator/
    - Function: calculate_rolling_var(returns, window, confidence_level)
    - Use: Extreme volatility regime detection

13. ML FEATURE EXTRACTION (Full Pipeline)
    - Location: /home/jgrusewski/Work/foxhunt/ml/src/features/extraction.rs
    - Class: UnifiedFeatureExtractor
    - Function: extract_ml_features(bars) -> Vec<FeatureVector>
    - Features: 256-dimensional feature vectors

14. TIME-BASED FEATURES (Correlation Regime)
    - Location: /home/jgrusewski/Work/foxhunt/ml/src/features/time_features.rs
    - Function: correlation_regime() -> f64
    - Use: Intrabar correlation (trending: ~1.0, ranging: ~0.0)

===============================================================================
READY-TO-USE DESIGN PATTERNS
===============================================================================

PATTERN 1: O(1) Amortized Min/Max Tracking
- Use MonotonicDeque structure (statistical_features.rs lines 60-170)
- Replaces O(n) sorting per update with O(1) amortized
- Scales to 1000+ bars efficiently

PATTERN 2: Numerically Stable Variance (Welford's Algorithm)
- Use WelfordState (statistical_features.rs lines 52-115)
- Supports add/remove operations without recomputation
- Prevents overflow on long series (no sum of squares)

PATTERN 3: Dual EWMA Tracking
- EWMACalculator for mean + separate for variance
- Detects both level and volatility shifts
- O(1) per update, ideal for streaming data

PATTERN 4: Safe Numerical Operations
- All functions include NaN/Inf handling
- safe_clip(value, min, max) prevents propagation
- safe_log_return() handles edge cases

===============================================================================
PERFORMANCE BUDGET AVAILABLE FOR WAVE D
===============================================================================

Per-Bar Computation Budget: <500μs

Component Allocations:
- Autocorrelation detection:     <50μs (✓ Available)
- Volatility estimation:         <100μs (✓ Available)
- Rolling statistics:            <100μs (✓ Available)
- Correlation:                   <100μs (✓ Available)
- EWMA updates:                  <10μs (✓ Available)
- CUSUM (new):                   <50μs (Estimate)
- Regime classification (new):   <100μs (Estimate)
Total Available:                 ~500μs ✓

===============================================================================
IMPLEMENTATION STRATEGY FOR WAVE D
===============================================================================

PHASE 1: Structural Break Detection (Agents D1-D4)
- Reuse: EWMACalculator, compute_rolling_std, compute_autocorrelation
- New: CUSUM algorithm (mean/variance/multivariate variants)
- New: Bayesian online changepoint detection

PHASE 2: Regime Classification (Agents D5-D8)
- Reuse: volatility functions, hurst exponent, correlations, amihud
- New: Threshold-based regime classifiers
- New: Multi-feature regime decision logic

PHASE 3: Adaptive Strategies (Agents D9-D12)
- Reuse: RegimeTransitionTracker, RegimePerformanceTracker, calculate_rolling_var
- New: Position sizing by regime
- New: Dynamic stop placement
- New: Strategy switching logic

===============================================================================
KEY FILES TO EXAMINE
===============================================================================

1. Autocorrelation:
   /home/jgrusewski/Work/foxhunt/ml/src/features/statistical_features.rs:330-440

2. Volatility Estimators:
   /home/jgrusewski/Work/foxhunt/ml/src/features/price_features.rs:128-160

3. Rolling Statistics:
   /home/jgrusewski/Work/foxhunt/ml/src/features/statistical_features.rs:235-310

4. EWMA Implementation:
   /home/jgrusewski/Work/foxhunt/ml/src/features/ewma.rs:80-120, 220-260

5. Microstructure Features:
   /home/jgrusewski/Work/foxhunt/ml/src/features/microstructure_features.rs:1-300

6. Regime Framework:
   /home/jgrusewski/Work/foxhunt/adaptive-strategy/src/regime/mod.rs (full file)

7. Regime Module Structure (Wave D placeholder):
   /home/jgrusewski/Work/foxhunt/ml/src/regime/mod.rs

===============================================================================
SUMMARY
===============================================================================

✓ 50+ production-ready functions available
✓ 14 modules containing reusable infrastructure
✓ Existing regime detection framework ready
✓ Performance budgets available (500μs per bar)
✓ Design patterns (O(1) updates, numerically stable, NaN-safe)
✓ Full feature normalization pipeline
✓ Technical indicator foundation (Wave A integration)

RECOMMENDATION: Implement Wave D by creating new modules in ml/src/regime/
and REUSING these 50+ functions rather than reimplementing.

Principle: "REUSE existing infrastructure. DO NOT rebuild components."

Full detailed report saved to:
/home/jgrusewski/Work/foxhunt/WAVE_D_REUSABLE_UTILITIES_INVESTIGATION.md