From 261bbef86ec76ab833083d18b8991bcb1c463f0c Mon Sep 17 00:00:00 2001 From: jgrusewski Date: Sun, 19 Oct 2025 09:45:54 +0200 Subject: [PATCH] feat(wire-02): Document Wave D adaptive position sizer integration gap MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CRITICAL FINDING: RegimeAdaptiveFeatures (Features 221-224) are fully implemented but NOT integrated into trading decision flow. Analysis Results: - ✅ RegimeAdaptiveFeatures: 644 lines, 12/12 tests passing - ✅ Database schema: regime_states, regime_transitions, adaptive_strategy_metrics - ✅ gRPC endpoints: GetRegimeState, GetRegimeTransitions defined - ❌ Trading Agent Service: NO regime integration in allocation.rs - ❌ Order Generation: NO stop-loss multiplier application Impact: - ML models train with regime features - Production trading IGNORES regime state - Position sizes remain STATIC (no 0.2x-1.5x adjustment) - Expected Sharpe improvement: 0% (instead of +25-50%) Integration Plan (11 hours): 1. Phase 1: Database query layer (2h) - regime.rs 2. Phase 2: Allocation integration (3h) - RegimeAdaptive method 3. Phase 3: Service wiring (2h) - RegimeDetector in service 4. Phase 4: Order generation (1h) - stop-loss multipliers 5. Phase 5: Testing (3h) - regime allocation tests Code Changes: - New files: regime.rs (200 lines), tests (300 lines) - Modified: allocation.rs (+100), service.rs (+50), orders.rs (+30) - Total: ~500 new lines, ~180 modified lines Performance: +3ms latency (acceptable for +25-50% Sharpe) Risk: Low (feature flag + 3-level rollback plan) Recommendation: PROCEED before 225-feature ML retraining Files: - AGENT_WIRE02_ADAPTIVE_SIZER_INTEGRATION.md (full analysis) - AGENT_WIRE02_QUICK_SUMMARY.md (executive summary) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- AGENT_WIRE02_ADAPTIVE_SIZER_INTEGRATION.md | 763 +++++++++++++++++++++ AGENT_WIRE02_QUICK_SUMMARY.md | 164 +++++ AGENT_WIRE14_INTEGRATION_GAPS.txt | 129 ++++ AGENT_WIRE14_PAPER_TRADING_STATUS.md | 427 ++++++++++++ 4 files changed, 1483 insertions(+) create mode 100644 AGENT_WIRE02_ADAPTIVE_SIZER_INTEGRATION.md create mode 100644 AGENT_WIRE02_QUICK_SUMMARY.md create mode 100644 AGENT_WIRE14_INTEGRATION_GAPS.txt create mode 100644 AGENT_WIRE14_PAPER_TRADING_STATUS.md diff --git a/AGENT_WIRE02_ADAPTIVE_SIZER_INTEGRATION.md b/AGENT_WIRE02_ADAPTIVE_SIZER_INTEGRATION.md new file mode 100644 index 000000000..0da945423 --- /dev/null +++ b/AGENT_WIRE02_ADAPTIVE_SIZER_INTEGRATION.md @@ -0,0 +1,763 @@ +# Agent WIRE-02: Wave D Adaptive Position Sizer Integration Analysis + +**Date**: 2025-10-19 +**Agent**: WIRE-02 +**Mission**: Investigate why AdaptivePositionSizer is implemented but not integrated into trading flow +**Status**: ✅ **COMPLETE** - Gap identified, integration plan ready + +--- + +## Executive Summary + +**CRITICAL FINDING**: Wave D's regime-adaptive position sizing (Features 221-224) is **IMPLEMENTED but NOT INTEGRATED** into the trading decision flow. + +**Impact**: The 225-feature ML models will train with regime-adaptive features, but **production trading will not apply regime-based position sizing** unless we complete the missing integration. + +**Root Cause**: +1. `RegimeAdaptiveFeatures` exists in `ml/src/features/regime_adaptive.rs` (644 lines, 12/12 tests passing) +2. Database schema exists (`regime_states`, `regime_transitions`, `adaptive_strategy_metrics`) +3. gRPC endpoints exist (`GetRegimeState`, `GetRegimeTransitions`) +4. **BUT**: `services/trading_agent_service/src/allocation.rs` has NO imports/usage of regime detection + +**Gap**: Position sizing multipliers (0.2x-1.5x) and stop-loss multipliers (1.5x-4.0x ATR) are computed as ML features but **never applied to actual order sizing**. + +--- + +## Investigation Results + +### 1. Implementation Status ✅ + +#### ✅ Feature Extraction (`ml/src/features/regime_adaptive.rs`) + +**Location**: `/home/jgrusewski/Work/foxhunt/ml/src/features/regime_adaptive.rs` (644 lines) + +**Struct**: `RegimeAdaptiveFeatures` +```rust +pub struct RegimeAdaptiveFeatures { + current_regime: MarketRegime, + returns_window: VecDeque, + window_size: usize, + current_position_size: f64, + max_position_size: f64, + atr_period: usize, +} +``` + +**Key Methods**: +```rust +pub fn update( + &mut self, + regime: MarketRegime, + return_value: f64, + current_position: f64, + bars: &[OHLCVBar], +) -> [f64; 4] { + // Returns: + // [0]: Position multiplier (0.2x-1.5x based on regime) + // [1]: Stop-loss multiplier (1.5x-4.0x ATR based on regime) + // [2]: Regime-adjusted Sharpe ratio + // [3]: Risk budget utilization +} +``` + +**Position Multipliers** (Feature 221): +```rust +const POSITION_MULTIPLIERS: [(MarketRegime, f64); 7] = [ + (MarketRegime::Normal, 1.0), // Baseline + (MarketRegime::Trending, 1.5), // Increase size in trends + (MarketRegime::Sideways, 0.8), // Reduce in choppy markets + (MarketRegime::Bull, 1.2), // Moderate increase + (MarketRegime::Bear, 0.7), // Reduce in downtrends + (MarketRegime::HighVolatility, 0.5), // Reduce risk + (MarketRegime::Crisis, 0.2), // Extreme risk reduction +]; +``` + +**Stop-Loss Multipliers** (Feature 222): +```rust +const STOPLOSS_MULTIPLIERS: [(MarketRegime, f64); 7] = [ + (MarketRegime::Normal, 2.0), // 2x ATR + (MarketRegime::Trending, 2.5), // Wider for trends + (MarketRegime::Sideways, 1.5), // Tighter in ranges + (MarketRegime::Bull, 2.0), // Standard + (MarketRegime::Bear, 2.5), // Wider in bear + (MarketRegime::HighVolatility, 3.0), // Wide for volatility + (MarketRegime::Crisis, 4.0), // Very wide to avoid panic exits +]; +``` + +**Test Status**: 12/12 tests passing (100%) + +--- + +#### ✅ Database Schema (`migrations/045_wave_d_regime_tracking.sql`) + +**Tables Created**: + +1. **`regime_states`** (14 columns): + - `symbol`, `event_timestamp`, `regime`, `confidence` + - CUSUM metrics: `cusum_s_plus`, `cusum_s_minus`, `cusum_alert_count` + - ADX metrics: `adx`, `plus_di`, `minus_di` + - Stability: `stability`, `entropy` + - Indexes: `idx_regime_states_symbol_timestamp`, `idx_regime_states_regime`, `idx_regime_states_confidence` + +2. **`regime_transitions`** (10 columns): + - `symbol`, `event_timestamp`, `from_regime`, `to_regime` + - `duration_bars`, `transition_probability`, `adx_at_transition` + - Indexes: `idx_regime_transitions_symbol_timestamp`, `idx_regime_transitions_from_to` + +3. **`adaptive_strategy_metrics`** (12 columns): + - `symbol`, `event_timestamp`, `regime` + - **`position_multiplier`** (0.0-2.0) + - **`stop_loss_multiplier`** (1.0-5.0) + - `regime_sharpe`, `risk_budget_utilization` + - Performance: `total_trades`, `winning_trades`, `total_pnl` + +**Functions**: +- `get_latest_regime(p_symbol TEXT)`: Fetch current regime +- `get_regime_transition_matrix(p_symbol TEXT, p_window_hours INTEGER)`: Transition probabilities +- `get_regime_performance(p_symbol TEXT, p_window_hours INTEGER)`: Regime-specific performance + +**Migration Status**: Applied (verified in docs) + +--- + +#### ✅ gRPC API Endpoints + +**Proto Definitions** (confirmed in 100+ doc references): + +```protobuf +rpc GetRegimeState(GetRegimeStateRequest) returns (GetRegimeStateResponse); +rpc GetRegimeTransitions(GetRegimeTransitionsRequest) returns (GetRegimeTransitionsResponse); + +message GetRegimeStateRequest { + string symbol = 1; +} + +message GetRegimeStateResponse { + string regime = 1; + double confidence = 2; + int64 event_timestamp = 3; + double cusum_s_plus = 4; + double cusum_s_minus = 5; + double adx = 6; + double stability = 7; + double entropy = 8; +} +``` + +**Implementation Status**: Endpoints defined and routed through API Gateway (confirmed in `AGENT_F8_REGIME_ROUTING_VALIDATION_REPORT.md`) + +--- + +### 2. Integration Gap ❌ + +#### ❌ Trading Agent Service Allocation (`services/trading_agent_service/src/allocation.rs`) + +**Current State**: 716 lines, 5 allocation methods, **ZERO regime integration** + +**File**: `/home/jgrusewski/Work/foxhunt/services/trading_agent_service/src/allocation.rs` + +**Existing Allocation Methods**: +1. `EqualWeight` - 1/N allocation +2. `RiskParity` - Inverse volatility weighting +3. `MeanVariance` - Markowitz optimization +4. `MLOptimized` - ML scores as expected returns +5. `KellyCriterion` - Edge-based sizing + +**Missing**: +```rust +// ❌ NO IMPORTS +// use ml::features::regime_adaptive::RegimeAdaptiveFeatures; +// use ml::ensemble::MarketRegime; + +// ❌ NO REGIME DETECTION +pub fn allocate(&self, assets: &[AssetInfo], total_capital: Decimal) -> Result> { + match &self.method { + AllocationMethod::EqualWeight => self.equal_weight(assets, total_capital), + AllocationMethod::RiskParity => self.risk_parity(assets, total_capital), + // ... NO REGIME ADAPTATION + } +} +``` + +**Impact**: All allocation methods compute static weights WITHOUT regime-based adjustments. + +--- + +#### ❌ Service Implementation (`services/trading_agent_service/src/service.rs`) + +**File**: `/home/jgrusewski/Work/foxhunt/services/trading_agent_service/src/service.rs` (675 lines) + +**Current `allocate_portfolio` Implementation**: +```rust +async fn allocate_portfolio( + &self, + _request: Request, +) -> Result, Status> { + info!("AllocatePortfolio called (placeholder)"); + + Ok(Response::new(AllocatePortfolioResponse { + allocations: vec![], + metrics: Some(AllocationMetrics { + total_weight: 0.0, + portfolio_volatility: 0.0, + portfolio_sharpe: 0.0, + var_95: 0.0, + max_drawdown_estimate: 0.0, + }), + timestamp: chrono::Utc::now().timestamp_nanos_opt().unwrap_or(0), + allocation_id: uuid::Uuid::new_v4().to_string(), + })) +} +``` + +**Status**: **PLACEHOLDER** - no actual allocation logic, no regime detection + +--- + +### 3. Data Flow Analysis + +#### Current Flow (Wave C - 201 Features) +``` +1. Universe Selection → [ES.FUT, NQ.FUT, ZN.FUT] +2. Asset Selection → AssetInfo[] (with ML scores) +3. PortfolioAllocator::allocate() → Static weights (e.g., 1/N) +4. Order Generation → Fixed position sizes +5. Trading Service → Execute orders +``` + +#### Missing Flow (Wave D - 225 Features) +``` +1. Universe Selection → [ES.FUT, NQ.FUT, ZN.FUT] +2. FOR EACH symbol: + a. Query regime_states table → get_latest_regime(symbol) + b. Retrieve MarketRegime (Normal/Trending/Volatile/Crisis) + c. Look up position_multiplier (0.2x-1.5x) + d. Look up stop_loss_multiplier (1.5x-4.0x ATR) +3. Asset Selection → AssetInfo[] (with ML scores) +4. ❌ MISSING: Apply regime multipliers to allocation weights +5. ❌ MISSING: Adjust position sizes by regime +6. Order Generation → ❌ Uses unadjusted sizes +7. Trading Service → ❌ Executes with wrong position sizes +``` + +--- + +## Integration Plan + +### Phase 1: Database Query Layer (2 hours) + +**File**: `services/trading_agent_service/src/regime.rs` (NEW) + +```rust +//! Regime Detection Integration +//! +//! Provides regime state queries for position sizing adjustments. + +use sqlx::PgPool; +use anyhow::{Context, Result}; + +/// Market regime classification +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum MarketRegime { + Normal, + Trending, + Sideways, + Bull, + Bear, + HighVolatility, + Crisis, +} + +/// Regime state with metrics +#[derive(Debug, Clone)] +pub struct RegimeState { + pub regime: MarketRegime, + pub confidence: f64, + pub event_timestamp: chrono::DateTime, + pub position_multiplier: f64, // From adaptive_strategy_metrics + pub stop_loss_multiplier: f64, // From adaptive_strategy_metrics +} + +/// Regime detection client +pub struct RegimeDetector { + db_pool: PgPool, +} + +impl RegimeDetector { + pub fn new(db_pool: PgPool) -> Self { + Self { db_pool } + } + + /// Get current regime state for symbol + pub async fn get_regime(&self, symbol: &str) -> Result { + // Query database using get_latest_regime() function + let row = sqlx::query!( + r#" + SELECT regime, confidence, event_timestamp + FROM get_latest_regime($1) + "#, + symbol + ) + .fetch_one(&self.db_pool) + .await + .context("Failed to fetch regime state")?; + + // Query adaptive metrics + let metrics = sqlx::query!( + r#" + SELECT position_multiplier, stop_loss_multiplier + FROM adaptive_strategy_metrics + WHERE symbol = $1 AND event_timestamp = $2 + ORDER BY event_timestamp DESC + LIMIT 1 + "#, + symbol, + row.event_timestamp + ) + .fetch_one(&self.db_pool) + .await + .context("Failed to fetch adaptive metrics")?; + + Ok(RegimeState { + regime: parse_regime(&row.regime.unwrap_or_default())?, + confidence: row.confidence.unwrap_or(0.0), + event_timestamp: row.event_timestamp.unwrap(), + position_multiplier: metrics.position_multiplier, + stop_loss_multiplier: metrics.stop_loss_multiplier, + }) + } + + /// Get regime for multiple symbols (batch query) + pub async fn get_regimes_batch(&self, symbols: &[String]) -> Result> { + let mut results = Vec::new(); + for symbol in symbols { + if let Ok(state) = self.get_regime(symbol).await { + results.push((symbol.clone(), state)); + } + } + Ok(results) + } +} + +fn parse_regime(s: &str) -> Result { + match s { + "Normal" => Ok(MarketRegime::Normal), + "Trending" => Ok(MarketRegime::Trending), + "Ranging" | "Sideways" => Ok(MarketRegime::Sideways), + "Bull" => Ok(MarketRegime::Bull), + "Bear" => Ok(MarketRegime::Bear), + "Volatile" | "HighVolatility" => Ok(MarketRegime::HighVolatility), + "Crisis" => Ok(MarketRegime::Crisis), + _ => Ok(MarketRegime::Normal), // Default fallback + } +} +``` + +--- + +### Phase 2: Allocation Integration (3 hours) + +**File**: `services/trading_agent_service/src/allocation.rs` (MODIFY) + +**Changes**: + +1. **Add regime-aware allocation method**: +```rust +#[derive(Debug, Clone)] +pub enum AllocationMethod { + EqualWeight, + RiskParity, + MeanVariance { lambda: f64 }, + MLOptimized, + KellyCriterion { fraction: f64 }, + // ✅ NEW: Regime-adaptive allocation + RegimeAdaptive { + base_method: Box, + regime_detector: Arc, + }, +} +``` + +2. **Implement regime-adaptive wrapper**: +```rust +async fn regime_adaptive( + &self, + assets: &[AssetInfo], + total_capital: Decimal, + base_method: &AllocationMethod, + regime_detector: &RegimeDetector, +) -> Result> { + // Step 1: Get base allocation (e.g., from MLOptimized) + let base_allocator = PortfolioAllocator::new((**base_method).clone()); + let base_allocation = base_allocator.allocate(assets, total_capital)?; + + // Step 2: Fetch regime states for all symbols + let symbols: Vec = assets.iter().map(|a| a.symbol.clone()).collect(); + let regime_states = regime_detector.get_regimes_batch(&symbols).await?; + + // Step 3: Apply regime multipliers + let mut adjusted_allocation = HashMap::new(); + for (symbol, base_capital) in base_allocation { + if let Some((_, regime_state)) = regime_states.iter().find(|(s, _)| s == &symbol) { + // Apply position multiplier (0.2x-1.5x) + let adjusted_capital = base_capital * Decimal::from_f64_retain(regime_state.position_multiplier) + .unwrap_or(Decimal::ONE); + + adjusted_allocation.insert(symbol, adjusted_capital); + } else { + // No regime data → use base allocation + adjusted_allocation.insert(symbol, base_capital); + } + } + + // Step 4: Renormalize to total_capital (multipliers may exceed 100%) + let sum: Decimal = adjusted_allocation.values().sum(); + if sum > total_capital { + for capital in adjusted_allocation.values_mut() { + *capital = (*capital / sum) * total_capital; + } + } + + Ok(adjusted_allocation) +} +``` + +3. **Update `allocate()` to support regime method**: +```rust +pub async fn allocate_async( + &self, + assets: &[AssetInfo], + total_capital: Decimal, +) -> Result> { + match &self.method { + AllocationMethod::EqualWeight => self.equal_weight(assets, total_capital), + AllocationMethod::RiskParity => self.risk_parity(assets, total_capital), + AllocationMethod::MeanVariance { lambda } => self.mean_variance(assets, total_capital, *lambda), + AllocationMethod::MLOptimized => self.ml_optimized(assets, total_capital), + AllocationMethod::KellyCriterion { fraction } => self.kelly_criterion(assets, total_capital, *fraction), + // ✅ NEW + AllocationMethod::RegimeAdaptive { base_method, regime_detector } => { + self.regime_adaptive(assets, total_capital, base_method, regime_detector).await + }, + } +} +``` + +--- + +### Phase 3: Service Wiring (2 hours) + +**File**: `services/trading_agent_service/src/service.rs` (MODIFY) + +**Changes**: + +1. **Add RegimeDetector to service state**: +```rust +pub struct TradingAgentServiceImpl { + db_pool: PgPool, + universe_selector: UniverseSelector, + strategy_coordinator: StrategyCoordinator, + metrics: TradingAgentMetrics, + regime_detector: Arc, // ✅ NEW +} + +impl TradingAgentServiceImpl { + pub fn new(db_pool: PgPool) -> Self { + Self { + universe_selector: UniverseSelector::new(db_pool.clone()), + strategy_coordinator: StrategyCoordinator::new(db_pool.clone()), + metrics: TradingAgentMetrics::new(), + regime_detector: Arc::new(RegimeDetector::new(db_pool.clone())), // ✅ NEW + db_pool, + } + } +} +``` + +2. **Implement `allocate_portfolio` with regime adaptation**: +```rust +async fn allocate_portfolio( + &self, + request: Request, +) -> Result, Status> { + let req = request.into_inner(); + + // Convert proto assets to AssetInfo + let assets: Vec = req.assets.iter().map(|a| { + AssetInfo { + symbol: a.symbol.clone(), + expected_return: a.expected_return, + volatility: a.volatility, + ml_score: a.ml_score, + // ... other fields + } + }).collect(); + + let total_capital = Decimal::from_f64_retain(req.total_capital) + .ok_or_else(|| Status::invalid_argument("Invalid total capital"))?; + + // ✅ Use regime-adaptive allocation + let base_method = AllocationMethod::MLOptimized; + let allocator = PortfolioAllocator::new(AllocationMethod::RegimeAdaptive { + base_method: Box::new(base_method), + regime_detector: self.regime_detector.clone(), + }); + + let allocations = allocator.allocate_async(&assets, total_capital) + .await + .map_err(|e| Status::internal(format!("Allocation failed: {}", e)))?; + + // Convert to proto + let proto_allocations: Vec = allocations.iter().map(|(symbol, capital)| { + Allocation { + symbol: symbol.clone(), + capital: capital.to_f64().unwrap_or(0.0), + weight: (capital / total_capital).to_f64().unwrap_or(0.0), + } + }).collect(); + + Ok(Response::new(AllocatePortfolioResponse { + allocations: proto_allocations, + // ... metrics + })) +} +``` + +--- + +### Phase 4: Order Generation Integration (1 hour) + +**File**: `services/trading_agent_service/src/orders.rs` (MODIFY) + +**Changes**: + +1. **Add stop-loss multiplier to order generation**: +```rust +pub async fn generate_order( + &self, + allocation: &PortfolioAllocation, + regime_state: &RegimeState, // ✅ NEW parameter +) -> Result { + let price = self.get_current_price(&allocation.symbol).await?; + let quantity = (allocation.capital / price).floor(); + + // ✅ Apply regime-based stop-loss + let atr = self.calculate_atr(&allocation.symbol, 14).await?; + let stop_loss_distance = atr * regime_state.stop_loss_multiplier; + + let stop_loss_price = if allocation.direction == Direction::Long { + price - stop_loss_distance + } else { + price + stop_loss_distance + }; + + Ok(Order { + symbol: allocation.symbol.clone(), + quantity, + price, + stop_loss: Some(stop_loss_price), + // ... other fields + }) +} +``` + +--- + +### Phase 5: Testing (3 hours) + +**File**: `services/trading_agent_service/tests/regime_allocation_test.rs` (NEW) + +**Test Cases**: + +1. **Test regime multiplier application**: + - Crisis regime (0.2x) → $100K allocation → $20K actual + - Trending regime (1.5x) → $100K allocation → $150K actual (then renormalized) + - Normal regime (1.0x) → $100K allocation → $100K actual + +2. **Test stop-loss adjustment**: + - Volatile regime (3.0x ATR) → ATR=$10 → Stop=$30 away + - Sideways regime (1.5x ATR) → ATR=$10 → Stop=$15 away + +3. **Test database fallback**: + - No regime data → Use base allocation without multipliers + - Stale regime data (>1 hour old) → Fall back to Normal regime + +4. **Test end-to-end flow**: + - Mock regime_states table with test data + - Call `allocate_portfolio` gRPC endpoint + - Verify allocations reflect regime multipliers + +--- + +## Validation Checklist + +### Pre-Integration ✅ +- [x] RegimeAdaptiveFeatures implemented (644 lines, 12/12 tests) +- [x] Database schema created (regime_states, regime_transitions, adaptive_strategy_metrics) +- [x] gRPC endpoints defined (GetRegimeState, GetRegimeTransitions) +- [x] Migration 045 applied + +### Post-Integration (To Be Verified) +- [ ] `services/trading_agent_service/src/regime.rs` created +- [ ] `services/trading_agent_service/src/allocation.rs` updated with RegimeAdaptive method +- [ ] `services/trading_agent_service/src/service.rs` wired with RegimeDetector +- [ ] `services/trading_agent_service/src/orders.rs` applies stop-loss multipliers +- [ ] Test suite validates regime multipliers applied correctly +- [ ] End-to-end test: Crisis regime → 0.2x position size +- [ ] End-to-end test: Trending regime → 1.5x position size +- [ ] End-to-end test: Volatile regime → 3.0x ATR stop-loss + +--- + +## Risk Assessment + +### Risk 1: Database Query Latency +- **Impact**: Regime queries add latency to allocation decisions +- **Mitigation**: + - Use database indexes (already created: `idx_regime_states_symbol_timestamp`) + - Batch queries for multiple symbols (`get_regimes_batch`) + - Cache regime states (TTL: 1 minute) +- **Acceptable Latency**: <5ms per symbol (P99) + +### Risk 2: Stale Regime Data +- **Impact**: Trading with outdated regime classifications +- **Mitigation**: + - Check `event_timestamp` (reject data >1 hour old) + - Fall back to Normal regime (1.0x multiplier) if stale + - Monitor `regime_states` freshness via Grafana + +### Risk 3: Over-Leverage in Trending Regimes +- **Impact**: 1.5x multiplier could exceed risk limits +- **Mitigation**: + - Renormalize allocations to 100% after applying multipliers + - Hard cap: No single position >20% (already in allocation.rs) + - Risk budget monitoring (Feature 224) + +### Risk 4: Whipsaw in Crisis Regimes +- **Impact**: 0.2x multiplier during false alarms → missed opportunities +- **Mitigation**: + - Require high confidence (>0.8) for Crisis regime + - Monitor regime flip-flopping (>50/hour alert) + - Manual override capability + +--- + +## Performance Impact + +### Latency Breakdown (Estimated) +``` +Current (Wave C - 201 features): +- Universe Selection: 50ms +- Asset Selection: 20ms +- Allocation (MLOptimized): 0.5ms +- Order Generation: 10ms +- TOTAL: 80.5ms + +With Regime Integration (Wave D - 225 features): +- Universe Selection: 50ms +- Asset Selection: 20ms +- Regime Detection (batch query): +3ms ← NEW +- Allocation (RegimeAdaptive): 0.5ms +- Order Generation (with stop-loss): 10ms +- TOTAL: 83.5ms (+3.7% overhead) +``` + +**Conclusion**: +3ms overhead is acceptable for 25-50% Sharpe improvement. + +--- + +## Code Statistics + +### Files to Create +1. `services/trading_agent_service/src/regime.rs` (~200 lines) +2. `services/trading_agent_service/tests/regime_allocation_test.rs` (~300 lines) + +### Files to Modify +1. `services/trading_agent_service/src/allocation.rs` (+100 lines) +2. `services/trading_agent_service/src/service.rs` (+50 lines) +3. `services/trading_agent_service/src/orders.rs` (+30 lines) +4. `services/trading_agent_service/src/lib.rs` (+1 line for module export) + +### Total Code Changes +- **New Code**: ~500 lines +- **Modified Code**: ~180 lines +- **Total Effort**: ~11 hours (2+3+2+1+3) + +--- + +## Rollback Plan + +### Level 1: Feature Flag (Immediate) +```rust +const ENABLE_REGIME_ADAPTIVE: bool = false; // Set to true to enable + +if ENABLE_REGIME_ADAPTIVE { + AllocationMethod::RegimeAdaptive { ... } +} else { + AllocationMethod::MLOptimized // Fall back to Wave C behavior +} +``` + +### Level 2: Database Rollback (5 minutes) +```sql +-- Disable regime tables (keep data) +REVOKE SELECT ON regime_states FROM foxhunt; +REVOKE SELECT ON adaptive_strategy_metrics FROM foxhunt; +``` + +### Level 3: Code Rollback (10 minutes) +```bash +git revert +cargo build --release -p trading_agent_service +systemctl restart trading_agent_service +``` + +--- + +## Next Steps + +### Immediate (Priority 1) +1. ✅ **This Report**: Document integration gap +2. ⏳ **User Decision**: Approve integration plan (11 hours effort) +3. ⏳ **Implementation**: Execute 5-phase integration plan + +### Short-term (After Integration) +4. ⏳ **Testing**: Run regime allocation tests (3 hours) +5. ⏳ **Validation**: Paper trading with regime-adaptive sizing (1 week) +6. ⏳ **Monitoring**: Set up Grafana alerts for regime metrics + +### Medium-term (Production) +7. ⏳ **Performance Tuning**: Optimize database queries (<5ms P99) +8. ⏳ **Caching**: Add 1-minute TTL cache for regime states +9. ⏳ **Documentation**: Update production runbooks + +--- + +## Conclusion + +**Gap Confirmed**: Wave D's adaptive position sizing (Features 221-224) is **fully implemented** but **NOT integrated** into the trading decision flow. + +**Impact**: Without integration, the 225-feature ML models will include regime-adaptive features in training, but **production trading will ignore regime-based position sizing**. + +**Recommendation**: **PROCEED WITH INTEGRATION** before ML retraining. + +**Rationale**: +1. **Effort**: 11 hours (manageable) +2. **Risk**: Low (feature flag + rollback plan) +3. **Benefit**: +25-50% Sharpe improvement (per Wave D hypothesis) +4. **Urgency**: Must complete before 225-feature ML retraining (4-6 weeks) + +**Decision Required**: Approve integration plan and proceed with implementation? + +--- + +**Files Analyzed**: +- `/home/jgrusewski/Work/foxhunt/ml/src/features/regime_adaptive.rs` (644 lines) +- `/home/jgrusewski/Work/foxhunt/services/trading_agent_service/src/allocation.rs` (716 lines) +- `/home/jgrusewski/Work/foxhunt/services/trading_agent_service/src/service.rs` (675 lines) +- `/home/jgrusewski/Work/foxhunt/migrations/045_wave_d_regime_tracking.sql` (12,819 bytes) +- `/home/jgrusewski/Work/foxhunt/AGENT_D11_PORTFOLIO_ALLOCATION_IMPLEMENTATION_REPORT.md` (analysis) + +**Agent WIRE-02 Complete** | 2025-10-19 diff --git a/AGENT_WIRE02_QUICK_SUMMARY.md b/AGENT_WIRE02_QUICK_SUMMARY.md new file mode 100644 index 000000000..d1bf5ee1f --- /dev/null +++ b/AGENT_WIRE02_QUICK_SUMMARY.md @@ -0,0 +1,164 @@ +# Agent WIRE-02: Quick Summary + +**Date**: 2025-10-19 +**Status**: ✅ COMPLETE - Integration gap identified + +--- + +## 🔴 CRITICAL FINDING + +**Wave D's adaptive position sizing is IMPLEMENTED but NOT INTEGRATED into trading flow.** + +--- + +## Gap Analysis + +### ✅ What's Done +1. **RegimeAdaptiveFeatures** - Fully implemented (644 lines, 12/12 tests) + - Position multipliers: 0.2x (Crisis) to 1.5x (Trending) + - Stop-loss multipliers: 1.5x (Sideways) to 4.0x ATR (Crisis) + - Location: `ml/src/features/regime_adaptive.rs` + +2. **Database Schema** - Migration 045 applied + - Tables: `regime_states`, `regime_transitions`, `adaptive_strategy_metrics` + - Functions: `get_latest_regime()`, `get_regime_transition_matrix()` + +3. **gRPC Endpoints** - Defined and routed + - `GetRegimeState(symbol) → RegimeStateResponse` + - `GetRegimeTransitions(symbol) → TransitionsResponse` + +### ❌ What's Missing +1. **Trading Agent Service** - NO regime integration + - File: `services/trading_agent_service/src/allocation.rs` (716 lines) + - Status: 5 allocation methods (EqualWeight, RiskParity, MeanVariance, MLOptimized, KellyCriterion) + - **NO imports** of `RegimeAdaptiveFeatures` + - **NO database queries** to `regime_states` + - **NO application** of position/stop-loss multipliers + +2. **Order Generation** - NO stop-loss adjustment + - File: `services/trading_agent_service/src/orders.rs` + - Status: Static stop-loss logic, no regime-based ATR multipliers + +--- + +## Impact + +**Without Integration**: +- ML models train with Features 221-224 (regime multipliers) +- **BUT** production trading ignores regime state +- Position sizes stay STATIC (no 0.2x-1.5x adjustment) +- Stop-losses stay STATIC (no 1.5x-4.0x ATR adjustment) +- **Expected Sharpe improvement: 0%** (instead of +25-50%) + +--- + +## Integration Plan + +### 5-Phase Implementation (11 hours total) + +**Phase 1**: Database Query Layer (2h) +- Create `services/trading_agent_service/src/regime.rs` +- Implement `RegimeDetector` to query `regime_states` table +- Add `get_regime(symbol) → RegimeState` method + +**Phase 2**: Allocation Integration (3h) +- Add `AllocationMethod::RegimeAdaptive` enum variant +- Implement regime multiplier wrapper around base allocation +- Apply position multipliers (0.2x-1.5x) to allocation weights + +**Phase 3**: Service Wiring (2h) +- Add `RegimeDetector` to `TradingAgentServiceImpl` +- Wire `allocate_portfolio` gRPC endpoint to use regime-adaptive allocation +- Add database connection pooling + +**Phase 4**: Order Generation (1h) +- Update `OrderGenerator::generate_order()` to accept `RegimeState` +- Apply stop-loss multipliers (1.5x-4.0x ATR) based on regime + +**Phase 5**: Testing (3h) +- Create `tests/regime_allocation_test.rs` +- Validate Crisis regime → 0.2x position size +- Validate Trending regime → 1.5x position size +- Validate Volatile regime → 3.0x ATR stop-loss + +--- + +## Code Changes + +### Files to Create +1. `services/trading_agent_service/src/regime.rs` (~200 lines) +2. `services/trading_agent_service/tests/regime_allocation_test.rs` (~300 lines) + +### Files to Modify +1. `services/trading_agent_service/src/allocation.rs` (+100 lines) +2. `services/trading_agent_service/src/service.rs` (+50 lines) +3. `services/trading_agent_service/src/orders.rs` (+30 lines) +4. `services/trading_agent_service/src/lib.rs` (+1 line) + +**Total**: ~500 lines new, ~180 lines modified + +--- + +## Performance Impact + +**Latency Addition**: +3ms (batch regime queries) +- Current: 80.5ms end-to-end +- With regime: 83.5ms (+3.7% overhead) +- **Acceptable** for +25-50% Sharpe improvement + +--- + +## Risk Mitigation + +1. **Feature Flag**: Easy on/off toggle +2. **Database Indexes**: Already created (`idx_regime_states_symbol_timestamp`) +3. **Fallback**: Use Normal regime (1.0x) if data stale/missing +4. **Renormalization**: Prevent over-leverage from 1.5x multipliers +5. **Rollback**: 3-level plan (flag → database → code) + +--- + +## Recommendation + +**✅ PROCEED WITH INTEGRATION** before ML retraining + +**Why**: +- **Effort**: 11 hours (manageable) +- **Risk**: Low (feature flag + rollback plan) +- **Benefit**: Unlock +25-50% Sharpe improvement +- **Urgency**: Must complete before 225-feature ML retraining (4-6 weeks) + +**Next Step**: User approval to execute 5-phase integration plan + +--- + +## Example: Crisis Regime Behavior + +**Scenario**: Market crash detected (Crisis regime) + +**Without Integration** (Current): +- Base allocation: $100K to ES.FUT +- **Actual position**: $100K (FULL RISK) +- Stop-loss: 2.0x ATR = $20 away +- **Result**: Full exposure during crisis ❌ + +**With Integration** (After Fix): +- Base allocation: $100K to ES.FUT +- Regime multiplier: 0.2x (Crisis) +- **Actual position**: $20K (80% RISK REDUCTION) ✅ +- Stop-loss: 4.0x ATR = $40 away (wider to avoid panic exit) +- **Result**: Protected capital during crisis ✅ + +--- + +## Files Referenced + +- ✅ `/home/jgrusewski/Work/foxhunt/ml/src/features/regime_adaptive.rs` +- ✅ `/home/jgrusewski/Work/foxhunt/migrations/045_wave_d_regime_tracking.sql` +- ❌ `/home/jgrusewski/Work/foxhunt/services/trading_agent_service/src/allocation.rs` (NO integration) +- ❌ `/home/jgrusewski/Work/foxhunt/services/trading_agent_service/src/service.rs` (placeholder only) +- ❌ `/home/jgrusewski/Work/foxhunt/services/trading_agent_service/src/orders.rs` (static stop-loss) + +--- + +**Agent WIRE-02 Complete** | 2025-10-19 diff --git a/AGENT_WIRE14_INTEGRATION_GAPS.txt b/AGENT_WIRE14_INTEGRATION_GAPS.txt new file mode 100644 index 000000000..01acf1202 --- /dev/null +++ b/AGENT_WIRE14_INTEGRATION_GAPS.txt @@ -0,0 +1,129 @@ +╔══════════════════════════════════════════════════════════════════════════════╗ +║ AGENT WIRE-14: PAPER TRADING WAVE D INTEGRATION GAPS ║ +╚══════════════════════════════════════════════════════════════════════════════╝ + +CURRENT STATE (Paper Trading Executor): +┌──────────────────────────────────────────────────────────────────────────┐ +│ Component │ Status │ Wave C Baseline │ Wave D Required │ +├────────────────────────┼─────────┼──────────────────┼───────────────────┤ +│ ML Strategy │ ✅ YES │ SharedMLStrategy │ SharedMLStrategy │ +│ Feature Config │ ❌ NO │ Hardcoded (20) │ FeatureConfig:: │ +│ │ │ │ wave_d() │ +│ Feature Count │ ❌ NO │ 201 features │ 225 features │ +│ Regime Queries │ ❌ NO │ None │ get_latest_regime │ +│ Position Sizing │ ❌ NO │ Fixed 1.0 │ Adaptive 0.2-1.5x │ +│ Kelly Criterion │ ❌ NO │ Comment only │ Implemented │ +└────────────────────────┴─────────┴──────────────────┴───────────────────┘ + +CRITICAL GAPS: + +1. FEATURE EXTRACTION (Line 154-157) + ┌─────────────────────────────────────────────────────────────────────┐ + │ Current: SharedMLStrategy::new(20, 0.6) │ + │ ^^^^ Hardcoded - NO feature config │ + │ │ + │ Required: SharedMLStrategy::new_with_config( │ + │ 20, 0.6, │ + │ FeatureConfig::wave_d() // ✅ 225 features │ + │ ) │ + └─────────────────────────────────────────────────────────────────────┘ + +2. REGIME STATE QUERIES (Missing) + ┌─────────────────────────────────────────────────────────────────────┐ + │ Current: No database queries for regime_states │ + │ │ + │ Required: let regime = sqlx::query!( │ + │ "SELECT regime FROM get_latest_regime($1)", │ + │ symbol │ + │ ).fetch_one(&self.db_pool).await?; │ + └─────────────────────────────────────────────────────────────────────┘ + +3. ADAPTIVE POSITION SIZING (Line 567-575) + ┌─────────────────────────────────────────────────────────────────────┐ + │ Current: let position_size = 1.0; // Fixed │ + │ │ + │ Required: let regime_mult = match regime { │ + │ "Trending" => 1.5, │ + │ "Ranging" => 0.8, │ + │ "Volatile" => 0.5, │ + │ "Transition" => 0.2, │ + │ }; │ + │ let size = base * regime_mult * confidence; │ + └─────────────────────────────────────────────────────────────────────┘ + +ARCHITECTURE MISMATCH: + + common::ml_strategy::MLFeatureExtractor + ├─ Expected feature count: hardcoded comment (26/36/65) + ├─ NO FeatureConfig integration + └─ NO Wave D support (225 features) + + ml::features::config::FeatureConfig + ├─ wave_d() method exists ✅ + ├─ enable_wave_d_regime: true ✅ + └─ 225 feature support ✅ + + ⚠️ PROBLEM: These two systems are NOT connected! + +TESTING RISK: + + Paper Trading Current: + ┌─────────────────────────────────────────────────────────────────────┐ + │ Features: 201 (Wave C baseline) │ + │ Sizing: Fixed 1.0 contracts │ + │ Regime: Not aware │ + │ │ + │ Result: Tests Wave C, NOT Wave D ❌ │ + └─────────────────────────────────────────────────────────────────────┘ + + Paper Trading Required: + ┌─────────────────────────────────────────────────────────────────────┐ + │ Features: 225 (Wave D regime detection) │ + │ Sizing: Adaptive 0.2x-1.5x based on regime │ + │ Regime: Queries regime_states before each trade │ + │ │ + │ Result: Validates Wave D before production ✅ │ + └─────────────────────────────────────────────────────────────────────┘ + +ACTION PLAN (6 hours): + + [P1] Modify SharedMLStrategy constructor (2h) + └─ Accept FeatureConfig parameter + └─ Update paper_trading_executor.rs + └─ Verify 225-feature extraction + + [P2] Add regime state queries (1h) + └─ Implement get_regime_for_symbol() + └─ Query get_latest_regime() before trades + └─ Log regime transitions + + [P3] Adaptive position sizing (2h) + └─ Replace calculate_position_size() + └─ Implement regime multipliers (0.2x-1.5x) + └─ Add confidence-based Kelly factor + + [P4] Testing & validation (1h) + └─ Run 24-hour paper trading test + └─ Monitor regime vs. sizing correlation + └─ Document Wave C vs. Wave D performance + +RECOMMENDATION: + + ⛔ BLOCK production deployment until paper trading validates Wave D + + Why? Paper trading is the ONLY pre-production validation step. + If it tests Wave C config, we have ZERO evidence that: + - 225-feature extraction works + - Regime detection improves performance + - Adaptive sizing reduces drawdowns + + Next Steps: + 1. Implement action items (6 hours) + 2. Run 24-hour paper trading validation + 3. Compare Wave C baseline vs. Wave D adaptive results + 4. Document findings in PAPER_TRADING_WAVE_D_VALIDATION.md + +═══════════════════════════════════════════════════════════════════════════════ +Agent WIRE-14 Status: ⚠️ PARTIAL INTEGRATION - CRITICAL GAPS IDENTIFIED +Next Agent: WIRE-15 (Adaptive Position Sizing Implementation) +═══════════════════════════════════════════════════════════════════════════════ diff --git a/AGENT_WIRE14_PAPER_TRADING_STATUS.md b/AGENT_WIRE14_PAPER_TRADING_STATUS.md new file mode 100644 index 000000000..0757c3683 --- /dev/null +++ b/AGENT_WIRE14_PAPER_TRADING_STATUS.md @@ -0,0 +1,427 @@ +# AGENT WIRE-14: Paper Trading Executor Wave D Integration Status + +**Agent**: WIRE-14 +**Mission**: Verify paper trading executor uses Wave D features and adaptive sizing +**Status**: ⚠️ PARTIAL INTEGRATION - Missing Wave D Features +**Priority**: HIGH - Paper trading must test Wave D before live deployment +**Date**: 2025-10-19 + +--- + +## Executive Summary + +The paper trading executor (`services/trading_service/src/paper_trading_executor.rs`) currently uses `SharedMLStrategy` but is **NOT configured for Wave D features**. Critical gaps identified: + +1. ✅ Uses `SharedMLStrategy` (ONE SINGLE SYSTEM architecture) +2. ❌ **NO Wave D feature configuration** - Uses hardcoded defaults (20 lookback, 0.6 confidence) +3. ❌ **NO regime state queries** - Does not check `regime_states` table +4. ❌ **NO adaptive position sizing** - Uses fixed 1.0 contract size +5. ⚠️ **Kelly Criterion mentioned but not implemented** (line 569 comment only) + +**Risk**: Paper trading will test Wave C baseline (201 features) instead of Wave D (225 features + regime detection). + +--- + +## Code Analysis + +### 1. ML Strategy Initialization + +**File**: `services/trading_service/src/paper_trading_executor.rs` +**Lines**: 154-157 + +```rust +pub fn new(db_pool: PgPool, config: PaperTradingConfig) -> Self { + // Initialize with shared ML strategy (default configuration) + let ml_strategy = SharedMLStrategy::new(20, 0.6); + // ^^^ HARDCODED: 20 lookback, 0.6 confidence - NO Wave D config +``` + +**Issue**: `SharedMLStrategy::new()` does NOT accept `FeatureConfig` parameter. The constructor signature is: +```rust +pub fn new(lookback_periods: usize, min_confidence_threshold: f64) -> Self +``` + +**Missing**: No way to pass `FeatureConfig::wave_d()` to enable 225-feature extraction. + +--- + +### 2. Position Sizing Logic + +**File**: `services/trading_service/src/paper_trading_executor.rs` +**Lines**: 567-575 + +```rust +fn calculate_position_size(&self, _prediction: &PendingPrediction) -> Result { + // Simple fixed position size for paper trading + // In production, this could use Kelly Criterion or volatility-adjusted sizing + let position_size = 1.0; // 1 contract + // ^^^ FIXED SIZE: No adaptive sizing based on regime or confidence + + if position_size > self.config.max_position_size { + return Err(anyhow!("Position size {} exceeds maximum {}", position_size, self.config.max_position_size)); + } + + Ok(position_size) +} +``` + +**Missing Wave D Adaptive Logic**: +- No regime state queries (`SELECT regime FROM regime_states`) +- No adaptive multipliers (0.2x-1.5x based on regime) +- No Kelly Criterion position sizing +- No volatility-adjusted sizing + +**Expected Behavior** (from Wave D design): +```rust +// Query regime state +let regime = sqlx::query!("SELECT regime FROM get_latest_regime($1)", symbol) + .fetch_one(&self.db_pool).await?; + +// Apply regime-adaptive multiplier +let base_size = 1.0; +let regime_multiplier = match regime.regime.as_str() { + "Trending" => 1.5, // Increase size in trending markets + "Ranging" => 0.8, // Reduce size in ranging markets + "Volatile" => 0.5, // Minimize size in volatile markets + "Transition" => 0.2, // Avoid trading during transitions + _ => 1.0, // Normal sizing for unknown regimes +}; + +let position_size = base_size * regime_multiplier * confidence_factor; +``` + +--- + +### 3. Regime State Integration + +**Search Results**: ❌ NO regime queries found in `paper_trading_executor.rs` + +```bash +$ grep -rn "regime_states\|regime_transitions\|get_latest_regime" \ + services/trading_service/src/paper_trading_executor.rs + +# Result: 0 matches +``` + +**Contrast with `trading.rs` (Trading Service)**: +```rust +// services/trading_service/src/services/trading.rs:992-1023 +async fn get_regime_state(&self, req: Request) -> Result, Status> { + let regime_state = sqlx::query!( + r#"SELECT regime, confidence, detected_at FROM get_latest_regime($1)"#, + req.symbol + ).fetch_one(&self.db_pool).await?; + + Ok(Response::new(GetRegimeStateResponse { + current_regime: regime_state.regime.unwrap_or("Normal".to_string()), + confidence: regime_state.confidence.unwrap_or(0.0), + // ... + })) +} +``` + +**Paper Trading Executor**: No equivalent logic. + +--- + +### 4. Feature Configuration Architecture + +**Analysis**: `SharedMLStrategy` uses `MLFeatureExtractor` which has a **legacy field** for feature count: + +**File**: `common/src/ml_strategy.rs` (lines 66-84) +```rust +pub struct MLFeatureExtractor { + pub lookback_periods: usize, + /// Expected feature count (26=Wave A, 30=Wave A+4 extra, 36=Wave B, 65=Wave C) + expected_feature_count: usize, // ❌ Outdated comment - no Wave D (225) + price_history: Vec, + volume_history: Vec, + // ... +} +``` + +**Problem**: `MLFeatureExtractor` does NOT use `FeatureConfig` from `ml/src/features/config.rs` which supports Wave D: + +**File**: `ml/src/features/config.rs` (lines 345-355) +```rust +pub fn wave_d() -> Self { + Self { + enable_wave_a: true, + enable_wave_b: true, + enable_wave_c: true, + enable_wave_d_regime: true, // ✅ Enables 24 regime features (201→225) + // ... + } +} +``` + +**Root Cause**: Architecture mismatch between `common::ml_strategy` (legacy extractor) and `ml::features::config` (Wave D-aware). + +--- + +## Integration Gaps + +### Gap 1: No Wave D Feature Config +**Current**: `SharedMLStrategy::new(20, 0.6)` - hardcoded defaults +**Required**: Pass `FeatureConfig::wave_d()` to enable 225-feature extraction +**Blocker**: `SharedMLStrategy` constructor does NOT accept `FeatureConfig` + +**Solution**: +```rust +// Option A: Add new constructor +impl SharedMLStrategy { + pub fn new_with_feature_config( + lookback: usize, + confidence: f64, + feature_config: FeatureConfig, + ) -> Self { + // ... + } +} + +// Option B: Modify existing constructor +pub fn new( + lookback: usize, + confidence: f64, + feature_config: Option, +) -> Self { + let config = feature_config.unwrap_or(FeatureConfig::wave_a()); + // ... +} +``` + +--- + +### Gap 2: No Regime State Queries +**Current**: No database queries for `regime_states` or `regime_transitions` +**Required**: Query latest regime before position sizing decisions +**Blocker**: Database access exists (`self.db_pool`) but not used + +**Solution**: +```rust +async fn get_regime_for_symbol(&self, symbol: &str) -> Result { + let regime = sqlx::query!( + r#" + SELECT regime, confidence, detected_at + FROM get_latest_regime($1) + "#, + symbol + ) + .fetch_one(&self.db_pool) + .await + .context("Failed to fetch regime state")?; + + Ok(RegimeState { + regime: regime.regime.unwrap_or("Normal".to_string()), + confidence: regime.confidence.unwrap_or(0.0), + detected_at: regime.detected_at, + }) +} +``` + +--- + +### Gap 3: No Adaptive Position Sizing +**Current**: Fixed 1.0 contract size (line 570) +**Required**: Regime-adaptive sizing (0.2x-1.5x) + confidence-based Kelly multiplier +**Blocker**: Regime state not queried, Kelly logic not implemented + +**Solution**: +```rust +async fn calculate_adaptive_position_size( + &self, + prediction: &PendingPrediction, +) -> Result { + // Step 1: Get regime state + let regime = self.get_regime_for_symbol(&prediction.symbol).await?; + + // Step 2: Apply regime-adaptive multiplier (Wave D design) + let regime_multiplier = match regime.regime.as_str() { + "Trending" => 1.5, + "Ranging" => 0.8, + "Volatile" => 0.5, + "Transition" => 0.2, + _ => 1.0, + }; + + // Step 3: Apply confidence-based Kelly multiplier + // Kelly formula: f* = (p*b - q) / b + // For trading: simplified to linear confidence scaling + let confidence_factor = prediction.ensemble_confidence.clamp(0.6, 1.0); + let kelly_multiplier = (confidence_factor - 0.6) / 0.4; // 0.6→0.0, 1.0→1.0 + + // Step 4: Calculate final position size + let base_size = 1.0; // Base contract size + let adaptive_size = base_size * regime_multiplier * (1.0 + kelly_multiplier); + + // Step 5: Apply safety limits + Ok(adaptive_size.clamp(0.2, 5.0)) +} +``` + +--- + +## Testing Implications + +### Current Paper Trading Behavior +1. **Feature Set**: Uses Wave C baseline (201 features) - NO regime detection +2. **Position Sizing**: Fixed 1.0 contracts - NO adaptive sizing +3. **Regime Awareness**: None - trades blindly across all market conditions + +### Expected Wave D Behavior +1. **Feature Set**: 225 features (201 + 24 regime detection) +2. **Position Sizing**: 0.2x-1.5x adaptive multipliers based on regime +3. **Regime Awareness**: Queries `regime_states`, avoids transitions + +### Risk Assessment +⚠️ **HIGH RISK**: Paper trading will NOT validate Wave D features before production deployment. + +**Scenario**: If paper trading passes with Wave C config, we have NO evidence that: +- 225-feature extraction works in production +- Regime detection improves performance +- Adaptive sizing reduces drawdowns + +**Recommendation**: Block production deployment until paper trading uses Wave D config. + +--- + +## Action Items + +### Priority 1: Enable Wave D Features (2 hours) +- [ ] Modify `SharedMLStrategy::new()` to accept `FeatureConfig` parameter +- [ ] Update `paper_trading_executor.rs` to use `FeatureConfig::wave_d()` +- [ ] Verify 225-feature extraction in paper trading logs + +### Priority 2: Implement Regime Queries (1 hour) +- [ ] Add `get_regime_for_symbol()` method to `PaperTradingExecutor` +- [ ] Query `regime_states` table before each trade +- [ ] Log regime transitions for debugging + +### Priority 3: Adaptive Position Sizing (2 hours) +- [ ] Replace `calculate_position_size()` with `calculate_adaptive_position_size()` +- [ ] Implement regime multipliers (0.2x-1.5x) +- [ ] Add confidence-based Kelly multiplier +- [ ] Validate position size range (0.2-5.0 contracts) + +### Priority 4: Testing & Validation (1 hour) +- [ ] Run paper trading with ES.FUT, NQ.FUT for 24 hours +- [ ] Monitor regime transitions vs. position sizing +- [ ] Compare performance: Wave C baseline vs. Wave D adaptive +- [ ] Document results in `PAPER_TRADING_WAVE_D_VALIDATION.md` + +**Total Effort**: 6 hours + +--- + +## Technical Debt + +### Issue 1: Architecture Mismatch +**Problem**: `common::ml_strategy::MLFeatureExtractor` does NOT use `ml::features::config::FeatureConfig`. + +**Current State**: +- `MLFeatureExtractor` has hardcoded feature count expectations (comment: "26=Wave A, 36=Wave B, 65=Wave C") +- No mention of Wave D (225 features) +- No integration with `FeatureConfig::wave_d()` + +**Solution**: +```rust +// common/src/ml_strategy.rs +pub struct MLFeatureExtractor { + pub lookback_periods: usize, + feature_config: ml::features::config::FeatureConfig, // ✅ Use canonical config + price_history: Vec, + // ... +} + +impl MLFeatureExtractor { + pub fn new(lookback: usize, feature_config: FeatureConfig) -> Self { + Self { + lookback_periods: lookback, + feature_config, + // ... + } + } +} +``` + +**Blocker**: Cross-crate dependency (`common` depends on `ml`). + +--- + +### Issue 2: Kelly Criterion Stub +**Problem**: Line 569 comment says "could use Kelly Criterion" but NOT implemented. + +**Current Code**: +```rust +// In production, this could use Kelly Criterion or volatility-adjusted sizing +let position_size = 1.0; // 1 contract +``` + +**Required Implementation**: +```rust +use risk::kelly_sizing::{KellyResult, KellySizer}; + +async fn calculate_kelly_position(&self, prediction: &PendingPrediction) -> Result { + // Query historical performance for win rate + let win_rate = self.get_strategy_win_rate(&prediction.symbol).await?; + + // Use ensemble confidence as win probability + let win_prob = prediction.ensemble_confidence; + let loss_prob = 1.0 - win_prob; + + // Expected profit/loss ratio (from historical data) + let profit_loss_ratio = 1.5; // 1.5:1 risk/reward + + // Kelly formula: f* = (p*b - q) / b + let kelly_fraction = (win_prob * profit_loss_ratio - loss_prob) / profit_loss_ratio; + + // Use fractional Kelly (25%) for safety + let fractional_kelly = kelly_fraction * 0.25; + + Ok(fractional_kelly.clamp(0.0, 1.0)) +} +``` + +**Existing Code**: `services/trading_service/src/core/risk_manager.rs` has `KellySizer` but NOT used in paper trading. + +--- + +## References + +### Codebase Files +- `services/trading_service/src/paper_trading_executor.rs` (897 lines) +- `common/src/ml_strategy.rs` (MLFeatureExtractor definition) +- `ml/src/features/config.rs` (FeatureConfig::wave_d() implementation) +- `services/trading_service/src/services/trading.rs` (GetRegimeState gRPC method) +- `services/trading_service/src/core/risk_manager.rs` (KellySizer implementation) + +### Database Schema +- `migrations/045_regime_detection.sql` (regime_states, regime_transitions tables) +- Stored function: `get_latest_regime(symbol TEXT)` + +### Wave D Documentation +- `CLAUDE.md` (Wave D Phase 6 status, production targets) +- `WAVE_D_DEPLOYMENT_GUIDE.md` (regime detection integration guide) +- `WAVE_D_QUICK_REFERENCE.md` (adaptive sizing formulas) + +--- + +## Conclusion + +**Status**: ⚠️ **PARTIAL INTEGRATION - CRITICAL GAPS** + +The paper trading executor is architecturally sound (uses `SharedMLStrategy`, ONE SINGLE SYSTEM) but **NOT configured for Wave D testing**: + +1. ❌ No 225-feature extraction (stuck on Wave C baseline) +2. ❌ No regime state queries (blind to market conditions) +3. ❌ No adaptive position sizing (fixed 1.0 contracts) + +**Recommendation**: **BLOCK production deployment** until paper trading validates Wave D features. Implement action items (6 hours) and run 24-hour validation before proceeding. + +**Next Agent**: WIRE-15 should implement `calculate_adaptive_position_size()` with regime multipliers and Kelly logic. + +--- + +**Agent WIRE-14 signing off.** +**Mission**: PARTIAL - Integration gaps identified, action plan provided. +**Handoff**: WIRE-15 (Adaptive Position Sizing Implementation)