feat(wire-02): Document Wave D adaptive position sizer integration gap

CRITICAL FINDING: RegimeAdaptiveFeatures (Features 221-224) are fully implemented but NOT integrated into trading decision flow. Analysis Results: - ✅ RegimeAdaptiveFeatures: 644 lines, 12/12 tests passing - ✅ Database schema: regime_states, regime_transitions, adaptive_strategy_metrics - ✅ gRPC endpoints: GetRegimeState, GetRegimeTransitions defined - ❌ Trading Agent Service: NO regime integration in allocation.rs - ❌ Order Generation: NO stop-loss multiplier application Impact: - ML models train with regime features - Production trading IGNORES regime state - Position sizes remain STATIC (no 0.2x-1.5x adjustment) - Expected Sharpe improvement: 0% (instead of +25-50%) Integration Plan (11 hours): 1. Phase 1: Database query layer (2h) - regime.rs 2. Phase 2: Allocation integration (3h) - RegimeAdaptive method 3. Phase 3: Service wiring (2h) - RegimeDetector in service 4. Phase 4: Order generation (1h) - stop-loss multipliers 5. Phase 5: Testing (3h) - regime allocation tests Code Changes: - New files: regime.rs (200 lines), tests (300 lines) - Modified: allocation.rs (+100), service.rs (+50), orders.rs (+30) - Total: ~500 new lines, ~180 modified lines Performance: +3ms latency (acceptable for +25-50% Sharpe) Risk: Low (feature flag + 3-level rollback plan) Recommendation: PROCEED before 225-feature ML retraining Files: - AGENT_WIRE02_ADAPTIVE_SIZER_INTEGRATION.md (full analysis) - AGENT_WIRE02_QUICK_SUMMARY.md (executive summary) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-19 09:45:54 +02:00
parent 9504a3bd4b
commit 261bbef86e
4 changed files with 1483 additions and 0 deletions
--- a/AGENT_WIRE14_PAPER_TRADING_STATUS.md
+++ b/AGENT_WIRE14_PAPER_TRADING_STATUS.md
@@ -0,0 +1,427 @@
+# AGENT WIRE-14: Paper Trading Executor Wave D Integration Status
+
+**Agent**: WIRE-14
+**Mission**: Verify paper trading executor uses Wave D features and adaptive sizing
+**Status**: ⚠️ PARTIAL INTEGRATION - Missing Wave D Features
+**Priority**: HIGH - Paper trading must test Wave D before live deployment
+**Date**: 2025-10-19
+
+---
+
+## Executive Summary
+
+The paper trading executor (`services/trading_service/src/paper_trading_executor.rs`) currently uses `SharedMLStrategy` but is **NOT configured for Wave D features**. Critical gaps identified:
+
+1. ✅ Uses `SharedMLStrategy` (ONE SINGLE SYSTEM architecture)
+2. ❌ **NO Wave D feature configuration** - Uses hardcoded defaults (20 lookback, 0.6 confidence)
+3. ❌ **NO regime state queries** - Does not check `regime_states` table
+4. ❌ **NO adaptive position sizing** - Uses fixed 1.0 contract size
+5. ⚠️ **Kelly Criterion mentioned but not implemented** (line 569 comment only)
+
+**Risk**: Paper trading will test Wave C baseline (201 features) instead of Wave D (225 features + regime detection).
+
+---
+
+## Code Analysis
+
+### 1. ML Strategy Initialization
+
+**File**: `services/trading_service/src/paper_trading_executor.rs`
+**Lines**: 154-157
+
+```rust
+pub fn new(db_pool: PgPool, config: PaperTradingConfig) -> Self {
+    // Initialize with shared ML strategy (default configuration)
+    let ml_strategy = SharedMLStrategy::new(20, 0.6);
+    // ^^^ HARDCODED: 20 lookback, 0.6 confidence - NO Wave D config
+```
+
+**Issue**: `SharedMLStrategy::new()` does NOT accept `FeatureConfig` parameter. The constructor signature is:
+```rust
+pub fn new(lookback_periods: usize, min_confidence_threshold: f64) -> Self
+```
+
+**Missing**: No way to pass `FeatureConfig::wave_d()` to enable 225-feature extraction.
+
+---
+
+### 2. Position Sizing Logic
+
+**File**: `services/trading_service/src/paper_trading_executor.rs`
+**Lines**: 567-575
+
+```rust
+fn calculate_position_size(&self, _prediction: &PendingPrediction) -> Result<f64> {
+    // Simple fixed position size for paper trading
+    // In production, this could use Kelly Criterion or volatility-adjusted sizing
+    let position_size = 1.0; // 1 contract
+    // ^^^ FIXED SIZE: No adaptive sizing based on regime or confidence
+
+    if position_size > self.config.max_position_size {
+        return Err(anyhow!("Position size {} exceeds maximum {}", position_size, self.config.max_position_size));
+    }
+
+    Ok(position_size)
+}
+```
+
+**Missing Wave D Adaptive Logic**:
+- No regime state queries (`SELECT regime FROM regime_states`)
+- No adaptive multipliers (0.2x-1.5x based on regime)
+- No Kelly Criterion position sizing
+- No volatility-adjusted sizing
+
+**Expected Behavior** (from Wave D design):
+```rust
+// Query regime state
+let regime = sqlx::query!("SELECT regime FROM get_latest_regime($1)", symbol)
+    .fetch_one(&self.db_pool).await?;
+
+// Apply regime-adaptive multiplier
+let base_size = 1.0;
+let regime_multiplier = match regime.regime.as_str() {
+    "Trending" => 1.5,      // Increase size in trending markets
+    "Ranging" => 0.8,       // Reduce size in ranging markets
+    "Volatile" => 0.5,      // Minimize size in volatile markets
+    "Transition" => 0.2,    // Avoid trading during transitions
+    _ => 1.0,               // Normal sizing for unknown regimes
+};
+
+let position_size = base_size * regime_multiplier * confidence_factor;
+```
+
+---
+
+### 3. Regime State Integration
+
+**Search Results**: ❌ NO regime queries found in `paper_trading_executor.rs`
+
+```bash
+$ grep -rn "regime_states\|regime_transitions\|get_latest_regime" \
+  services/trading_service/src/paper_trading_executor.rs
+
+# Result: 0 matches
+```
+
+**Contrast with `trading.rs` (Trading Service)**:
+```rust
+// services/trading_service/src/services/trading.rs:992-1023
+async fn get_regime_state(&self, req: Request<GetRegimeStateRequest>) -> Result<Response<GetRegimeStateResponse>, Status> {
+    let regime_state = sqlx::query!(
+        r#"SELECT regime, confidence, detected_at FROM get_latest_regime($1)"#,
+        req.symbol
+    ).fetch_one(&self.db_pool).await?;
+
+    Ok(Response::new(GetRegimeStateResponse {
+        current_regime: regime_state.regime.unwrap_or("Normal".to_string()),
+        confidence: regime_state.confidence.unwrap_or(0.0),
+        // ...
+    }))
+}
+```
+
+**Paper Trading Executor**: No equivalent logic.
+
+---
+
+### 4. Feature Configuration Architecture
+
+**Analysis**: `SharedMLStrategy` uses `MLFeatureExtractor` which has a **legacy field** for feature count:
+
+**File**: `common/src/ml_strategy.rs` (lines 66-84)
+```rust
+pub struct MLFeatureExtractor {
+    pub lookback_periods: usize,
+    /// Expected feature count (26=Wave A, 30=Wave A+4 extra, 36=Wave B, 65=Wave C)
+    expected_feature_count: usize,  // ❌ Outdated comment - no Wave D (225)
+    price_history: Vec<f64>,
+    volume_history: Vec<f64>,
+    // ...
+}
+```
+
+**Problem**: `MLFeatureExtractor` does NOT use `FeatureConfig` from `ml/src/features/config.rs` which supports Wave D:
+
+**File**: `ml/src/features/config.rs` (lines 345-355)
+```rust
+pub fn wave_d() -> Self {
+    Self {
+        enable_wave_a: true,
+        enable_wave_b: true,
+        enable_wave_c: true,
+        enable_wave_d_regime: true,  // ✅ Enables 24 regime features (201→225)
+        // ...
+    }
+}
+```
+
+**Root Cause**: Architecture mismatch between `common::ml_strategy` (legacy extractor) and `ml::features::config` (Wave D-aware).
+
+---
+
+## Integration Gaps
+
+### Gap 1: No Wave D Feature Config
+**Current**: `SharedMLStrategy::new(20, 0.6)` - hardcoded defaults
+**Required**: Pass `FeatureConfig::wave_d()` to enable 225-feature extraction
+**Blocker**: `SharedMLStrategy` constructor does NOT accept `FeatureConfig`
+
+**Solution**:
+```rust
+// Option A: Add new constructor
+impl SharedMLStrategy {
+    pub fn new_with_feature_config(
+        lookback: usize,
+        confidence: f64,
+        feature_config: FeatureConfig,
+    ) -> Self {
+        // ...
+    }
+}
+
+// Option B: Modify existing constructor
+pub fn new(
+    lookback: usize,
+    confidence: f64,
+    feature_config: Option<FeatureConfig>,
+) -> Self {
+    let config = feature_config.unwrap_or(FeatureConfig::wave_a());
+    // ...
+}
+```
+
+---
+
+### Gap 2: No Regime State Queries
+**Current**: No database queries for `regime_states` or `regime_transitions`
+**Required**: Query latest regime before position sizing decisions
+**Blocker**: Database access exists (`self.db_pool`) but not used
+
+**Solution**:
+```rust
+async fn get_regime_for_symbol(&self, symbol: &str) -> Result<RegimeState> {
+    let regime = sqlx::query!(
+        r#"
+        SELECT regime, confidence, detected_at
+        FROM get_latest_regime($1)
+        "#,
+        symbol
+    )
+    .fetch_one(&self.db_pool)
+    .await
+    .context("Failed to fetch regime state")?;
+
+    Ok(RegimeState {
+        regime: regime.regime.unwrap_or("Normal".to_string()),
+        confidence: regime.confidence.unwrap_or(0.0),
+        detected_at: regime.detected_at,
+    })
+}
+```
+
+---
+
+### Gap 3: No Adaptive Position Sizing
+**Current**: Fixed 1.0 contract size (line 570)
+**Required**: Regime-adaptive sizing (0.2x-1.5x) + confidence-based Kelly multiplier
+**Blocker**: Regime state not queried, Kelly logic not implemented
+
+**Solution**:
+```rust
+async fn calculate_adaptive_position_size(
+    &self,
+    prediction: &PendingPrediction,
+) -> Result<f64> {
+    // Step 1: Get regime state
+    let regime = self.get_regime_for_symbol(&prediction.symbol).await?;
+
+    // Step 2: Apply regime-adaptive multiplier (Wave D design)
+    let regime_multiplier = match regime.regime.as_str() {
+        "Trending" => 1.5,
+        "Ranging" => 0.8,
+        "Volatile" => 0.5,
+        "Transition" => 0.2,
+        _ => 1.0,
+    };
+
+    // Step 3: Apply confidence-based Kelly multiplier
+    // Kelly formula: f* = (p*b - q) / b
+    // For trading: simplified to linear confidence scaling
+    let confidence_factor = prediction.ensemble_confidence.clamp(0.6, 1.0);
+    let kelly_multiplier = (confidence_factor - 0.6) / 0.4; // 0.6→0.0, 1.0→1.0
+
+    // Step 4: Calculate final position size
+    let base_size = 1.0; // Base contract size
+    let adaptive_size = base_size * regime_multiplier * (1.0 + kelly_multiplier);
+
+    // Step 5: Apply safety limits
+    Ok(adaptive_size.clamp(0.2, 5.0))
+}
+```
+
+---
+
+## Testing Implications
+
+### Current Paper Trading Behavior
+1. **Feature Set**: Uses Wave C baseline (201 features) - NO regime detection
+2. **Position Sizing**: Fixed 1.0 contracts - NO adaptive sizing
+3. **Regime Awareness**: None - trades blindly across all market conditions
+
+### Expected Wave D Behavior
+1. **Feature Set**: 225 features (201 + 24 regime detection)
+2. **Position Sizing**: 0.2x-1.5x adaptive multipliers based on regime
+3. **Regime Awareness**: Queries `regime_states`, avoids transitions
+
+### Risk Assessment
+⚠️ **HIGH RISK**: Paper trading will NOT validate Wave D features before production deployment.
+
+**Scenario**: If paper trading passes with Wave C config, we have NO evidence that:
+- 225-feature extraction works in production
+- Regime detection improves performance
+- Adaptive sizing reduces drawdowns
+
+**Recommendation**: Block production deployment until paper trading uses Wave D config.
+
+---
+
+## Action Items
+
+### Priority 1: Enable Wave D Features (2 hours)
+- [ ] Modify `SharedMLStrategy::new()` to accept `FeatureConfig` parameter
+- [ ] Update `paper_trading_executor.rs` to use `FeatureConfig::wave_d()`
+- [ ] Verify 225-feature extraction in paper trading logs
+
+### Priority 2: Implement Regime Queries (1 hour)
+- [ ] Add `get_regime_for_symbol()` method to `PaperTradingExecutor`
+- [ ] Query `regime_states` table before each trade
+- [ ] Log regime transitions for debugging
+
+### Priority 3: Adaptive Position Sizing (2 hours)
+- [ ] Replace `calculate_position_size()` with `calculate_adaptive_position_size()`
+- [ ] Implement regime multipliers (0.2x-1.5x)
+- [ ] Add confidence-based Kelly multiplier
+- [ ] Validate position size range (0.2-5.0 contracts)
+
+### Priority 4: Testing & Validation (1 hour)
+- [ ] Run paper trading with ES.FUT, NQ.FUT for 24 hours
+- [ ] Monitor regime transitions vs. position sizing
+- [ ] Compare performance: Wave C baseline vs. Wave D adaptive
+- [ ] Document results in `PAPER_TRADING_WAVE_D_VALIDATION.md`
+
+**Total Effort**: 6 hours
+
+---
+
+## Technical Debt
+
+### Issue 1: Architecture Mismatch
+**Problem**: `common::ml_strategy::MLFeatureExtractor` does NOT use `ml::features::config::FeatureConfig`.
+
+**Current State**:
+- `MLFeatureExtractor` has hardcoded feature count expectations (comment: "26=Wave A, 36=Wave B, 65=Wave C")
+- No mention of Wave D (225 features)
+- No integration with `FeatureConfig::wave_d()`
+
+**Solution**:
+```rust
+// common/src/ml_strategy.rs
+pub struct MLFeatureExtractor {
+    pub lookback_periods: usize,
+    feature_config: ml::features::config::FeatureConfig, // ✅ Use canonical config
+    price_history: Vec<f64>,
+    // ...
+}
+
+impl MLFeatureExtractor {
+    pub fn new(lookback: usize, feature_config: FeatureConfig) -> Self {
+        Self {
+            lookback_periods: lookback,
+            feature_config,
+            // ...
+        }
+    }
+}
+```
+
+**Blocker**: Cross-crate dependency (`common` depends on `ml`).
+
+---
+
+### Issue 2: Kelly Criterion Stub
+**Problem**: Line 569 comment says "could use Kelly Criterion" but NOT implemented.
+
+**Current Code**:
+```rust
+// In production, this could use Kelly Criterion or volatility-adjusted sizing
+let position_size = 1.0; // 1 contract
+```
+
+**Required Implementation**:
+```rust
+use risk::kelly_sizing::{KellyResult, KellySizer};
+
+async fn calculate_kelly_position(&self, prediction: &PendingPrediction) -> Result<f64> {
+    // Query historical performance for win rate
+    let win_rate = self.get_strategy_win_rate(&prediction.symbol).await?;
+
+    // Use ensemble confidence as win probability
+    let win_prob = prediction.ensemble_confidence;
+    let loss_prob = 1.0 - win_prob;
+
+    // Expected profit/loss ratio (from historical data)
+    let profit_loss_ratio = 1.5; // 1.5:1 risk/reward
+
+    // Kelly formula: f* = (p*b - q) / b
+    let kelly_fraction = (win_prob * profit_loss_ratio - loss_prob) / profit_loss_ratio;
+
+    // Use fractional Kelly (25%) for safety
+    let fractional_kelly = kelly_fraction * 0.25;
+
+    Ok(fractional_kelly.clamp(0.0, 1.0))
+}
+```
+
+**Existing Code**: `services/trading_service/src/core/risk_manager.rs` has `KellySizer` but NOT used in paper trading.
+
+---
+
+## References
+
+### Codebase Files
+- `services/trading_service/src/paper_trading_executor.rs` (897 lines)
+- `common/src/ml_strategy.rs` (MLFeatureExtractor definition)
+- `ml/src/features/config.rs` (FeatureConfig::wave_d() implementation)
+- `services/trading_service/src/services/trading.rs` (GetRegimeState gRPC method)
+- `services/trading_service/src/core/risk_manager.rs` (KellySizer implementation)
+
+### Database Schema
+- `migrations/045_regime_detection.sql` (regime_states, regime_transitions tables)
+- Stored function: `get_latest_regime(symbol TEXT)`
+
+### Wave D Documentation
+- `CLAUDE.md` (Wave D Phase 6 status, production targets)
+- `WAVE_D_DEPLOYMENT_GUIDE.md` (regime detection integration guide)
+- `WAVE_D_QUICK_REFERENCE.md` (adaptive sizing formulas)
+
+---
+
+## Conclusion
+
+**Status**: ⚠️ **PARTIAL INTEGRATION - CRITICAL GAPS**
+
+The paper trading executor is architecturally sound (uses `SharedMLStrategy`, ONE SINGLE SYSTEM) but **NOT configured for Wave D testing**:
+
+1. ❌ No 225-feature extraction (stuck on Wave C baseline)
+2. ❌ No regime state queries (blind to market conditions)
+3. ❌ No adaptive position sizing (fixed 1.0 contracts)
+
+**Recommendation**: **BLOCK production deployment** until paper trading validates Wave D features. Implement action items (6 hours) and run 24-hour validation before proceeding.
+
+**Next Agent**: WIRE-15 should implement `calculate_adaptive_position_size()` with regime multipliers and Kelly logic.
+
+---
+
+**Agent WIRE-14 signing off.**
+**Mission**: PARTIAL - Integration gaps identified, action plan provided.
+**Handoff**: WIRE-15 (Adaptive Position Sizing Implementation)