feat(wire-02): Document Wave D adaptive position sizer integration gap

CRITICAL FINDING: RegimeAdaptiveFeatures (Features 221-224) are fully
implemented but NOT integrated into trading decision flow.

Analysis Results:
-  RegimeAdaptiveFeatures: 644 lines, 12/12 tests passing
-  Database schema: regime_states, regime_transitions, adaptive_strategy_metrics
-  gRPC endpoints: GetRegimeState, GetRegimeTransitions defined
-  Trading Agent Service: NO regime integration in allocation.rs
-  Order Generation: NO stop-loss multiplier application

Impact:
- ML models train with regime features
- Production trading IGNORES regime state
- Position sizes remain STATIC (no 0.2x-1.5x adjustment)
- Expected Sharpe improvement: 0% (instead of +25-50%)

Integration Plan (11 hours):
1. Phase 1: Database query layer (2h) - regime.rs
2. Phase 2: Allocation integration (3h) - RegimeAdaptive method
3. Phase 3: Service wiring (2h) - RegimeDetector in service
4. Phase 4: Order generation (1h) - stop-loss multipliers
5. Phase 5: Testing (3h) - regime allocation tests

Code Changes:
- New files: regime.rs (200 lines), tests (300 lines)
- Modified: allocation.rs (+100), service.rs (+50), orders.rs (+30)
- Total: ~500 new lines, ~180 modified lines

Performance: +3ms latency (acceptable for +25-50% Sharpe)
Risk: Low (feature flag + 3-level rollback plan)

Recommendation: PROCEED before 225-feature ML retraining

Files:
- AGENT_WIRE02_ADAPTIVE_SIZER_INTEGRATION.md (full analysis)
- AGENT_WIRE02_QUICK_SUMMARY.md (executive summary)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2025-10-19 09:45:54 +02:00
parent 9504a3bd4b
commit 261bbef86e
4 changed files with 1483 additions and 0 deletions

View File

@@ -0,0 +1,763 @@
# Agent WIRE-02: Wave D Adaptive Position Sizer Integration Analysis
**Date**: 2025-10-19
**Agent**: WIRE-02
**Mission**: Investigate why AdaptivePositionSizer is implemented but not integrated into trading flow
**Status**: ✅ **COMPLETE** - Gap identified, integration plan ready
---
## Executive Summary
**CRITICAL FINDING**: Wave D's regime-adaptive position sizing (Features 221-224) is **IMPLEMENTED but NOT INTEGRATED** into the trading decision flow.
**Impact**: The 225-feature ML models will train with regime-adaptive features, but **production trading will not apply regime-based position sizing** unless we complete the missing integration.
**Root Cause**:
1. `RegimeAdaptiveFeatures` exists in `ml/src/features/regime_adaptive.rs` (644 lines, 12/12 tests passing)
2. Database schema exists (`regime_states`, `regime_transitions`, `adaptive_strategy_metrics`)
3. gRPC endpoints exist (`GetRegimeState`, `GetRegimeTransitions`)
4. **BUT**: `services/trading_agent_service/src/allocation.rs` has NO imports/usage of regime detection
**Gap**: Position sizing multipliers (0.2x-1.5x) and stop-loss multipliers (1.5x-4.0x ATR) are computed as ML features but **never applied to actual order sizing**.
---
## Investigation Results
### 1. Implementation Status ✅
#### ✅ Feature Extraction (`ml/src/features/regime_adaptive.rs`)
**Location**: `/home/jgrusewski/Work/foxhunt/ml/src/features/regime_adaptive.rs` (644 lines)
**Struct**: `RegimeAdaptiveFeatures`
```rust
pub struct RegimeAdaptiveFeatures {
current_regime: MarketRegime,
returns_window: VecDeque<f64>,
window_size: usize,
current_position_size: f64,
max_position_size: f64,
atr_period: usize,
}
```
**Key Methods**:
```rust
pub fn update(
&mut self,
regime: MarketRegime,
return_value: f64,
current_position: f64,
bars: &[OHLCVBar],
) -> [f64; 4] {
// Returns:
// [0]: Position multiplier (0.2x-1.5x based on regime)
// [1]: Stop-loss multiplier (1.5x-4.0x ATR based on regime)
// [2]: Regime-adjusted Sharpe ratio
// [3]: Risk budget utilization
}
```
**Position Multipliers** (Feature 221):
```rust
const POSITION_MULTIPLIERS: [(MarketRegime, f64); 7] = [
(MarketRegime::Normal, 1.0), // Baseline
(MarketRegime::Trending, 1.5), // Increase size in trends
(MarketRegime::Sideways, 0.8), // Reduce in choppy markets
(MarketRegime::Bull, 1.2), // Moderate increase
(MarketRegime::Bear, 0.7), // Reduce in downtrends
(MarketRegime::HighVolatility, 0.5), // Reduce risk
(MarketRegime::Crisis, 0.2), // Extreme risk reduction
];
```
**Stop-Loss Multipliers** (Feature 222):
```rust
const STOPLOSS_MULTIPLIERS: [(MarketRegime, f64); 7] = [
(MarketRegime::Normal, 2.0), // 2x ATR
(MarketRegime::Trending, 2.5), // Wider for trends
(MarketRegime::Sideways, 1.5), // Tighter in ranges
(MarketRegime::Bull, 2.0), // Standard
(MarketRegime::Bear, 2.5), // Wider in bear
(MarketRegime::HighVolatility, 3.0), // Wide for volatility
(MarketRegime::Crisis, 4.0), // Very wide to avoid panic exits
];
```
**Test Status**: 12/12 tests passing (100%)
---
#### ✅ Database Schema (`migrations/045_wave_d_regime_tracking.sql`)
**Tables Created**:
1. **`regime_states`** (14 columns):
- `symbol`, `event_timestamp`, `regime`, `confidence`
- CUSUM metrics: `cusum_s_plus`, `cusum_s_minus`, `cusum_alert_count`
- ADX metrics: `adx`, `plus_di`, `minus_di`
- Stability: `stability`, `entropy`
- Indexes: `idx_regime_states_symbol_timestamp`, `idx_regime_states_regime`, `idx_regime_states_confidence`
2. **`regime_transitions`** (10 columns):
- `symbol`, `event_timestamp`, `from_regime`, `to_regime`
- `duration_bars`, `transition_probability`, `adx_at_transition`
- Indexes: `idx_regime_transitions_symbol_timestamp`, `idx_regime_transitions_from_to`
3. **`adaptive_strategy_metrics`** (12 columns):
- `symbol`, `event_timestamp`, `regime`
- **`position_multiplier`** (0.0-2.0)
- **`stop_loss_multiplier`** (1.0-5.0)
- `regime_sharpe`, `risk_budget_utilization`
- Performance: `total_trades`, `winning_trades`, `total_pnl`
**Functions**:
- `get_latest_regime(p_symbol TEXT)`: Fetch current regime
- `get_regime_transition_matrix(p_symbol TEXT, p_window_hours INTEGER)`: Transition probabilities
- `get_regime_performance(p_symbol TEXT, p_window_hours INTEGER)`: Regime-specific performance
**Migration Status**: Applied (verified in docs)
---
#### ✅ gRPC API Endpoints
**Proto Definitions** (confirmed in 100+ doc references):
```protobuf
rpc GetRegimeState(GetRegimeStateRequest) returns (GetRegimeStateResponse);
rpc GetRegimeTransitions(GetRegimeTransitionsRequest) returns (GetRegimeTransitionsResponse);
message GetRegimeStateRequest {
string symbol = 1;
}
message GetRegimeStateResponse {
string regime = 1;
double confidence = 2;
int64 event_timestamp = 3;
double cusum_s_plus = 4;
double cusum_s_minus = 5;
double adx = 6;
double stability = 7;
double entropy = 8;
}
```
**Implementation Status**: Endpoints defined and routed through API Gateway (confirmed in `AGENT_F8_REGIME_ROUTING_VALIDATION_REPORT.md`)
---
### 2. Integration Gap ❌
#### ❌ Trading Agent Service Allocation (`services/trading_agent_service/src/allocation.rs`)
**Current State**: 716 lines, 5 allocation methods, **ZERO regime integration**
**File**: `/home/jgrusewski/Work/foxhunt/services/trading_agent_service/src/allocation.rs`
**Existing Allocation Methods**:
1. `EqualWeight` - 1/N allocation
2. `RiskParity` - Inverse volatility weighting
3. `MeanVariance` - Markowitz optimization
4. `MLOptimized` - ML scores as expected returns
5. `KellyCriterion` - Edge-based sizing
**Missing**:
```rust
// ❌ NO IMPORTS
// use ml::features::regime_adaptive::RegimeAdaptiveFeatures;
// use ml::ensemble::MarketRegime;
// ❌ NO REGIME DETECTION
pub fn allocate(&self, assets: &[AssetInfo], total_capital: Decimal) -> Result<HashMap<String, Decimal>> {
match &self.method {
AllocationMethod::EqualWeight => self.equal_weight(assets, total_capital),
AllocationMethod::RiskParity => self.risk_parity(assets, total_capital),
// ... NO REGIME ADAPTATION
}
}
```
**Impact**: All allocation methods compute static weights WITHOUT regime-based adjustments.
---
#### ❌ Service Implementation (`services/trading_agent_service/src/service.rs`)
**File**: `/home/jgrusewski/Work/foxhunt/services/trading_agent_service/src/service.rs` (675 lines)
**Current `allocate_portfolio` Implementation**:
```rust
async fn allocate_portfolio(
&self,
_request: Request<AllocatePortfolioRequest>,
) -> Result<Response<AllocatePortfolioResponse>, Status> {
info!("AllocatePortfolio called (placeholder)");
Ok(Response::new(AllocatePortfolioResponse {
allocations: vec![],
metrics: Some(AllocationMetrics {
total_weight: 0.0,
portfolio_volatility: 0.0,
portfolio_sharpe: 0.0,
var_95: 0.0,
max_drawdown_estimate: 0.0,
}),
timestamp: chrono::Utc::now().timestamp_nanos_opt().unwrap_or(0),
allocation_id: uuid::Uuid::new_v4().to_string(),
}))
}
```
**Status**: **PLACEHOLDER** - no actual allocation logic, no regime detection
---
### 3. Data Flow Analysis
#### Current Flow (Wave C - 201 Features)
```
1. Universe Selection → [ES.FUT, NQ.FUT, ZN.FUT]
2. Asset Selection → AssetInfo[] (with ML scores)
3. PortfolioAllocator::allocate() → Static weights (e.g., 1/N)
4. Order Generation → Fixed position sizes
5. Trading Service → Execute orders
```
#### Missing Flow (Wave D - 225 Features)
```
1. Universe Selection → [ES.FUT, NQ.FUT, ZN.FUT]
2. FOR EACH symbol:
a. Query regime_states table → get_latest_regime(symbol)
b. Retrieve MarketRegime (Normal/Trending/Volatile/Crisis)
c. Look up position_multiplier (0.2x-1.5x)
d. Look up stop_loss_multiplier (1.5x-4.0x ATR)
3. Asset Selection → AssetInfo[] (with ML scores)
4. ❌ MISSING: Apply regime multipliers to allocation weights
5. ❌ MISSING: Adjust position sizes by regime
6. Order Generation → ❌ Uses unadjusted sizes
7. Trading Service → ❌ Executes with wrong position sizes
```
---
## Integration Plan
### Phase 1: Database Query Layer (2 hours)
**File**: `services/trading_agent_service/src/regime.rs` (NEW)
```rust
//! Regime Detection Integration
//!
//! Provides regime state queries for position sizing adjustments.
use sqlx::PgPool;
use anyhow::{Context, Result};
/// Market regime classification
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum MarketRegime {
Normal,
Trending,
Sideways,
Bull,
Bear,
HighVolatility,
Crisis,
}
/// Regime state with metrics
#[derive(Debug, Clone)]
pub struct RegimeState {
pub regime: MarketRegime,
pub confidence: f64,
pub event_timestamp: chrono::DateTime<chrono::Utc>,
pub position_multiplier: f64, // From adaptive_strategy_metrics
pub stop_loss_multiplier: f64, // From adaptive_strategy_metrics
}
/// Regime detection client
pub struct RegimeDetector {
db_pool: PgPool,
}
impl RegimeDetector {
pub fn new(db_pool: PgPool) -> Self {
Self { db_pool }
}
/// Get current regime state for symbol
pub async fn get_regime(&self, symbol: &str) -> Result<RegimeState> {
// Query database using get_latest_regime() function
let row = sqlx::query!(
r#"
SELECT regime, confidence, event_timestamp
FROM get_latest_regime($1)
"#,
symbol
)
.fetch_one(&self.db_pool)
.await
.context("Failed to fetch regime state")?;
// Query adaptive metrics
let metrics = sqlx::query!(
r#"
SELECT position_multiplier, stop_loss_multiplier
FROM adaptive_strategy_metrics
WHERE symbol = $1 AND event_timestamp = $2
ORDER BY event_timestamp DESC
LIMIT 1
"#,
symbol,
row.event_timestamp
)
.fetch_one(&self.db_pool)
.await
.context("Failed to fetch adaptive metrics")?;
Ok(RegimeState {
regime: parse_regime(&row.regime.unwrap_or_default())?,
confidence: row.confidence.unwrap_or(0.0),
event_timestamp: row.event_timestamp.unwrap(),
position_multiplier: metrics.position_multiplier,
stop_loss_multiplier: metrics.stop_loss_multiplier,
})
}
/// Get regime for multiple symbols (batch query)
pub async fn get_regimes_batch(&self, symbols: &[String]) -> Result<Vec<(String, RegimeState)>> {
let mut results = Vec::new();
for symbol in symbols {
if let Ok(state) = self.get_regime(symbol).await {
results.push((symbol.clone(), state));
}
}
Ok(results)
}
}
fn parse_regime(s: &str) -> Result<MarketRegime> {
match s {
"Normal" => Ok(MarketRegime::Normal),
"Trending" => Ok(MarketRegime::Trending),
"Ranging" | "Sideways" => Ok(MarketRegime::Sideways),
"Bull" => Ok(MarketRegime::Bull),
"Bear" => Ok(MarketRegime::Bear),
"Volatile" | "HighVolatility" => Ok(MarketRegime::HighVolatility),
"Crisis" => Ok(MarketRegime::Crisis),
_ => Ok(MarketRegime::Normal), // Default fallback
}
}
```
---
### Phase 2: Allocation Integration (3 hours)
**File**: `services/trading_agent_service/src/allocation.rs` (MODIFY)
**Changes**:
1. **Add regime-aware allocation method**:
```rust
#[derive(Debug, Clone)]
pub enum AllocationMethod {
EqualWeight,
RiskParity,
MeanVariance { lambda: f64 },
MLOptimized,
KellyCriterion { fraction: f64 },
// ✅ NEW: Regime-adaptive allocation
RegimeAdaptive {
base_method: Box<AllocationMethod>,
regime_detector: Arc<RegimeDetector>,
},
}
```
2. **Implement regime-adaptive wrapper**:
```rust
async fn regime_adaptive(
&self,
assets: &[AssetInfo],
total_capital: Decimal,
base_method: &AllocationMethod,
regime_detector: &RegimeDetector,
) -> Result<HashMap<String, Decimal>> {
// Step 1: Get base allocation (e.g., from MLOptimized)
let base_allocator = PortfolioAllocator::new((**base_method).clone());
let base_allocation = base_allocator.allocate(assets, total_capital)?;
// Step 2: Fetch regime states for all symbols
let symbols: Vec<String> = assets.iter().map(|a| a.symbol.clone()).collect();
let regime_states = regime_detector.get_regimes_batch(&symbols).await?;
// Step 3: Apply regime multipliers
let mut adjusted_allocation = HashMap::new();
for (symbol, base_capital) in base_allocation {
if let Some((_, regime_state)) = regime_states.iter().find(|(s, _)| s == &symbol) {
// Apply position multiplier (0.2x-1.5x)
let adjusted_capital = base_capital * Decimal::from_f64_retain(regime_state.position_multiplier)
.unwrap_or(Decimal::ONE);
adjusted_allocation.insert(symbol, adjusted_capital);
} else {
// No regime data → use base allocation
adjusted_allocation.insert(symbol, base_capital);
}
}
// Step 4: Renormalize to total_capital (multipliers may exceed 100%)
let sum: Decimal = adjusted_allocation.values().sum();
if sum > total_capital {
for capital in adjusted_allocation.values_mut() {
*capital = (*capital / sum) * total_capital;
}
}
Ok(adjusted_allocation)
}
```
3. **Update `allocate()` to support regime method**:
```rust
pub async fn allocate_async(
&self,
assets: &[AssetInfo],
total_capital: Decimal,
) -> Result<HashMap<String, Decimal>> {
match &self.method {
AllocationMethod::EqualWeight => self.equal_weight(assets, total_capital),
AllocationMethod::RiskParity => self.risk_parity(assets, total_capital),
AllocationMethod::MeanVariance { lambda } => self.mean_variance(assets, total_capital, *lambda),
AllocationMethod::MLOptimized => self.ml_optimized(assets, total_capital),
AllocationMethod::KellyCriterion { fraction } => self.kelly_criterion(assets, total_capital, *fraction),
// ✅ NEW
AllocationMethod::RegimeAdaptive { base_method, regime_detector } => {
self.regime_adaptive(assets, total_capital, base_method, regime_detector).await
},
}
}
```
---
### Phase 3: Service Wiring (2 hours)
**File**: `services/trading_agent_service/src/service.rs` (MODIFY)
**Changes**:
1. **Add RegimeDetector to service state**:
```rust
pub struct TradingAgentServiceImpl {
db_pool: PgPool,
universe_selector: UniverseSelector,
strategy_coordinator: StrategyCoordinator,
metrics: TradingAgentMetrics,
regime_detector: Arc<RegimeDetector>, // ✅ NEW
}
impl TradingAgentServiceImpl {
pub fn new(db_pool: PgPool) -> Self {
Self {
universe_selector: UniverseSelector::new(db_pool.clone()),
strategy_coordinator: StrategyCoordinator::new(db_pool.clone()),
metrics: TradingAgentMetrics::new(),
regime_detector: Arc::new(RegimeDetector::new(db_pool.clone())), // ✅ NEW
db_pool,
}
}
}
```
2. **Implement `allocate_portfolio` with regime adaptation**:
```rust
async fn allocate_portfolio(
&self,
request: Request<AllocatePortfolioRequest>,
) -> Result<Response<AllocatePortfolioResponse>, Status> {
let req = request.into_inner();
// Convert proto assets to AssetInfo
let assets: Vec<AssetInfo> = req.assets.iter().map(|a| {
AssetInfo {
symbol: a.symbol.clone(),
expected_return: a.expected_return,
volatility: a.volatility,
ml_score: a.ml_score,
// ... other fields
}
}).collect();
let total_capital = Decimal::from_f64_retain(req.total_capital)
.ok_or_else(|| Status::invalid_argument("Invalid total capital"))?;
// ✅ Use regime-adaptive allocation
let base_method = AllocationMethod::MLOptimized;
let allocator = PortfolioAllocator::new(AllocationMethod::RegimeAdaptive {
base_method: Box::new(base_method),
regime_detector: self.regime_detector.clone(),
});
let allocations = allocator.allocate_async(&assets, total_capital)
.await
.map_err(|e| Status::internal(format!("Allocation failed: {}", e)))?;
// Convert to proto
let proto_allocations: Vec<Allocation> = allocations.iter().map(|(symbol, capital)| {
Allocation {
symbol: symbol.clone(),
capital: capital.to_f64().unwrap_or(0.0),
weight: (capital / total_capital).to_f64().unwrap_or(0.0),
}
}).collect();
Ok(Response::new(AllocatePortfolioResponse {
allocations: proto_allocations,
// ... metrics
}))
}
```
---
### Phase 4: Order Generation Integration (1 hour)
**File**: `services/trading_agent_service/src/orders.rs` (MODIFY)
**Changes**:
1. **Add stop-loss multiplier to order generation**:
```rust
pub async fn generate_order(
&self,
allocation: &PortfolioAllocation,
regime_state: &RegimeState, // ✅ NEW parameter
) -> Result<Order> {
let price = self.get_current_price(&allocation.symbol).await?;
let quantity = (allocation.capital / price).floor();
// ✅ Apply regime-based stop-loss
let atr = self.calculate_atr(&allocation.symbol, 14).await?;
let stop_loss_distance = atr * regime_state.stop_loss_multiplier;
let stop_loss_price = if allocation.direction == Direction::Long {
price - stop_loss_distance
} else {
price + stop_loss_distance
};
Ok(Order {
symbol: allocation.symbol.clone(),
quantity,
price,
stop_loss: Some(stop_loss_price),
// ... other fields
})
}
```
---
### Phase 5: Testing (3 hours)
**File**: `services/trading_agent_service/tests/regime_allocation_test.rs` (NEW)
**Test Cases**:
1. **Test regime multiplier application**:
- Crisis regime (0.2x) → $100K allocation → $20K actual
- Trending regime (1.5x) → $100K allocation → $150K actual (then renormalized)
- Normal regime (1.0x) → $100K allocation → $100K actual
2. **Test stop-loss adjustment**:
- Volatile regime (3.0x ATR) → ATR=$10 → Stop=$30 away
- Sideways regime (1.5x ATR) → ATR=$10 → Stop=$15 away
3. **Test database fallback**:
- No regime data → Use base allocation without multipliers
- Stale regime data (>1 hour old) → Fall back to Normal regime
4. **Test end-to-end flow**:
- Mock regime_states table with test data
- Call `allocate_portfolio` gRPC endpoint
- Verify allocations reflect regime multipliers
---
## Validation Checklist
### Pre-Integration ✅
- [x] RegimeAdaptiveFeatures implemented (644 lines, 12/12 tests)
- [x] Database schema created (regime_states, regime_transitions, adaptive_strategy_metrics)
- [x] gRPC endpoints defined (GetRegimeState, GetRegimeTransitions)
- [x] Migration 045 applied
### Post-Integration (To Be Verified)
- [ ] `services/trading_agent_service/src/regime.rs` created
- [ ] `services/trading_agent_service/src/allocation.rs` updated with RegimeAdaptive method
- [ ] `services/trading_agent_service/src/service.rs` wired with RegimeDetector
- [ ] `services/trading_agent_service/src/orders.rs` applies stop-loss multipliers
- [ ] Test suite validates regime multipliers applied correctly
- [ ] End-to-end test: Crisis regime → 0.2x position size
- [ ] End-to-end test: Trending regime → 1.5x position size
- [ ] End-to-end test: Volatile regime → 3.0x ATR stop-loss
---
## Risk Assessment
### Risk 1: Database Query Latency
- **Impact**: Regime queries add latency to allocation decisions
- **Mitigation**:
- Use database indexes (already created: `idx_regime_states_symbol_timestamp`)
- Batch queries for multiple symbols (`get_regimes_batch`)
- Cache regime states (TTL: 1 minute)
- **Acceptable Latency**: <5ms per symbol (P99)
### Risk 2: Stale Regime Data
- **Impact**: Trading with outdated regime classifications
- **Mitigation**:
- Check `event_timestamp` (reject data >1 hour old)
- Fall back to Normal regime (1.0x multiplier) if stale
- Monitor `regime_states` freshness via Grafana
### Risk 3: Over-Leverage in Trending Regimes
- **Impact**: 1.5x multiplier could exceed risk limits
- **Mitigation**:
- Renormalize allocations to 100% after applying multipliers
- Hard cap: No single position >20% (already in allocation.rs)
- Risk budget monitoring (Feature 224)
### Risk 4: Whipsaw in Crisis Regimes
- **Impact**: 0.2x multiplier during false alarms → missed opportunities
- **Mitigation**:
- Require high confidence (>0.8) for Crisis regime
- Monitor regime flip-flopping (>50/hour alert)
- Manual override capability
---
## Performance Impact
### Latency Breakdown (Estimated)
```
Current (Wave C - 201 features):
- Universe Selection: 50ms
- Asset Selection: 20ms
- Allocation (MLOptimized): 0.5ms
- Order Generation: 10ms
- TOTAL: 80.5ms
With Regime Integration (Wave D - 225 features):
- Universe Selection: 50ms
- Asset Selection: 20ms
- Regime Detection (batch query): +3ms ← NEW
- Allocation (RegimeAdaptive): 0.5ms
- Order Generation (with stop-loss): 10ms
- TOTAL: 83.5ms (+3.7% overhead)
```
**Conclusion**: +3ms overhead is acceptable for 25-50% Sharpe improvement.
---
## Code Statistics
### Files to Create
1. `services/trading_agent_service/src/regime.rs` (~200 lines)
2. `services/trading_agent_service/tests/regime_allocation_test.rs` (~300 lines)
### Files to Modify
1. `services/trading_agent_service/src/allocation.rs` (+100 lines)
2. `services/trading_agent_service/src/service.rs` (+50 lines)
3. `services/trading_agent_service/src/orders.rs` (+30 lines)
4. `services/trading_agent_service/src/lib.rs` (+1 line for module export)
### Total Code Changes
- **New Code**: ~500 lines
- **Modified Code**: ~180 lines
- **Total Effort**: ~11 hours (2+3+2+1+3)
---
## Rollback Plan
### Level 1: Feature Flag (Immediate)
```rust
const ENABLE_REGIME_ADAPTIVE: bool = false; // Set to true to enable
if ENABLE_REGIME_ADAPTIVE {
AllocationMethod::RegimeAdaptive { ... }
} else {
AllocationMethod::MLOptimized // Fall back to Wave C behavior
}
```
### Level 2: Database Rollback (5 minutes)
```sql
-- Disable regime tables (keep data)
REVOKE SELECT ON regime_states FROM foxhunt;
REVOKE SELECT ON adaptive_strategy_metrics FROM foxhunt;
```
### Level 3: Code Rollback (10 minutes)
```bash
git revert <integration-commit-hash>
cargo build --release -p trading_agent_service
systemctl restart trading_agent_service
```
---
## Next Steps
### Immediate (Priority 1)
1.**This Report**: Document integration gap
2.**User Decision**: Approve integration plan (11 hours effort)
3.**Implementation**: Execute 5-phase integration plan
### Short-term (After Integration)
4.**Testing**: Run regime allocation tests (3 hours)
5.**Validation**: Paper trading with regime-adaptive sizing (1 week)
6.**Monitoring**: Set up Grafana alerts for regime metrics
### Medium-term (Production)
7.**Performance Tuning**: Optimize database queries (<5ms P99)
8.**Caching**: Add 1-minute TTL cache for regime states
9.**Documentation**: Update production runbooks
---
## Conclusion
**Gap Confirmed**: Wave D's adaptive position sizing (Features 221-224) is **fully implemented** but **NOT integrated** into the trading decision flow.
**Impact**: Without integration, the 225-feature ML models will include regime-adaptive features in training, but **production trading will ignore regime-based position sizing**.
**Recommendation**: **PROCEED WITH INTEGRATION** before ML retraining.
**Rationale**:
1. **Effort**: 11 hours (manageable)
2. **Risk**: Low (feature flag + rollback plan)
3. **Benefit**: +25-50% Sharpe improvement (per Wave D hypothesis)
4. **Urgency**: Must complete before 225-feature ML retraining (4-6 weeks)
**Decision Required**: Approve integration plan and proceed with implementation?
---
**Files Analyzed**:
- `/home/jgrusewski/Work/foxhunt/ml/src/features/regime_adaptive.rs` (644 lines)
- `/home/jgrusewski/Work/foxhunt/services/trading_agent_service/src/allocation.rs` (716 lines)
- `/home/jgrusewski/Work/foxhunt/services/trading_agent_service/src/service.rs` (675 lines)
- `/home/jgrusewski/Work/foxhunt/migrations/045_wave_d_regime_tracking.sql` (12,819 bytes)
- `/home/jgrusewski/Work/foxhunt/AGENT_D11_PORTFOLIO_ALLOCATION_IMPLEMENTATION_REPORT.md` (analysis)
**Agent WIRE-02 Complete** | 2025-10-19

View File

@@ -0,0 +1,164 @@
# Agent WIRE-02: Quick Summary
**Date**: 2025-10-19
**Status**: ✅ COMPLETE - Integration gap identified
---
## 🔴 CRITICAL FINDING
**Wave D's adaptive position sizing is IMPLEMENTED but NOT INTEGRATED into trading flow.**
---
## Gap Analysis
### ✅ What's Done
1. **RegimeAdaptiveFeatures** - Fully implemented (644 lines, 12/12 tests)
- Position multipliers: 0.2x (Crisis) to 1.5x (Trending)
- Stop-loss multipliers: 1.5x (Sideways) to 4.0x ATR (Crisis)
- Location: `ml/src/features/regime_adaptive.rs`
2. **Database Schema** - Migration 045 applied
- Tables: `regime_states`, `regime_transitions`, `adaptive_strategy_metrics`
- Functions: `get_latest_regime()`, `get_regime_transition_matrix()`
3. **gRPC Endpoints** - Defined and routed
- `GetRegimeState(symbol) → RegimeStateResponse`
- `GetRegimeTransitions(symbol) → TransitionsResponse`
### ❌ What's Missing
1. **Trading Agent Service** - NO regime integration
- File: `services/trading_agent_service/src/allocation.rs` (716 lines)
- Status: 5 allocation methods (EqualWeight, RiskParity, MeanVariance, MLOptimized, KellyCriterion)
- **NO imports** of `RegimeAdaptiveFeatures`
- **NO database queries** to `regime_states`
- **NO application** of position/stop-loss multipliers
2. **Order Generation** - NO stop-loss adjustment
- File: `services/trading_agent_service/src/orders.rs`
- Status: Static stop-loss logic, no regime-based ATR multipliers
---
## Impact
**Without Integration**:
- ML models train with Features 221-224 (regime multipliers)
- **BUT** production trading ignores regime state
- Position sizes stay STATIC (no 0.2x-1.5x adjustment)
- Stop-losses stay STATIC (no 1.5x-4.0x ATR adjustment)
- **Expected Sharpe improvement: 0%** (instead of +25-50%)
---
## Integration Plan
### 5-Phase Implementation (11 hours total)
**Phase 1**: Database Query Layer (2h)
- Create `services/trading_agent_service/src/regime.rs`
- Implement `RegimeDetector` to query `regime_states` table
- Add `get_regime(symbol) → RegimeState` method
**Phase 2**: Allocation Integration (3h)
- Add `AllocationMethod::RegimeAdaptive` enum variant
- Implement regime multiplier wrapper around base allocation
- Apply position multipliers (0.2x-1.5x) to allocation weights
**Phase 3**: Service Wiring (2h)
- Add `RegimeDetector` to `TradingAgentServiceImpl`
- Wire `allocate_portfolio` gRPC endpoint to use regime-adaptive allocation
- Add database connection pooling
**Phase 4**: Order Generation (1h)
- Update `OrderGenerator::generate_order()` to accept `RegimeState`
- Apply stop-loss multipliers (1.5x-4.0x ATR) based on regime
**Phase 5**: Testing (3h)
- Create `tests/regime_allocation_test.rs`
- Validate Crisis regime → 0.2x position size
- Validate Trending regime → 1.5x position size
- Validate Volatile regime → 3.0x ATR stop-loss
---
## Code Changes
### Files to Create
1. `services/trading_agent_service/src/regime.rs` (~200 lines)
2. `services/trading_agent_service/tests/regime_allocation_test.rs` (~300 lines)
### Files to Modify
1. `services/trading_agent_service/src/allocation.rs` (+100 lines)
2. `services/trading_agent_service/src/service.rs` (+50 lines)
3. `services/trading_agent_service/src/orders.rs` (+30 lines)
4. `services/trading_agent_service/src/lib.rs` (+1 line)
**Total**: ~500 lines new, ~180 lines modified
---
## Performance Impact
**Latency Addition**: +3ms (batch regime queries)
- Current: 80.5ms end-to-end
- With regime: 83.5ms (+3.7% overhead)
- **Acceptable** for +25-50% Sharpe improvement
---
## Risk Mitigation
1. **Feature Flag**: Easy on/off toggle
2. **Database Indexes**: Already created (`idx_regime_states_symbol_timestamp`)
3. **Fallback**: Use Normal regime (1.0x) if data stale/missing
4. **Renormalization**: Prevent over-leverage from 1.5x multipliers
5. **Rollback**: 3-level plan (flag → database → code)
---
## Recommendation
**✅ PROCEED WITH INTEGRATION** before ML retraining
**Why**:
- **Effort**: 11 hours (manageable)
- **Risk**: Low (feature flag + rollback plan)
- **Benefit**: Unlock +25-50% Sharpe improvement
- **Urgency**: Must complete before 225-feature ML retraining (4-6 weeks)
**Next Step**: User approval to execute 5-phase integration plan
---
## Example: Crisis Regime Behavior
**Scenario**: Market crash detected (Crisis regime)
**Without Integration** (Current):
- Base allocation: $100K to ES.FUT
- **Actual position**: $100K (FULL RISK)
- Stop-loss: 2.0x ATR = $20 away
- **Result**: Full exposure during crisis ❌
**With Integration** (After Fix):
- Base allocation: $100K to ES.FUT
- Regime multiplier: 0.2x (Crisis)
- **Actual position**: $20K (80% RISK REDUCTION) ✅
- Stop-loss: 4.0x ATR = $40 away (wider to avoid panic exit)
- **Result**: Protected capital during crisis ✅
---
## Files Referenced
-`/home/jgrusewski/Work/foxhunt/ml/src/features/regime_adaptive.rs`
-`/home/jgrusewski/Work/foxhunt/migrations/045_wave_d_regime_tracking.sql`
-`/home/jgrusewski/Work/foxhunt/services/trading_agent_service/src/allocation.rs` (NO integration)
-`/home/jgrusewski/Work/foxhunt/services/trading_agent_service/src/service.rs` (placeholder only)
-`/home/jgrusewski/Work/foxhunt/services/trading_agent_service/src/orders.rs` (static stop-loss)
---
**Agent WIRE-02 Complete** | 2025-10-19

View File

@@ -0,0 +1,129 @@
╔══════════════════════════════════════════════════════════════════════════════╗
║ AGENT WIRE-14: PAPER TRADING WAVE D INTEGRATION GAPS ║
╚══════════════════════════════════════════════════════════════════════════════╝
CURRENT STATE (Paper Trading Executor):
┌──────────────────────────────────────────────────────────────────────────┐
│ Component │ Status │ Wave C Baseline │ Wave D Required │
├────────────────────────┼─────────┼──────────────────┼───────────────────┤
│ ML Strategy │ ✅ YES │ SharedMLStrategy │ SharedMLStrategy │
│ Feature Config │ ❌ NO │ Hardcoded (20) │ FeatureConfig:: │
│ │ │ │ wave_d() │
│ Feature Count │ ❌ NO │ 201 features │ 225 features │
│ Regime Queries │ ❌ NO │ None │ get_latest_regime │
│ Position Sizing │ ❌ NO │ Fixed 1.0 │ Adaptive 0.2-1.5x │
│ Kelly Criterion │ ❌ NO │ Comment only │ Implemented │
└────────────────────────┴─────────┴──────────────────┴───────────────────┘
CRITICAL GAPS:
1. FEATURE EXTRACTION (Line 154-157)
┌─────────────────────────────────────────────────────────────────────┐
│ Current: SharedMLStrategy::new(20, 0.6) │
│ ^^^^ Hardcoded - NO feature config │
│ │
│ Required: SharedMLStrategy::new_with_config( │
│ 20, 0.6, │
│ FeatureConfig::wave_d() // ✅ 225 features │
│ ) │
└─────────────────────────────────────────────────────────────────────┘
2. REGIME STATE QUERIES (Missing)
┌─────────────────────────────────────────────────────────────────────┐
│ Current: No database queries for regime_states │
│ │
│ Required: let regime = sqlx::query!( │
│ "SELECT regime FROM get_latest_regime($1)", │
│ symbol │
│ ).fetch_one(&self.db_pool).await?; │
└─────────────────────────────────────────────────────────────────────┘
3. ADAPTIVE POSITION SIZING (Line 567-575)
┌─────────────────────────────────────────────────────────────────────┐
│ Current: let position_size = 1.0; // Fixed │
│ │
│ Required: let regime_mult = match regime { │
│ "Trending" => 1.5, │
│ "Ranging" => 0.8, │
│ "Volatile" => 0.5, │
│ "Transition" => 0.2, │
│ }; │
│ let size = base * regime_mult * confidence; │
└─────────────────────────────────────────────────────────────────────┘
ARCHITECTURE MISMATCH:
common::ml_strategy::MLFeatureExtractor
├─ Expected feature count: hardcoded comment (26/36/65)
├─ NO FeatureConfig integration
└─ NO Wave D support (225 features)
ml::features::config::FeatureConfig
├─ wave_d() method exists ✅
├─ enable_wave_d_regime: true ✅
└─ 225 feature support ✅
⚠️ PROBLEM: These two systems are NOT connected!
TESTING RISK:
Paper Trading Current:
┌─────────────────────────────────────────────────────────────────────┐
│ Features: 201 (Wave C baseline) │
│ Sizing: Fixed 1.0 contracts │
│ Regime: Not aware │
│ │
│ Result: Tests Wave C, NOT Wave D ❌ │
└─────────────────────────────────────────────────────────────────────┘
Paper Trading Required:
┌─────────────────────────────────────────────────────────────────────┐
│ Features: 225 (Wave D regime detection) │
│ Sizing: Adaptive 0.2x-1.5x based on regime │
│ Regime: Queries regime_states before each trade │
│ │
│ Result: Validates Wave D before production ✅ │
└─────────────────────────────────────────────────────────────────────┘
ACTION PLAN (6 hours):
[P1] Modify SharedMLStrategy constructor (2h)
└─ Accept FeatureConfig parameter
└─ Update paper_trading_executor.rs
└─ Verify 225-feature extraction
[P2] Add regime state queries (1h)
└─ Implement get_regime_for_symbol()
└─ Query get_latest_regime() before trades
└─ Log regime transitions
[P3] Adaptive position sizing (2h)
└─ Replace calculate_position_size()
└─ Implement regime multipliers (0.2x-1.5x)
└─ Add confidence-based Kelly factor
[P4] Testing & validation (1h)
└─ Run 24-hour paper trading test
└─ Monitor regime vs. sizing correlation
└─ Document Wave C vs. Wave D performance
RECOMMENDATION:
⛔ BLOCK production deployment until paper trading validates Wave D
Why? Paper trading is the ONLY pre-production validation step.
If it tests Wave C config, we have ZERO evidence that:
- 225-feature extraction works
- Regime detection improves performance
- Adaptive sizing reduces drawdowns
Next Steps:
1. Implement action items (6 hours)
2. Run 24-hour paper trading validation
3. Compare Wave C baseline vs. Wave D adaptive results
4. Document findings in PAPER_TRADING_WAVE_D_VALIDATION.md
═══════════════════════════════════════════════════════════════════════════════
Agent WIRE-14 Status: ⚠️ PARTIAL INTEGRATION - CRITICAL GAPS IDENTIFIED
Next Agent: WIRE-15 (Adaptive Position Sizing Implementation)
═══════════════════════════════════════════════════════════════════════════════

View File

@@ -0,0 +1,427 @@
# AGENT WIRE-14: Paper Trading Executor Wave D Integration Status
**Agent**: WIRE-14
**Mission**: Verify paper trading executor uses Wave D features and adaptive sizing
**Status**: ⚠️ PARTIAL INTEGRATION - Missing Wave D Features
**Priority**: HIGH - Paper trading must test Wave D before live deployment
**Date**: 2025-10-19
---
## Executive Summary
The paper trading executor (`services/trading_service/src/paper_trading_executor.rs`) currently uses `SharedMLStrategy` but is **NOT configured for Wave D features**. Critical gaps identified:
1. ✅ Uses `SharedMLStrategy` (ONE SINGLE SYSTEM architecture)
2.**NO Wave D feature configuration** - Uses hardcoded defaults (20 lookback, 0.6 confidence)
3.**NO regime state queries** - Does not check `regime_states` table
4.**NO adaptive position sizing** - Uses fixed 1.0 contract size
5. ⚠️ **Kelly Criterion mentioned but not implemented** (line 569 comment only)
**Risk**: Paper trading will test Wave C baseline (201 features) instead of Wave D (225 features + regime detection).
---
## Code Analysis
### 1. ML Strategy Initialization
**File**: `services/trading_service/src/paper_trading_executor.rs`
**Lines**: 154-157
```rust
pub fn new(db_pool: PgPool, config: PaperTradingConfig) -> Self {
// Initialize with shared ML strategy (default configuration)
let ml_strategy = SharedMLStrategy::new(20, 0.6);
// ^^^ HARDCODED: 20 lookback, 0.6 confidence - NO Wave D config
```
**Issue**: `SharedMLStrategy::new()` does NOT accept `FeatureConfig` parameter. The constructor signature is:
```rust
pub fn new(lookback_periods: usize, min_confidence_threshold: f64) -> Self
```
**Missing**: No way to pass `FeatureConfig::wave_d()` to enable 225-feature extraction.
---
### 2. Position Sizing Logic
**File**: `services/trading_service/src/paper_trading_executor.rs`
**Lines**: 567-575
```rust
fn calculate_position_size(&self, _prediction: &PendingPrediction) -> Result<f64> {
// Simple fixed position size for paper trading
// In production, this could use Kelly Criterion or volatility-adjusted sizing
let position_size = 1.0; // 1 contract
// ^^^ FIXED SIZE: No adaptive sizing based on regime or confidence
if position_size > self.config.max_position_size {
return Err(anyhow!("Position size {} exceeds maximum {}", position_size, self.config.max_position_size));
}
Ok(position_size)
}
```
**Missing Wave D Adaptive Logic**:
- No regime state queries (`SELECT regime FROM regime_states`)
- No adaptive multipliers (0.2x-1.5x based on regime)
- No Kelly Criterion position sizing
- No volatility-adjusted sizing
**Expected Behavior** (from Wave D design):
```rust
// Query regime state
let regime = sqlx::query!("SELECT regime FROM get_latest_regime($1)", symbol)
.fetch_one(&self.db_pool).await?;
// Apply regime-adaptive multiplier
let base_size = 1.0;
let regime_multiplier = match regime.regime.as_str() {
"Trending" => 1.5, // Increase size in trending markets
"Ranging" => 0.8, // Reduce size in ranging markets
"Volatile" => 0.5, // Minimize size in volatile markets
"Transition" => 0.2, // Avoid trading during transitions
_ => 1.0, // Normal sizing for unknown regimes
};
let position_size = base_size * regime_multiplier * confidence_factor;
```
---
### 3. Regime State Integration
**Search Results**: ❌ NO regime queries found in `paper_trading_executor.rs`
```bash
$ grep -rn "regime_states\|regime_transitions\|get_latest_regime" \
services/trading_service/src/paper_trading_executor.rs
# Result: 0 matches
```
**Contrast with `trading.rs` (Trading Service)**:
```rust
// services/trading_service/src/services/trading.rs:992-1023
async fn get_regime_state(&self, req: Request<GetRegimeStateRequest>) -> Result<Response<GetRegimeStateResponse>, Status> {
let regime_state = sqlx::query!(
r#"SELECT regime, confidence, detected_at FROM get_latest_regime($1)"#,
req.symbol
).fetch_one(&self.db_pool).await?;
Ok(Response::new(GetRegimeStateResponse {
current_regime: regime_state.regime.unwrap_or("Normal".to_string()),
confidence: regime_state.confidence.unwrap_or(0.0),
// ...
}))
}
```
**Paper Trading Executor**: No equivalent logic.
---
### 4. Feature Configuration Architecture
**Analysis**: `SharedMLStrategy` uses `MLFeatureExtractor` which has a **legacy field** for feature count:
**File**: `common/src/ml_strategy.rs` (lines 66-84)
```rust
pub struct MLFeatureExtractor {
pub lookback_periods: usize,
/// Expected feature count (26=Wave A, 30=Wave A+4 extra, 36=Wave B, 65=Wave C)
expected_feature_count: usize, // ❌ Outdated comment - no Wave D (225)
price_history: Vec<f64>,
volume_history: Vec<f64>,
// ...
}
```
**Problem**: `MLFeatureExtractor` does NOT use `FeatureConfig` from `ml/src/features/config.rs` which supports Wave D:
**File**: `ml/src/features/config.rs` (lines 345-355)
```rust
pub fn wave_d() -> Self {
Self {
enable_wave_a: true,
enable_wave_b: true,
enable_wave_c: true,
enable_wave_d_regime: true, // ✅ Enables 24 regime features (201→225)
// ...
}
}
```
**Root Cause**: Architecture mismatch between `common::ml_strategy` (legacy extractor) and `ml::features::config` (Wave D-aware).
---
## Integration Gaps
### Gap 1: No Wave D Feature Config
**Current**: `SharedMLStrategy::new(20, 0.6)` - hardcoded defaults
**Required**: Pass `FeatureConfig::wave_d()` to enable 225-feature extraction
**Blocker**: `SharedMLStrategy` constructor does NOT accept `FeatureConfig`
**Solution**:
```rust
// Option A: Add new constructor
impl SharedMLStrategy {
pub fn new_with_feature_config(
lookback: usize,
confidence: f64,
feature_config: FeatureConfig,
) -> Self {
// ...
}
}
// Option B: Modify existing constructor
pub fn new(
lookback: usize,
confidence: f64,
feature_config: Option<FeatureConfig>,
) -> Self {
let config = feature_config.unwrap_or(FeatureConfig::wave_a());
// ...
}
```
---
### Gap 2: No Regime State Queries
**Current**: No database queries for `regime_states` or `regime_transitions`
**Required**: Query latest regime before position sizing decisions
**Blocker**: Database access exists (`self.db_pool`) but not used
**Solution**:
```rust
async fn get_regime_for_symbol(&self, symbol: &str) -> Result<RegimeState> {
let regime = sqlx::query!(
r#"
SELECT regime, confidence, detected_at
FROM get_latest_regime($1)
"#,
symbol
)
.fetch_one(&self.db_pool)
.await
.context("Failed to fetch regime state")?;
Ok(RegimeState {
regime: regime.regime.unwrap_or("Normal".to_string()),
confidence: regime.confidence.unwrap_or(0.0),
detected_at: regime.detected_at,
})
}
```
---
### Gap 3: No Adaptive Position Sizing
**Current**: Fixed 1.0 contract size (line 570)
**Required**: Regime-adaptive sizing (0.2x-1.5x) + confidence-based Kelly multiplier
**Blocker**: Regime state not queried, Kelly logic not implemented
**Solution**:
```rust
async fn calculate_adaptive_position_size(
&self,
prediction: &PendingPrediction,
) -> Result<f64> {
// Step 1: Get regime state
let regime = self.get_regime_for_symbol(&prediction.symbol).await?;
// Step 2: Apply regime-adaptive multiplier (Wave D design)
let regime_multiplier = match regime.regime.as_str() {
"Trending" => 1.5,
"Ranging" => 0.8,
"Volatile" => 0.5,
"Transition" => 0.2,
_ => 1.0,
};
// Step 3: Apply confidence-based Kelly multiplier
// Kelly formula: f* = (p*b - q) / b
// For trading: simplified to linear confidence scaling
let confidence_factor = prediction.ensemble_confidence.clamp(0.6, 1.0);
let kelly_multiplier = (confidence_factor - 0.6) / 0.4; // 0.6→0.0, 1.0→1.0
// Step 4: Calculate final position size
let base_size = 1.0; // Base contract size
let adaptive_size = base_size * regime_multiplier * (1.0 + kelly_multiplier);
// Step 5: Apply safety limits
Ok(adaptive_size.clamp(0.2, 5.0))
}
```
---
## Testing Implications
### Current Paper Trading Behavior
1. **Feature Set**: Uses Wave C baseline (201 features) - NO regime detection
2. **Position Sizing**: Fixed 1.0 contracts - NO adaptive sizing
3. **Regime Awareness**: None - trades blindly across all market conditions
### Expected Wave D Behavior
1. **Feature Set**: 225 features (201 + 24 regime detection)
2. **Position Sizing**: 0.2x-1.5x adaptive multipliers based on regime
3. **Regime Awareness**: Queries `regime_states`, avoids transitions
### Risk Assessment
⚠️ **HIGH RISK**: Paper trading will NOT validate Wave D features before production deployment.
**Scenario**: If paper trading passes with Wave C config, we have NO evidence that:
- 225-feature extraction works in production
- Regime detection improves performance
- Adaptive sizing reduces drawdowns
**Recommendation**: Block production deployment until paper trading uses Wave D config.
---
## Action Items
### Priority 1: Enable Wave D Features (2 hours)
- [ ] Modify `SharedMLStrategy::new()` to accept `FeatureConfig` parameter
- [ ] Update `paper_trading_executor.rs` to use `FeatureConfig::wave_d()`
- [ ] Verify 225-feature extraction in paper trading logs
### Priority 2: Implement Regime Queries (1 hour)
- [ ] Add `get_regime_for_symbol()` method to `PaperTradingExecutor`
- [ ] Query `regime_states` table before each trade
- [ ] Log regime transitions for debugging
### Priority 3: Adaptive Position Sizing (2 hours)
- [ ] Replace `calculate_position_size()` with `calculate_adaptive_position_size()`
- [ ] Implement regime multipliers (0.2x-1.5x)
- [ ] Add confidence-based Kelly multiplier
- [ ] Validate position size range (0.2-5.0 contracts)
### Priority 4: Testing & Validation (1 hour)
- [ ] Run paper trading with ES.FUT, NQ.FUT for 24 hours
- [ ] Monitor regime transitions vs. position sizing
- [ ] Compare performance: Wave C baseline vs. Wave D adaptive
- [ ] Document results in `PAPER_TRADING_WAVE_D_VALIDATION.md`
**Total Effort**: 6 hours
---
## Technical Debt
### Issue 1: Architecture Mismatch
**Problem**: `common::ml_strategy::MLFeatureExtractor` does NOT use `ml::features::config::FeatureConfig`.
**Current State**:
- `MLFeatureExtractor` has hardcoded feature count expectations (comment: "26=Wave A, 36=Wave B, 65=Wave C")
- No mention of Wave D (225 features)
- No integration with `FeatureConfig::wave_d()`
**Solution**:
```rust
// common/src/ml_strategy.rs
pub struct MLFeatureExtractor {
pub lookback_periods: usize,
feature_config: ml::features::config::FeatureConfig, // ✅ Use canonical config
price_history: Vec<f64>,
// ...
}
impl MLFeatureExtractor {
pub fn new(lookback: usize, feature_config: FeatureConfig) -> Self {
Self {
lookback_periods: lookback,
feature_config,
// ...
}
}
}
```
**Blocker**: Cross-crate dependency (`common` depends on `ml`).
---
### Issue 2: Kelly Criterion Stub
**Problem**: Line 569 comment says "could use Kelly Criterion" but NOT implemented.
**Current Code**:
```rust
// In production, this could use Kelly Criterion or volatility-adjusted sizing
let position_size = 1.0; // 1 contract
```
**Required Implementation**:
```rust
use risk::kelly_sizing::{KellyResult, KellySizer};
async fn calculate_kelly_position(&self, prediction: &PendingPrediction) -> Result<f64> {
// Query historical performance for win rate
let win_rate = self.get_strategy_win_rate(&prediction.symbol).await?;
// Use ensemble confidence as win probability
let win_prob = prediction.ensemble_confidence;
let loss_prob = 1.0 - win_prob;
// Expected profit/loss ratio (from historical data)
let profit_loss_ratio = 1.5; // 1.5:1 risk/reward
// Kelly formula: f* = (p*b - q) / b
let kelly_fraction = (win_prob * profit_loss_ratio - loss_prob) / profit_loss_ratio;
// Use fractional Kelly (25%) for safety
let fractional_kelly = kelly_fraction * 0.25;
Ok(fractional_kelly.clamp(0.0, 1.0))
}
```
**Existing Code**: `services/trading_service/src/core/risk_manager.rs` has `KellySizer` but NOT used in paper trading.
---
## References
### Codebase Files
- `services/trading_service/src/paper_trading_executor.rs` (897 lines)
- `common/src/ml_strategy.rs` (MLFeatureExtractor definition)
- `ml/src/features/config.rs` (FeatureConfig::wave_d() implementation)
- `services/trading_service/src/services/trading.rs` (GetRegimeState gRPC method)
- `services/trading_service/src/core/risk_manager.rs` (KellySizer implementation)
### Database Schema
- `migrations/045_regime_detection.sql` (regime_states, regime_transitions tables)
- Stored function: `get_latest_regime(symbol TEXT)`
### Wave D Documentation
- `CLAUDE.md` (Wave D Phase 6 status, production targets)
- `WAVE_D_DEPLOYMENT_GUIDE.md` (regime detection integration guide)
- `WAVE_D_QUICK_REFERENCE.md` (adaptive sizing formulas)
---
## Conclusion
**Status**: ⚠️ **PARTIAL INTEGRATION - CRITICAL GAPS**
The paper trading executor is architecturally sound (uses `SharedMLStrategy`, ONE SINGLE SYSTEM) but **NOT configured for Wave D testing**:
1. ❌ No 225-feature extraction (stuck on Wave C baseline)
2. ❌ No regime state queries (blind to market conditions)
3. ❌ No adaptive position sizing (fixed 1.0 contracts)
**Recommendation**: **BLOCK production deployment** until paper trading validates Wave D features. Implement action items (6 hours) and run 24-hour validation before proceeding.
**Next Agent**: WIRE-15 should implement `calculate_adaptive_position_size()` with regime multipliers and Kelly logic.
---
**Agent WIRE-14 signing off.**
**Mission**: PARTIAL - Integration gaps identified, action plan provided.
**Handoff**: WIRE-15 (Adaptive Position Sizing Implementation)