foxhunt/WAVE_D_PHASE_6_EXECUTION_READY.md

# Wave D Phase 6: Execution Ready

**Date**: 2025-10-18
**Status**: 📋 **READY FOR EXECUTION**
**Current State**: 95% Production Ready → Target: 100%

---

## Executive Summary

The comprehensive 24-agent execution plan (G1-G24) has been prepared using Zen MCP's 3-step planning workflow and is **ready for immediate execution**. All resource management strategies have been defined to prevent system exhaustion.

**IMPORTANT**: Due to current system load (100+ background processes from Phase 5 validation), the execution should be initiated when resources are available or via manual wave-by-wave execution.

---

## Resource Status Check

**Current Background Processes**: 100+ cargo test/build/check/bench processes running
**Token Usage**: ~130K/200K (65% consumed)
**Recommendation**: Clean up or wait for Phase 5 processes to complete before spawning Phase 6 agents

---

## Phase 6 Execution Plan (24 Agents: G1-G24)

### Wave 1: Memory Optimization & Regime Integration (6 agents, 4 days)

#### Agent G1: VecDeque → Ring Buffer (1 day)
**Priority**: P0 CRITICAL
**MCP Tools**: Corrode (Rust memory analysis) + Zen thinkdeep (ring buffer design)

**Files**:
- `ml/src/features/extraction.rs`
- `common/src/ml_strategy.rs`

**Task**: Replace VecDeque with fixed-size ring buffer for feature history

**Expected Savings**: 5-7 KB/symbol (21KB → 14-16KB)

**Validation**:
```bash
cargo test -p ml --lib features::extraction --release -- --nocapture
# Verify: Memory benchmark shows <5KB overhead (vs 21KB VecDeque)
```

---

#### Agent G2: Lazy Feature Allocation (1 day)
**Priority**: P0 CRITICAL
**MCP Tools**: Corrode (Rust memory analysis)

**Files**:
- `ml/src/features/pipeline.rs`
- `ml/src/features/mod.rs`

**Task**: Only allocate feature buffers when needed (sparse feature sets)

**Expected Savings**: 10-15 KB/symbol

**Validation**:
```bash
cargo test -p ml --lib features::pipeline --release
# Verify: Memory test shows 40-50% reduction for sparse feature sets
```

---

#### Agent G3: Share Normalizers via Arc (1 day)
**Priority**: P0 CRITICAL
**MCP Tools**: Corrode (Rust Arc/smart pointer analysis)

**Files**:
- `ml/src/features/normalization.rs`

**Task**: Use Arc<Normalizer> instead of per-symbol clones

**Expected Savings**: 20 KB/symbol (225 normalizers × 1 instance vs 100K × 225)

**Validation**:
```bash
cargo test -p ml --lib features::normalization --release
# Verify: Single Arc instance shared across all symbols
```

---

#### Agent G4: Memory Pooling (1 day)
**Priority**: P0 CRITICAL
**MCP Tools**: Corrode (Rust memory pooling) + Zen thinkdeep (pooling design)

**Files**:
- `ml/src/features/mod.rs`
- `ml/Cargo.toml` (add object_pool crate)

**Task**: Implement memory pool for reusable buffers

**Expected Savings**: 5 KB/symbol

**Validation**:
```bash
SQLX_OFFLINE=false cargo test -p ml --test wave_d_memory_stress_test wave_d_memory_stress_100k_symbols --release -- --ignored --nocapture
# Verify: Memory <1,500MB for 100K symbols (vs 5,463MB current)
```

---

#### Agent G5: Regime Multipliers → Trading Agent (4 hours)
**Priority**: P1 HIGH
**MCP Tools**: SkyDeck (file search) + Zen challenge (hypothesis validation)

**Files**:
- `services/trading_agent_service/src/allocation.rs` (update calculate_allocation)
- `adaptive-strategy/src/risk/ppo_position_sizer.rs` (regime multipliers)

**Task**: Integrate position sizing multipliers into Trading Agent

**Validation**:
```bash
cargo test -p trading_agent_service --lib allocation --release
# Verify: 1.5x position in TRENDING, 0.5x in VOLATILE, 0.2x in CRISIS
```

---

#### Agent G6: Dynamic Stops Integration (4 hours)
**Priority**: P1 HIGH
**MCP Tools**: SkyDeck (file search)

**Files**:
- `adaptive-strategy/src/execution/mod.rs` (dynamic_stop_loss method)
- `services/trading_agent_service/src/orders.rs` (order submission with stops)

**Task**: Connect ATR-based dynamic stops to Trading Agent execution

**Validation**:
```bash
cargo test -p adaptive-strategy --lib execution --release
# Verify: 2.0x ATR stop in NORMAL, 4.0x in VOLATILE
```

---

### Wave 2: Regime Sharpe, TFT, E2E Setup (4 agents, 2 days)

#### Agent G7: Regime-Conditioned Sharpe → Trading Agent (2 hours)
**Priority**: P1 HIGH
**MCP Tools**: SkyDeck (file search)

**Files**:
- `services/trading_agent_service/src/lib.rs` (decision loop)
- `adaptive-strategy/src/ensemble/weight_optimizer.rs` (regime Sharpe)

**Task**: Add regime-conditioned Sharpe to Trading Agent decision-making

**Validation**:
```bash
cargo test -p trading_agent_service --lib --release
# Verify: Trading Agent selects models with highest regime-conditioned Sharpe
```

---

#### Agent G8: TFT Model Architecture Update (1.5 hours)
**Priority**: P1 HIGH
**MCP Tools**: Corrode (Rust code analysis)

**Files**:
- `ml/src/tft/model.rs` (TFTConfig.input_dim: 50 → 225)
- `ml/src/tft/config.rs` (default config update)

**Task**: Update TFT config to support 225 features

**Validation**:
```bash
cargo test -p ml --lib tft::model --release
# Verify: Unit test creates TFT with 225-dim input
```

---

#### Agent G9: TFT Training Pipeline Update (1.5 hours)
**Priority**: P1 HIGH
**MCP Tools**: Corrode (Rust code analysis)

**Files**:
- `ml/examples/train_tft_dbn.rs` (use WaveDFeatureConfig)
- `ml/src/trainers/tft.rs` (feature count validation)

**Task**: Update TFT training to use WaveDFeatureConfig (225 features)

**Validation**:
```bash
cargo run -p ml --example train_tft_dbn --release -- --epochs 1
# Verify: Training script creates 225-feature tensors
```

---

#### Agent G10: ES.FUT E2E Validation (4 hours)
**Priority**: P2 MEDIUM
**MCP Tools**: SkyDeck (E2E test execution)

**Files**:
- `ml/tests/wave_d_e2e_es_fut_225_features_test.rs` (4 tests)

**Task**: Run end-to-end ES.FUT validation with all Phase 6 fixes

**Validation**:
```bash
cargo test -p ml --test wave_d_e2e_es_fut_225_features_test --no-fail-fast -- --nocapture
# Verify: 4/4 tests pass with <10ms latency
```

---

### Wave 3: Multi-Asset E2E Validation (4 agents, 2 days)

#### Agent G11: NQ.FUT E2E Validation (4 hours)
**Priority**: P2 MEDIUM
**MCP Tools**: SkyDeck (E2E test execution)

**Files**:
- `ml/tests/wave_d_e2e_nq_fut_225_features_test.rs` (3 tests)

**Task**: Run end-to-end NQ.FUT validation

**Validation**:
```bash
cargo test -p ml --test wave_d_e2e_nq_fut_225_features_test --no-fail-fast -- --nocapture
# Verify: 3/3 tests pass, regime distribution matches Phase 5
```

---

#### Agent G12: 6E.FUT E2E Validation (4 hours)
**Priority**: P2 MEDIUM
**MCP Tools**: SkyDeck (E2E test execution)

**Files**:
- `ml/tests/transition_6e_fut_integration_test.rs` (3 tests)

**Task**: Run end-to-end 6E.FUT validation

**Validation**:
```bash
cargo test -p ml --test transition_6e_fut_integration_test --no-fail-fast -- --nocapture
# Verify: 3/3 tests pass, Ranging dominance confirmed
```

---

#### Agent G13: ZN.FUT E2E Validation (4 hours)
**Priority**: P2 MEDIUM
**MCP Tools**: SkyDeck (E2E test execution)

**Files**:
- `ml/tests/wave_d_e2e_zn_fut_225_features_test.rs` (5 tests)

**Task**: Run end-to-end ZN.FUT validation

**Validation**:
```bash
SQLX_OFFLINE=false cargo test -p ml --test wave_d_e2e_zn_fut_225_features_test --no-fail-fast -- --nocapture
# Verify: 5/5 tests pass, 88.9% Normal regime maintained
```

---

#### Agent G14: Multi-Symbol Stress Test (4 hours)
**Priority**: P0 CRITICAL
**MCP Tools**: SkyDeck (stress test execution)

**Files**:
- `ml/tests/wave_d_memory_stress_test.rs` (100K symbols)

**Task**: Run multi-asset stress test with all 4 symbols after G1-G4 optimizations

**Validation**:
```bash
SQLX_OFFLINE=false cargo test -p ml --test wave_d_memory_stress_test wave_d_memory_stress_100k_symbols --release -- --ignored --nocapture
# Verify: Memory <1,500MB for 100K symbols (3x better than Phase 5)
```

---

### Wave 4: Performance, Deployment, ML Prep (10 agents, 3 days)

#### Agent G15: Wave D Features Benchmark (4 hours)
**Priority**: P2 MEDIUM
**MCP Tools**: SkyDeck (benchmark execution)

**Files**:
- `ml/benches/wave_d_features_bench.rs`

**Task**: Run Wave D features benchmark regression

**Validation**:
```bash
cargo bench -p ml --bench wave_d_features_bench
# Verify: All features <100μs P99
```

---

#### Agent G16: Alternative Bars Benchmark Regression (4 hours)
**Priority**: P2 MEDIUM
**MCP Tools**: SkyDeck (benchmark execution)

**Files**:
- `ml/benches/alternative_bars_bench.rs`

**Task**: Run alternative bars benchmark regression

**Validation**:
```bash
SQLX_OFFLINE=false cargo bench -p ml --bench alternative_bars_bench
# Verify: 0.70ms DBN loading maintained (Wave B baseline)
```

---

#### Agent G17: Profiling Test (4 hours)
**Priority**: P2 MEDIUM
**MCP Tools**: SkyDeck (profiling test execution)

**Files**:
- `ml/tests/wave_d_profiling_test.rs`

**Task**: Execute profiling test (feature extraction breakdown)

**Validation**:
```bash
SQLX_OFFLINE=false cargo test -p ml --test wave_d_profiling_test --release --no-default-features -- --ignored --nocapture
# Verify: CUSUM <1μs, ADX <2μs, normalization <1μs
```

---

#### Agent G18: Latency Distribution Validation (4 hours)
**Priority**: P2 MEDIUM
**MCP Tools**: SkyDeck (latency test execution)

**Files**:
- `ml/tests/wave_d_e2e_es_fut_225_features_test.rs` (latency test)

**Task**: Validate latency distribution (P50, P95, P99)

**Validation**:
```bash
cargo test -p ml --test wave_d_e2e_es_fut_225_features_test -- test_latency --release --nocapture
# Verify: P99 <10ms for complete 225-feature extraction
```

---

#### Agent G19: Release Build Compilation (4 hours)
**Priority**: P2 MEDIUM
**MCP Tools**: SkyDeck (build execution)

**Task**: Compile all services in release mode

**Validation**:
```bash
time cargo build --release --workspace
# Verify: All 6 services compile in <5 minutes
```

---

#### Agent G20: Docker Image Builds (4 hours)
**Priority**: P2 MEDIUM
**MCP Tools**: SkyDeck (Docker execution)

**Files**:
- `Dockerfile`
- `docker-compose.yml`

**Task**: Build Docker images for all services

**Validation**:
```bash
docker-compose build
docker-compose up -d
docker-compose ps
# Verify: All images build successfully, health checks pass
```

---

#### Agent G21: Staging Deployment (4 hours)
**Priority**: P2 MEDIUM
**MCP Tools**: SkyDeck (deployment execution)

**Task**: Deploy to staging environment

**Validation**:
```bash
# Staging deployment commands (TBD)
# Verify: All services start, gRPC health checks pass
```

---

#### Agent G22: Prometheus + Grafana Dashboards (4 hours)
**Priority**: P2 MEDIUM
**MCP Tools**: SkyDeck (monitoring configuration)

**Task**: Configure Prometheus + Grafana dashboards for regime metrics

**Validation**:
```bash
curl http://localhost:9090/api/v1/targets
curl http://localhost:3000/api/health
# Verify: Regime transition metrics visible in Grafana
```

---

#### Agent G23: Validate 225-Feature Training Pipeline (4 hours)
**Priority**: P2 MEDIUM
**MCP Tools**: SkyDeck (training pipeline validation)

**Files**:
- `ml/src/data_loaders/dbn_sequence_loader.rs`

**Task**: Validate 225-feature training data pipeline

**Validation**:
```bash
cargo test -p ml --lib data_loaders::dbn_sequence_loader --release
# Verify: Batch creation uses 225 features
```

---

#### Agent G24: Update All Training Scripts (4 hours)
**Priority**: P2 MEDIUM
**MCP Tools**: SkyDeck (training scripts update)

**Files**:
- `ml/examples/train_dqn.rs`
- `ml/examples/train_ppo.rs`
- `ml/examples/train_mamba2_dbn.rs`
- `ml/examples/train_tft_dbn.rs`

**Task**: Update training scripts for all models to use WaveDFeatureConfig

**Validation**:
```bash
# Run each training script with --epochs 1 to verify 225 features
cargo run -p ml --example train_dqn --release -- --epochs 1
cargo run -p ml --example train_ppo --release -- --epochs 1
cargo run -p ml --example train_mamba2_dbn --release -- --epochs 1
cargo run -p ml --example train_tft_dbn --release -- --epochs 1
# Verify: All scripts use WaveDFeatureConfig
```

---

## Execution Strategy

### Option A: Automated Wave Execution (Recommended when resources available)

**Command**:
```bash
# Clean up old processes first
ps aux | grep -E "cargo (test|check|build|bench)" | grep -v grep | awk '{if ($10 > 600) print $2}' | xargs -I {} kill -9 {}

# Spawn Wave 1 (6 agents: G1-G6)
# Wait 2 minutes
# Spawn Wave 2 (4 agents: G7-G10)
# Wait 2 minutes
# Spawn Wave 3 (4 agents: G11-G14)
# Wait 2 minutes
# Spawn Wave 4 (10 agents: G15-G24)
```

**Resource Controls**:
- Max 6 agents in parallel per wave
- 2-minute pause between waves
- Background process cleanup before each wave

---

### Option B: Manual Wave-by-Wave Execution (Current Recommendation)

Given current system load, execute agents manually:

**Week 1: Critical Fixes (G1-G9)**
- Day 1-2: G1 (Ring buffer) + G2 (Lazy allocation)
- Day 3-4: G3 (Arc normalizers) + G4 (Memory pooling)
- Day 5: G5 (Regime multipliers) + G6 (Dynamic stops)
- Day 6: G7 (Regime Sharpe) + G8 (TFT architecture)
- Day 7: G9 (TFT training) + G10 (ES.FUT E2E)

**Week 2: Validation & Deployment (G11-G24)**
- Day 8-9: G11-G14 (Multi-asset E2E + stress test)
- Day 10: G15-G18 (Performance regression)
- Day 11: G19-G20 (Release builds + Docker)
- Day 12: G21-G22 (Staging + monitoring)
- Day 13: G23-G24 (ML training prep)
- Day 14: Final validation & documentation

---

## Expected Outcomes

### Post-G1-G4 (Memory Optimization)
- **Memory**: 55.95 KB/symbol → 10-15 KB/symbol (73-82% reduction)
- **100K Symbols**: 5,463MB → <1,500MB (within acceptable range)
- **Status**: P0 CRITICAL blocker resolved

### Post-G5-G7 (Regime Integration)
- **Trading Agent**: Uses regime multipliers (1.0x normal, 1.5x trending, 0.5x volatile, 0.2x crisis)
- **Dynamic Stops**: ATR-based with regime multipliers (2.0x-4.0x)
- **Status**: P1 HIGH blocker resolved

### Post-G8-G9 (TFT Integration)
- **TFT**: Supports 225 features (vs 50 hardcoded)
- **Training**: Uses WaveDFeatureConfig
- **Status**: P1 HIGH blocker resolved

### Post-G10-G14 (E2E Validation)
- **Multi-Asset**: 15/15 tests pass (ES.FUT, NQ.FUT, 6E.FUT, ZN.FUT)
- **Latency**: <10ms P99 for 225-feature extraction
- **Status**: Production validation complete

### Post-G15-G24 (Deployment Ready)
- **Benchmarks**: All performance targets maintained
- **Release Builds**: All services compile successfully
- **Docker**: All images ready for deployment
- **Monitoring**: Grafana dashboards configured
- **ML Training**: All scripts ready for 225-feature retraining
- **Status**: 100% production readiness achieved

---

## Next Steps

1. **Immediate**: Wait for Phase 5 background processes to complete
2. **Option A**: Spawn 24 agents in 4 staggered waves (automated)
3. **Option B**: Execute agents manually wave-by-wave (recommended given current load)
4. **Timeline**: 2 weeks to 100% production readiness
5. **Post-Phase 6**: ML model retraining (4-6 weeks per `ML_TRAINING_ROADMAP.md`)

---

## Critical Success Factors

✅ Memory optimization (G1-G4) achieves 73-82% reduction
✅ Regime integration (G5-G7) connects Trading Agent to adaptive strategies
✅ TFT integration (G8-G9) supports 225 features
✅ Multi-asset E2E (G10-G14) validates all fixes
✅ Performance regression (G15-G18) maintains 72x improvement
✅ Deployment readiness (G19-G24) enables staging deployment

**Deployment Readiness**: 95% → 100% (via Phase 6 execution)

---

**Report Generated**: 2025-10-18
**Status**: 📋 **READY FOR EXECUTION**
**Recommendation**: Execute Option B (manual wave-by-wave) given current system load
**Expected Duration**: 2 weeks
**Target**: 100% Production Readiness