foxhunt/WAVE_D_PHASE_6_EXECUTION_READY.md

Wave D Phase 6: Execution Ready

Date: 2025-10-18 Status: 📋 READY FOR EXECUTION Current State: 95% Production Ready → Target: 100%

---

Executive Summary

The comprehensive 24-agent execution plan (G1-G24) has been prepared using Zen MCP's 3-step planning workflow and is ready for immediate execution. All resource management strategies have been defined to prevent system exhaustion.

IMPORTANT: Due to current system load (100+ background processes from Phase 5 validation), the execution should be initiated when resources are available or via manual wave-by-wave execution.

---

Resource Status Check

Current Background Processes: 100+ cargo test/build/check/bench processes running Token Usage: ~130K/200K (65% consumed) Recommendation: Clean up or wait for Phase 5 processes to complete before spawning Phase 6 agents

---

Phase 6 Execution Plan (24 Agents: G1-G24)

Wave 1: Memory Optimization & Regime Integration (6 agents, 4 days)

Agent G1: VecDeque → Ring Buffer (1 day)

Priority: P0 CRITICAL MCP Tools: Corrode (Rust memory analysis) + Zen thinkdeep (ring buffer design)

Files:

• ml/src/features/extraction.rs
• common/src/ml_strategy.rs

Task: Replace VecDeque with fixed-size ring buffer for feature history

Expected Savings: 5-7 KB/symbol (21KB → 14-16KB)

Validation:

cargo test -p ml --lib features::extraction --release -- --nocapture
# Verify: Memory benchmark shows <5KB overhead (vs 21KB VecDeque)

---

Agent G2: Lazy Feature Allocation (1 day)

Priority: P0 CRITICAL MCP Tools: Corrode (Rust memory analysis)

Files:

• ml/src/features/pipeline.rs
• ml/src/features/mod.rs

Task: Only allocate feature buffers when needed (sparse feature sets)

Expected Savings: 10-15 KB/symbol

Validation:

cargo test -p ml --lib features::pipeline --release
# Verify: Memory test shows 40-50% reduction for sparse feature sets

---

Agent G3: Share Normalizers via Arc (1 day)

Priority: P0 CRITICAL MCP Tools: Corrode (Rust Arc/smart pointer analysis)

Files:

• ml/src/features/normalization.rs

Task: Use Arc instead of per-symbol clones

Expected Savings: 20 KB/symbol (225 normalizers × 1 instance vs 100K × 225)

Validation:

cargo test -p ml --lib features::normalization --release
# Verify: Single Arc instance shared across all symbols

---

Agent G4: Memory Pooling (1 day)

Priority: P0 CRITICAL MCP Tools: Corrode (Rust memory pooling) + Zen thinkdeep (pooling design)

Files:

• ml/src/features/mod.rs
• ml/Cargo.toml (add object_pool crate)

Task: Implement memory pool for reusable buffers

Expected Savings: 5 KB/symbol

Validation:

SQLX_OFFLINE=false cargo test -p ml --test wave_d_memory_stress_test wave_d_memory_stress_100k_symbols --release -- --ignored --nocapture
# Verify: Memory <1,500MB for 100K symbols (vs 5,463MB current)

---

Agent G5: Regime Multipliers → Trading Agent (4 hours)

Priority: P1 HIGH MCP Tools: SkyDeck (file search) + Zen challenge (hypothesis validation)

Files:

• services/trading_agent_service/src/allocation.rs (update calculate_allocation)
• adaptive-strategy/src/risk/ppo_position_sizer.rs (regime multipliers)

Task: Integrate position sizing multipliers into Trading Agent

Validation:

cargo test -p trading_agent_service --lib allocation --release
# Verify: 1.5x position in TRENDING, 0.5x in VOLATILE, 0.2x in CRISIS

---

Agent G6: Dynamic Stops Integration (4 hours)

Priority: P1 HIGH MCP Tools: SkyDeck (file search)

Files:

• adaptive-strategy/src/execution/mod.rs (dynamic_stop_loss method)
• services/trading_agent_service/src/orders.rs (order submission with stops)

Task: Connect ATR-based dynamic stops to Trading Agent execution

Validation:

cargo test -p adaptive-strategy --lib execution --release
# Verify: 2.0x ATR stop in NORMAL, 4.0x in VOLATILE

---

Wave 2: Regime Sharpe, TFT, E2E Setup (4 agents, 2 days)

Agent G7: Regime-Conditioned Sharpe → Trading Agent (2 hours)

Priority: P1 HIGH MCP Tools: SkyDeck (file search)

Files:

• services/trading_agent_service/src/lib.rs (decision loop)
• adaptive-strategy/src/ensemble/weight_optimizer.rs (regime Sharpe)

Task: Add regime-conditioned Sharpe to Trading Agent decision-making

Validation:

cargo test -p trading_agent_service --lib --release
# Verify: Trading Agent selects models with highest regime-conditioned Sharpe

---

Agent G8: TFT Model Architecture Update (1.5 hours)

Priority: P1 HIGH MCP Tools: Corrode (Rust code analysis)

Files:

• ml/src/tft/model.rs (TFTConfig.input_dim: 50 → 225)
• ml/src/tft/config.rs (default config update)

Task: Update TFT config to support 225 features

Validation:

cargo test -p ml --lib tft::model --release
# Verify: Unit test creates TFT with 225-dim input

---

Agent G9: TFT Training Pipeline Update (1.5 hours)

Priority: P1 HIGH MCP Tools: Corrode (Rust code analysis)

Files:

• ml/examples/train_tft_dbn.rs (use WaveDFeatureConfig)
• ml/src/trainers/tft.rs (feature count validation)

Task: Update TFT training to use WaveDFeatureConfig (225 features)

Validation:

cargo run -p ml --example train_tft_dbn --release -- --epochs 1
# Verify: Training script creates 225-feature tensors

---

Agent G10: ES.FUT E2E Validation (4 hours)

Priority: P2 MEDIUM MCP Tools: SkyDeck (E2E test execution)

Files:

• ml/tests/wave_d_e2e_es_fut_225_features_test.rs (4 tests)

Task: Run end-to-end ES.FUT validation with all Phase 6 fixes

Validation:

cargo test -p ml --test wave_d_e2e_es_fut_225_features_test --no-fail-fast -- --nocapture
# Verify: 4/4 tests pass with <10ms latency

---

Wave 3: Multi-Asset E2E Validation (4 agents, 2 days)

Agent G11: NQ.FUT E2E Validation (4 hours)

Priority: P2 MEDIUM MCP Tools: SkyDeck (E2E test execution)

Files:

• ml/tests/wave_d_e2e_nq_fut_225_features_test.rs (3 tests)

Task: Run end-to-end NQ.FUT validation

Validation:

cargo test -p ml --test wave_d_e2e_nq_fut_225_features_test --no-fail-fast -- --nocapture
# Verify: 3/3 tests pass, regime distribution matches Phase 5

---

Agent G12: 6E.FUT E2E Validation (4 hours)

Priority: P2 MEDIUM MCP Tools: SkyDeck (E2E test execution)

Files:

• ml/tests/transition_6e_fut_integration_test.rs (3 tests)

Task: Run end-to-end 6E.FUT validation

Validation:

cargo test -p ml --test transition_6e_fut_integration_test --no-fail-fast -- --nocapture
# Verify: 3/3 tests pass, Ranging dominance confirmed

---

Agent G13: ZN.FUT E2E Validation (4 hours)

Priority: P2 MEDIUM MCP Tools: SkyDeck (E2E test execution)

Files:

• ml/tests/wave_d_e2e_zn_fut_225_features_test.rs (5 tests)

Task: Run end-to-end ZN.FUT validation

Validation:

SQLX_OFFLINE=false cargo test -p ml --test wave_d_e2e_zn_fut_225_features_test --no-fail-fast -- --nocapture
# Verify: 5/5 tests pass, 88.9% Normal regime maintained

---

Agent G14: Multi-Symbol Stress Test (4 hours)

Priority: P0 CRITICAL MCP Tools: SkyDeck (stress test execution)

Files:

• ml/tests/wave_d_memory_stress_test.rs (100K symbols)

Task: Run multi-asset stress test with all 4 symbols after G1-G4 optimizations

Validation:

SQLX_OFFLINE=false cargo test -p ml --test wave_d_memory_stress_test wave_d_memory_stress_100k_symbols --release -- --ignored --nocapture
# Verify: Memory <1,500MB for 100K symbols (3x better than Phase 5)

---

Wave 4: Performance, Deployment, ML Prep (10 agents, 3 days)

Agent G15: Wave D Features Benchmark (4 hours)

Priority: P2 MEDIUM MCP Tools: SkyDeck (benchmark execution)

Files:

• ml/benches/wave_d_features_bench.rs

Task: Run Wave D features benchmark regression

Validation:

cargo bench -p ml --bench wave_d_features_bench
# Verify: All features <100μs P99

---

Agent G16: Alternative Bars Benchmark Regression (4 hours)

Priority: P2 MEDIUM MCP Tools: SkyDeck (benchmark execution)

Files:

• ml/benches/alternative_bars_bench.rs

Task: Run alternative bars benchmark regression

Validation:

SQLX_OFFLINE=false cargo bench -p ml --bench alternative_bars_bench
# Verify: 0.70ms DBN loading maintained (Wave B baseline)

---

Agent G17: Profiling Test (4 hours)

Priority: P2 MEDIUM MCP Tools: SkyDeck (profiling test execution)

Files:

• ml/tests/wave_d_profiling_test.rs

Task: Execute profiling test (feature extraction breakdown)

Validation:

SQLX_OFFLINE=false cargo test -p ml --test wave_d_profiling_test --release --no-default-features -- --ignored --nocapture
# Verify: CUSUM <1μs, ADX <2μs, normalization <1μs

---

Agent G18: Latency Distribution Validation (4 hours)

Priority: P2 MEDIUM MCP Tools: SkyDeck (latency test execution)

Files:

• ml/tests/wave_d_e2e_es_fut_225_features_test.rs (latency test)

Task: Validate latency distribution (P50, P95, P99)

Validation:

cargo test -p ml --test wave_d_e2e_es_fut_225_features_test -- test_latency --release --nocapture
# Verify: P99 <10ms for complete 225-feature extraction

---

Agent G19: Release Build Compilation (4 hours)

Priority: P2 MEDIUM MCP Tools: SkyDeck (build execution)

Task: Compile all services in release mode

Validation:

time cargo build --release --workspace
# Verify: All 6 services compile in <5 minutes

---

Agent G20: Docker Image Builds (4 hours)

Priority: P2 MEDIUM MCP Tools: SkyDeck (Docker execution)

Files:

• Dockerfile
• docker-compose.yml

Task: Build Docker images for all services

Validation:

docker-compose build
docker-compose up -d
docker-compose ps
# Verify: All images build successfully, health checks pass

---

Agent G21: Staging Deployment (4 hours)

Priority: P2 MEDIUM MCP Tools: SkyDeck (deployment execution)

Task: Deploy to staging environment

Validation:

# Staging deployment commands (TBD)
# Verify: All services start, gRPC health checks pass

---

Agent G22: Prometheus + Grafana Dashboards (4 hours)

Priority: P2 MEDIUM MCP Tools: SkyDeck (monitoring configuration)

Task: Configure Prometheus + Grafana dashboards for regime metrics

Validation:

curl http://localhost:9090/api/v1/targets
curl http://localhost:3000/api/health
# Verify: Regime transition metrics visible in Grafana

---

Agent G23: Validate 225-Feature Training Pipeline (4 hours)

Priority: P2 MEDIUM MCP Tools: SkyDeck (training pipeline validation)

Files:

• ml/src/data_loaders/dbn_sequence_loader.rs

Task: Validate 225-feature training data pipeline

Validation:

cargo test -p ml --lib data_loaders::dbn_sequence_loader --release
# Verify: Batch creation uses 225 features

---

Agent G24: Update All Training Scripts (4 hours)

Priority: P2 MEDIUM MCP Tools: SkyDeck (training scripts update)

Files:

• ml/examples/train_dqn.rs
• ml/examples/train_ppo.rs
• ml/examples/train_mamba2_dbn.rs
• ml/examples/train_tft_dbn.rs

Task: Update training scripts for all models to use WaveDFeatureConfig

Validation:

# Run each training script with --epochs 1 to verify 225 features
cargo run -p ml --example train_dqn --release -- --epochs 1
cargo run -p ml --example train_ppo --release -- --epochs 1
cargo run -p ml --example train_mamba2_dbn --release -- --epochs 1
cargo run -p ml --example train_tft_dbn --release -- --epochs 1
# Verify: All scripts use WaveDFeatureConfig

---

Execution Strategy

Option A: Automated Wave Execution (Recommended when resources available)

Command:

# Clean up old processes first
ps aux | grep -E "cargo (test|check|build|bench)" | grep -v grep | awk '{if ($10 > 600) print $2}' | xargs -I {} kill -9 {}

# Spawn Wave 1 (6 agents: G1-G6)
# Wait 2 minutes
# Spawn Wave 2 (4 agents: G7-G10)
# Wait 2 minutes
# Spawn Wave 3 (4 agents: G11-G14)
# Wait 2 minutes
# Spawn Wave 4 (10 agents: G15-G24)

Resource Controls:

• Max 6 agents in parallel per wave
• 2-minute pause between waves
• Background process cleanup before each wave

---

Option B: Manual Wave-by-Wave Execution (Current Recommendation)

Given current system load, execute agents manually:

Week 1: Critical Fixes (G1-G9)

• Day 1-2: G1 (Ring buffer) + G2 (Lazy allocation)
• Day 3-4: G3 (Arc normalizers) + G4 (Memory pooling)
• Day 5: G5 (Regime multipliers) + G6 (Dynamic stops)
• Day 6: G7 (Regime Sharpe) + G8 (TFT architecture)
• Day 7: G9 (TFT training) + G10 (ES.FUT E2E)

Week 2: Validation & Deployment (G11-G24)

• Day 8-9: G11-G14 (Multi-asset E2E + stress test)
• Day 10: G15-G18 (Performance regression)
• Day 11: G19-G20 (Release builds + Docker)
• Day 12: G21-G22 (Staging + monitoring)
• Day 13: G23-G24 (ML training prep)
• Day 14: Final validation & documentation

---

Expected Outcomes

Post-G1-G4 (Memory Optimization)

• Memory: 55.95 KB/symbol → 10-15 KB/symbol (73-82% reduction)
• 100K Symbols: 5,463MB → <1,500MB (within acceptable range)
• Status: P0 CRITICAL blocker resolved

Post-G5-G7 (Regime Integration)

• Trading Agent: Uses regime multipliers (1.0x normal, 1.5x trending, 0.5x volatile, 0.2x crisis)
• Dynamic Stops: ATR-based with regime multipliers (2.0x-4.0x)
• Status: P1 HIGH blocker resolved

Post-G8-G9 (TFT Integration)

• TFT: Supports 225 features (vs 50 hardcoded)
• Training: Uses WaveDFeatureConfig
• Status: P1 HIGH blocker resolved

Post-G10-G14 (E2E Validation)

• Multi-Asset: 15/15 tests pass (ES.FUT, NQ.FUT, 6E.FUT, ZN.FUT)
• Latency: <10ms P99 for 225-feature extraction
• Status: Production validation complete

Post-G15-G24 (Deployment Ready)

• Benchmarks: All performance targets maintained
• Release Builds: All services compile successfully
• Docker: All images ready for deployment
• Monitoring: Grafana dashboards configured
• ML Training: All scripts ready for 225-feature retraining
• Status: 100% production readiness achieved

---

Next Steps

1. Immediate: Wait for Phase 5 background processes to complete
2. Option A: Spawn 24 agents in 4 staggered waves (automated)
3. Option B: Execute agents manually wave-by-wave (recommended given current load)
4. Timeline: 2 weeks to 100% production readiness
5. Post-Phase 6: ML model retraining (4-6 weeks per ML_TRAINING_ROADMAP.md)

---

Critical Success Factors

✅ Memory optimization (G1-G4) achieves 73-82% reduction ✅ Regime integration (G5-G7) connects Trading Agent to adaptive strategies ✅ TFT integration (G8-G9) supports 225 features ✅ Multi-asset E2E (G10-G14) validates all fixes ✅ Performance regression (G15-G18) maintains 72x improvement ✅ Deployment readiness (G19-G24) enables staging deployment

Deployment Readiness: 95% → 100% (via Phase 6 execution)

---

Report Generated: 2025-10-18 Status: 📋 READY FOR EXECUTION Recommendation: Execute Option B (manual wave-by-wave) given current system load Expected Duration: 2 weeks Target: 100% Production Readiness