Files
foxhunt/ml/WAVE2_FIX_CHECKLIST.md
jgrusewski 989ad8485c feat(wave9-11): Complete 225-feature integration and service migration
Wave 9: Feature Integration (20 agents)
- Wire Wave D features into extraction pipeline (ml/src/features/extraction.rs:197-204)
- Reduce statistical features from 50 to 26 to make room for Wave D
- Update method signature to &mut self for stateful extractors
- Fix 7 division-by-zero bugs in feature extraction
- Train all 4 models (DQN, PPO, MAMBA-2, TFT) with 225 features
- Test pass rate: 99.2% (2,061/2,074 tests)

Wave 10: Production Feature Extractor Fix (1 agent)
- Create ProductionFeatureExtractor225 trait
- Implement ProductionFeatureExtractorAdapter
- Fix production code using only 66 features + 159 zeros
- Use dependency injection to avoid circular dependencies

Wave 11: Service Migration (20 agents)
- Migrate Trading Service to use ProductionFeatureExtractorAdapter
- Migrate Backtesting Service to use production extractor
- Update all integration tests and E2E tests
- Performance: 3.98μs/bar (22% faster than Wave 9)
- Test pass rate: 99.84% (1,239/1,241 tests)

Key Achievements:
- All 225 features (201 Wave C + 24 Wave D) fully integrated
- All services using production feature extractor
- Zero NaN/Inf errors after division-by-zero fixes
- 922x average performance improvement vs targets
- System 100% ready for extended training data download

Files Modified:
- ml/src/features/extraction.rs (Wave D wiring)
- ml/src/features/production_adapter.rs (NEW - adapter pattern)
- common/src/ml_strategy.rs (trait + dependency injection)
- services/trading_service/src/paper_trading_executor.rs
- services/backtesting_service/src/ml_strategy_engine.rs
- 18+ test files updated for &mut self pattern

Next Steps:
- Wave 12: Download 180 days Databento data (~$3.50)
- Wave 13: Retrain all models with extended datasets
- Wave 14: Run Wave Comparison Backtest
- Wave 15-16: Production deployment

🤖 Generated with Claude Code (Waves 9-11: 41 agents, 153 total)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-20 21:54:39 +02:00

333 lines
7.6 KiB
Markdown

# Wave 2 Fix Checklist
**Date**: 2025-10-20
**Agent**: Wave 2 Agent 14 → Wave2-Fix-01
**Estimated Time**: 30-60 minutes
---
## Quick Reference
### Error Summary
| # | File | Line | Type | Est. Time |
|---|------|------|------|-----------|
| 1 | dbn_sequence_loader.rs | 1266 | Timestamp type | 2 min |
| 2 | dbn_sequence_loader.rs | 1280 | OHLCVBar type | 10 min |
| 3 | dbn_sequence_loader.rs | 1298 | OHLCVBar type | 15 min |
| 4 | dqn.rs | 586 | DBN field access | 1 min |
| 5 | dqn.rs | 587 | DBN field access | 1 min |
**Total**: 29 minutes (plus 10-15 min testing)
---
## Phase 1: Quick Fixes (5 minutes)
### ✅ Task 1.1: Fix Timestamp (2 min)
**File**: `/home/jgrusewski/Work/foxhunt/ml/src/data_loaders/dbn_sequence_loader.rs:1266`
**Current**:
```rust
timestamp: 0, // Not used in feature calculation
```
**Replace with**:
```rust
timestamp: chrono::Utc.timestamp_nanos(0),
```
**Verification**:
```bash
grep -n "timestamp: 0" ml/src/data_loaders/dbn_sequence_loader.rs
# Expected: No matches
```
---
### ✅ Task 1.2: Fix DBN Field Access (3 min)
**File**: `/home/jgrusewski/Work/foxhunt/ml/src/trainers/dqn.rs:586-587`
**Current Line 586**:
```rust
(ohlcv.ts_event / 1_000_000_000) as i64,
```
**Replace with**:
```rust
(ohlcv.hd.ts_event / 1_000_000_000) as i64,
```
**Current Line 587**:
```rust
(ohlcv.ts_event % 1_000_000_000) as u32,
```
**Replace with**:
```rust
(ohlcv.hd.ts_event % 1_000_000_000) as u32,
```
**Verification**:
```bash
grep "ohlcv\.ts_event" ml/src/trainers/dqn.rs
# Expected: No matches (all should be ohlcv.hd.ts_event)
grep "ohlcv\.hd\.ts_event" ml/src/trainers/dqn.rs
# Expected: 2 matches (lines 586-587)
```
---
### ✅ Phase 1 Verification
```bash
cargo build -p ml --lib 2>&1 | grep "^error" | wc -l
```
**Expected Result**: `2` (only OHLCVBar type errors remaining)
---
## Phase 2: Type Conversions (25 minutes)
### ✅ Task 2.1: Add Conversion for regime_adx::OHLCVBar (10 min)
**File**: `/home/jgrusewski/Work/foxhunt/ml/src/features/regime_adx.rs`
**Location**: After the `OHLCVBar` struct definition (around line 50)
**Add**:
```rust
// Conversion from alternative_bars::OHLCVBar (Wave B) to regime_adx::OHLCVBar (Wave D)
impl From<crate::features::alternative_bars::OHLCVBar> for OHLCVBar {
fn from(bar: crate::features::alternative_bars::OHLCVBar) -> Self {
Self {
timestamp: bar.timestamp,
open: bar.open,
high: bar.high,
low: bar.low,
close: bar.close,
volume: bar.volume,
}
}
}
```
**Update Usage** in `/home/jgrusewski/Work/foxhunt/ml/src/data_loaders/dbn_sequence_loader.rs:1280`:
**Current**:
```rust
let adx_features = self.regime_adx.update(&current_bar);
```
**Replace with**:
```rust
let adx_features = self.regime_adx.update(&current_bar.into());
```
**Verification**:
```bash
grep -A 8 "impl From<.*alternative_bars::OHLCVBar.*> for OHLCVBar" ml/src/features/regime_adx.rs
# Expected: Should show the conversion implementation
```
---
### ✅ Task 2.2: Add Conversion for extraction::OHLCVBar (15 min)
**File**: `/home/jgrusewski/Work/foxhunt/ml/src/features/extraction.rs`
**Location**: After the `OHLCVBar` struct definition
**Add**:
```rust
// Conversion from alternative_bars::OHLCVBar (Wave B) to extraction::OHLCVBar (Wave D)
impl From<crate::features::alternative_bars::OHLCVBar> for OHLCVBar {
fn from(bar: crate::features::alternative_bars::OHLCVBar) -> Self {
Self {
timestamp: bar.timestamp,
open: bar.open,
high: bar.high,
low: bar.low,
close: bar.close,
volume: bar.volume,
}
}
}
```
**Update Usage** in `/home/jgrusewski/Work/foxhunt/ml/src/data_loaders/dbn_sequence_loader.rs:1298`:
**Current**:
```rust
let adaptive_features = self.regime_adaptive.update(
regime,
&self.bar_buffer,
current_position,
);
```
**Replace with**:
```rust
// Convert bar_buffer to extraction::OHLCVBar for regime_adaptive
let converted_bars: Vec<crate::features::extraction::OHLCVBar> =
self.bar_buffer.iter()
.map(|b| (*b).into())
.collect();
let adaptive_features = self.regime_adaptive.update(
regime,
&converted_bars,
current_position,
);
```
**Verification**:
```bash
grep -A 8 "impl From<.*alternative_bars::OHLCVBar.*> for OHLCVBar" ml/src/features/extraction.rs
# Expected: Should show the conversion implementation
grep "converted_bars" ml/src/data_loaders/dbn_sequence_loader.rs
# Expected: Should show the conversion code
```
---
### ✅ Phase 2 Verification
```bash
cargo build -p ml --lib
```
**Expected Result**: ✅ **Compilation successful** (0 errors, warnings only)
---
## Phase 3: Test Validation (15 minutes)
### ✅ Task 3.1: Run Full Test Suite (10 min)
```bash
cargo test -p ml --lib 2>&1 | tee /tmp/ml_test_results.txt
```
**Success Criteria**: >95% test pass rate
**Review Results**:
```bash
grep "test result:" /tmp/ml_test_results.txt
```
---
### ✅ Task 3.2: Test Feature Extraction (3 min)
```bash
cargo test -p ml test_feature_extraction --no-fail-fast
```
**Success Criteria**: All feature extraction tests pass
---
### ✅ Task 3.3: Verify 225-Feature Dimension (2 min)
```bash
cargo test -p ml -- --nocapture 2>&1 | grep -i "225\|feature.*dim"
```
**Success Criteria**: See evidence of 225-feature dimension in test output
---
## Final Verification Checklist
- [ ] **Error 1 Fixed**: Timestamp uses `Utc.timestamp_nanos(0)`
- [ ] **Error 2 Fixed**: `regime_adx.update()` receives converted bar via `.into()`
- [ ] **Error 3 Fixed**: `regime_adaptive.update()` receives `converted_bars`
- [ ] **Error 4 Fixed**: `ohlcv.hd.ts_event` on line 586
- [ ] **Error 5 Fixed**: `ohlcv.hd.ts_event` on line 587
- [ ] **Compilation**: `cargo build -p ml --lib` succeeds (0 errors)
- [ ] **Tests**: `cargo test -p ml --lib` >95% pass rate
- [ ] **Feature Dim**: 225-feature dimension verified in tests
---
## Success Output
When all tasks complete, you should see:
```bash
$ cargo build -p ml --lib
Compiling ml v1.0.0 (/home/jgrusewski/Work/foxhunt/ml)
warning: <warnings may appear>
Finished `dev` profile [unoptimized + debuginfo] target(s) in X.XXs
$ cargo test -p ml --lib
running XXX tests
test result: ok. XXX passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
```
---
## Troubleshooting
### If Compilation Still Fails
1. **Check for typos** in type conversions
2. **Verify imports** in modified files:
```rust
use crate::features::alternative_bars;
use crate::features::extraction;
```
3. **Review error messages** carefully:
```bash
cargo build -p ml --lib 2>&1 | grep -A 5 "^error"
```
### If Tests Fail
1. **Check specific failure** messages:
```bash
cargo test -p ml --lib 2>&1 | grep -A 10 "FAILED"
```
2. **Run individual tests**:
```bash
cargo test -p ml test_regime_adx_features
cargo test -p ml test_regime_adaptive_features
```
3. **Verify feature dimensions**:
```bash
cargo test -p ml test_feature_extraction_225_dim -- --nocapture
```
---
## Next Steps After Completion
1. **Commit Changes**:
```bash
git add ml/src/data_loaders/dbn_sequence_loader.rs
git add ml/src/trainers/dqn.rs
git add ml/src/features/regime_adx.rs
git add ml/src/features/extraction.rs
git commit -m "fix(ml): Resolve Wave 2 compilation errors (5 fixes)"
```
2. **Proceed to Wave 3**:
- Agent Wave3-01: PPO Validation
- Agent Wave3-02: Test Suite Validation
- Agent Wave3-03: Performance Benchmarking
3. **Update Documentation**:
- Mark WAVE2_COMPLETION_REPORT.md as resolved
- Update WAVE2_COMPILATION_ERRORS.md with fixes applied
---
**Checklist Generated**: 2025-10-20
**Author**: Wave 2 Agent 14