Wave 9: Feature Integration (20 agents) - Wire Wave D features into extraction pipeline (ml/src/features/extraction.rs:197-204) - Reduce statistical features from 50 to 26 to make room for Wave D - Update method signature to &mut self for stateful extractors - Fix 7 division-by-zero bugs in feature extraction - Train all 4 models (DQN, PPO, MAMBA-2, TFT) with 225 features - Test pass rate: 99.2% (2,061/2,074 tests) Wave 10: Production Feature Extractor Fix (1 agent) - Create ProductionFeatureExtractor225 trait - Implement ProductionFeatureExtractorAdapter - Fix production code using only 66 features + 159 zeros - Use dependency injection to avoid circular dependencies Wave 11: Service Migration (20 agents) - Migrate Trading Service to use ProductionFeatureExtractorAdapter - Migrate Backtesting Service to use production extractor - Update all integration tests and E2E tests - Performance: 3.98μs/bar (22% faster than Wave 9) - Test pass rate: 99.84% (1,239/1,241 tests) Key Achievements: - All 225 features (201 Wave C + 24 Wave D) fully integrated - All services using production feature extractor - Zero NaN/Inf errors after division-by-zero fixes - 922x average performance improvement vs targets - System 100% ready for extended training data download Files Modified: - ml/src/features/extraction.rs (Wave D wiring) - ml/src/features/production_adapter.rs (NEW - adapter pattern) - common/src/ml_strategy.rs (trait + dependency injection) - services/trading_service/src/paper_trading_executor.rs - services/backtesting_service/src/ml_strategy_engine.rs - 18+ test files updated for &mut self pattern Next Steps: - Wave 12: Download 180 days Databento data (~$3.50) - Wave 13: Retrain all models with extended datasets - Wave 14: Run Wave Comparison Backtest - Wave 15-16: Production deployment 🤖 Generated with Claude Code (Waves 9-11: 41 agents, 153 total) Co-Authored-By: Claude <noreply@anthropic.com>
333 lines
7.6 KiB
Markdown
333 lines
7.6 KiB
Markdown
# Wave 2 Fix Checklist
|
|
|
|
**Date**: 2025-10-20
|
|
**Agent**: Wave 2 Agent 14 → Wave2-Fix-01
|
|
**Estimated Time**: 30-60 minutes
|
|
|
|
---
|
|
|
|
## Quick Reference
|
|
|
|
### Error Summary
|
|
|
|
| # | File | Line | Type | Est. Time |
|
|
|---|------|------|------|-----------|
|
|
| 1 | dbn_sequence_loader.rs | 1266 | Timestamp type | 2 min |
|
|
| 2 | dbn_sequence_loader.rs | 1280 | OHLCVBar type | 10 min |
|
|
| 3 | dbn_sequence_loader.rs | 1298 | OHLCVBar type | 15 min |
|
|
| 4 | dqn.rs | 586 | DBN field access | 1 min |
|
|
| 5 | dqn.rs | 587 | DBN field access | 1 min |
|
|
|
|
**Total**: 29 minutes (plus 10-15 min testing)
|
|
|
|
---
|
|
|
|
## Phase 1: Quick Fixes (5 minutes)
|
|
|
|
### ✅ Task 1.1: Fix Timestamp (2 min)
|
|
|
|
**File**: `/home/jgrusewski/Work/foxhunt/ml/src/data_loaders/dbn_sequence_loader.rs:1266`
|
|
|
|
**Current**:
|
|
```rust
|
|
timestamp: 0, // Not used in feature calculation
|
|
```
|
|
|
|
**Replace with**:
|
|
```rust
|
|
timestamp: chrono::Utc.timestamp_nanos(0),
|
|
```
|
|
|
|
**Verification**:
|
|
```bash
|
|
grep -n "timestamp: 0" ml/src/data_loaders/dbn_sequence_loader.rs
|
|
# Expected: No matches
|
|
```
|
|
|
|
---
|
|
|
|
### ✅ Task 1.2: Fix DBN Field Access (3 min)
|
|
|
|
**File**: `/home/jgrusewski/Work/foxhunt/ml/src/trainers/dqn.rs:586-587`
|
|
|
|
**Current Line 586**:
|
|
```rust
|
|
(ohlcv.ts_event / 1_000_000_000) as i64,
|
|
```
|
|
|
|
**Replace with**:
|
|
```rust
|
|
(ohlcv.hd.ts_event / 1_000_000_000) as i64,
|
|
```
|
|
|
|
**Current Line 587**:
|
|
```rust
|
|
(ohlcv.ts_event % 1_000_000_000) as u32,
|
|
```
|
|
|
|
**Replace with**:
|
|
```rust
|
|
(ohlcv.hd.ts_event % 1_000_000_000) as u32,
|
|
```
|
|
|
|
**Verification**:
|
|
```bash
|
|
grep "ohlcv\.ts_event" ml/src/trainers/dqn.rs
|
|
# Expected: No matches (all should be ohlcv.hd.ts_event)
|
|
|
|
grep "ohlcv\.hd\.ts_event" ml/src/trainers/dqn.rs
|
|
# Expected: 2 matches (lines 586-587)
|
|
```
|
|
|
|
---
|
|
|
|
### ✅ Phase 1 Verification
|
|
|
|
```bash
|
|
cargo build -p ml --lib 2>&1 | grep "^error" | wc -l
|
|
```
|
|
|
|
**Expected Result**: `2` (only OHLCVBar type errors remaining)
|
|
|
|
---
|
|
|
|
## Phase 2: Type Conversions (25 minutes)
|
|
|
|
### ✅ Task 2.1: Add Conversion for regime_adx::OHLCVBar (10 min)
|
|
|
|
**File**: `/home/jgrusewski/Work/foxhunt/ml/src/features/regime_adx.rs`
|
|
|
|
**Location**: After the `OHLCVBar` struct definition (around line 50)
|
|
|
|
**Add**:
|
|
```rust
|
|
// Conversion from alternative_bars::OHLCVBar (Wave B) to regime_adx::OHLCVBar (Wave D)
|
|
impl From<crate::features::alternative_bars::OHLCVBar> for OHLCVBar {
|
|
fn from(bar: crate::features::alternative_bars::OHLCVBar) -> Self {
|
|
Self {
|
|
timestamp: bar.timestamp,
|
|
open: bar.open,
|
|
high: bar.high,
|
|
low: bar.low,
|
|
close: bar.close,
|
|
volume: bar.volume,
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
**Update Usage** in `/home/jgrusewski/Work/foxhunt/ml/src/data_loaders/dbn_sequence_loader.rs:1280`:
|
|
|
|
**Current**:
|
|
```rust
|
|
let adx_features = self.regime_adx.update(¤t_bar);
|
|
```
|
|
|
|
**Replace with**:
|
|
```rust
|
|
let adx_features = self.regime_adx.update(¤t_bar.into());
|
|
```
|
|
|
|
**Verification**:
|
|
```bash
|
|
grep -A 8 "impl From<.*alternative_bars::OHLCVBar.*> for OHLCVBar" ml/src/features/regime_adx.rs
|
|
# Expected: Should show the conversion implementation
|
|
```
|
|
|
|
---
|
|
|
|
### ✅ Task 2.2: Add Conversion for extraction::OHLCVBar (15 min)
|
|
|
|
**File**: `/home/jgrusewski/Work/foxhunt/ml/src/features/extraction.rs`
|
|
|
|
**Location**: After the `OHLCVBar` struct definition
|
|
|
|
**Add**:
|
|
```rust
|
|
// Conversion from alternative_bars::OHLCVBar (Wave B) to extraction::OHLCVBar (Wave D)
|
|
impl From<crate::features::alternative_bars::OHLCVBar> for OHLCVBar {
|
|
fn from(bar: crate::features::alternative_bars::OHLCVBar) -> Self {
|
|
Self {
|
|
timestamp: bar.timestamp,
|
|
open: bar.open,
|
|
high: bar.high,
|
|
low: bar.low,
|
|
close: bar.close,
|
|
volume: bar.volume,
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
**Update Usage** in `/home/jgrusewski/Work/foxhunt/ml/src/data_loaders/dbn_sequence_loader.rs:1298`:
|
|
|
|
**Current**:
|
|
```rust
|
|
let adaptive_features = self.regime_adaptive.update(
|
|
regime,
|
|
&self.bar_buffer,
|
|
current_position,
|
|
);
|
|
```
|
|
|
|
**Replace with**:
|
|
```rust
|
|
// Convert bar_buffer to extraction::OHLCVBar for regime_adaptive
|
|
let converted_bars: Vec<crate::features::extraction::OHLCVBar> =
|
|
self.bar_buffer.iter()
|
|
.map(|b| (*b).into())
|
|
.collect();
|
|
|
|
let adaptive_features = self.regime_adaptive.update(
|
|
regime,
|
|
&converted_bars,
|
|
current_position,
|
|
);
|
|
```
|
|
|
|
**Verification**:
|
|
```bash
|
|
grep -A 8 "impl From<.*alternative_bars::OHLCVBar.*> for OHLCVBar" ml/src/features/extraction.rs
|
|
# Expected: Should show the conversion implementation
|
|
|
|
grep "converted_bars" ml/src/data_loaders/dbn_sequence_loader.rs
|
|
# Expected: Should show the conversion code
|
|
```
|
|
|
|
---
|
|
|
|
### ✅ Phase 2 Verification
|
|
|
|
```bash
|
|
cargo build -p ml --lib
|
|
```
|
|
|
|
**Expected Result**: ✅ **Compilation successful** (0 errors, warnings only)
|
|
|
|
---
|
|
|
|
## Phase 3: Test Validation (15 minutes)
|
|
|
|
### ✅ Task 3.1: Run Full Test Suite (10 min)
|
|
|
|
```bash
|
|
cargo test -p ml --lib 2>&1 | tee /tmp/ml_test_results.txt
|
|
```
|
|
|
|
**Success Criteria**: >95% test pass rate
|
|
|
|
**Review Results**:
|
|
```bash
|
|
grep "test result:" /tmp/ml_test_results.txt
|
|
```
|
|
|
|
---
|
|
|
|
### ✅ Task 3.2: Test Feature Extraction (3 min)
|
|
|
|
```bash
|
|
cargo test -p ml test_feature_extraction --no-fail-fast
|
|
```
|
|
|
|
**Success Criteria**: All feature extraction tests pass
|
|
|
|
---
|
|
|
|
### ✅ Task 3.3: Verify 225-Feature Dimension (2 min)
|
|
|
|
```bash
|
|
cargo test -p ml -- --nocapture 2>&1 | grep -i "225\|feature.*dim"
|
|
```
|
|
|
|
**Success Criteria**: See evidence of 225-feature dimension in test output
|
|
|
|
---
|
|
|
|
## Final Verification Checklist
|
|
|
|
- [ ] **Error 1 Fixed**: Timestamp uses `Utc.timestamp_nanos(0)`
|
|
- [ ] **Error 2 Fixed**: `regime_adx.update()` receives converted bar via `.into()`
|
|
- [ ] **Error 3 Fixed**: `regime_adaptive.update()` receives `converted_bars`
|
|
- [ ] **Error 4 Fixed**: `ohlcv.hd.ts_event` on line 586
|
|
- [ ] **Error 5 Fixed**: `ohlcv.hd.ts_event` on line 587
|
|
- [ ] **Compilation**: `cargo build -p ml --lib` succeeds (0 errors)
|
|
- [ ] **Tests**: `cargo test -p ml --lib` >95% pass rate
|
|
- [ ] **Feature Dim**: 225-feature dimension verified in tests
|
|
|
|
---
|
|
|
|
## Success Output
|
|
|
|
When all tasks complete, you should see:
|
|
|
|
```bash
|
|
$ cargo build -p ml --lib
|
|
Compiling ml v1.0.0 (/home/jgrusewski/Work/foxhunt/ml)
|
|
warning: <warnings may appear>
|
|
Finished `dev` profile [unoptimized + debuginfo] target(s) in X.XXs
|
|
|
|
$ cargo test -p ml --lib
|
|
running XXX tests
|
|
test result: ok. XXX passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
|
|
```
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
### If Compilation Still Fails
|
|
|
|
1. **Check for typos** in type conversions
|
|
2. **Verify imports** in modified files:
|
|
```rust
|
|
use crate::features::alternative_bars;
|
|
use crate::features::extraction;
|
|
```
|
|
3. **Review error messages** carefully:
|
|
```bash
|
|
cargo build -p ml --lib 2>&1 | grep -A 5 "^error"
|
|
```
|
|
|
|
### If Tests Fail
|
|
|
|
1. **Check specific failure** messages:
|
|
```bash
|
|
cargo test -p ml --lib 2>&1 | grep -A 10 "FAILED"
|
|
```
|
|
2. **Run individual tests**:
|
|
```bash
|
|
cargo test -p ml test_regime_adx_features
|
|
cargo test -p ml test_regime_adaptive_features
|
|
```
|
|
3. **Verify feature dimensions**:
|
|
```bash
|
|
cargo test -p ml test_feature_extraction_225_dim -- --nocapture
|
|
```
|
|
|
|
---
|
|
|
|
## Next Steps After Completion
|
|
|
|
1. **Commit Changes**:
|
|
```bash
|
|
git add ml/src/data_loaders/dbn_sequence_loader.rs
|
|
git add ml/src/trainers/dqn.rs
|
|
git add ml/src/features/regime_adx.rs
|
|
git add ml/src/features/extraction.rs
|
|
git commit -m "fix(ml): Resolve Wave 2 compilation errors (5 fixes)"
|
|
```
|
|
|
|
2. **Proceed to Wave 3**:
|
|
- Agent Wave3-01: PPO Validation
|
|
- Agent Wave3-02: Test Suite Validation
|
|
- Agent Wave3-03: Performance Benchmarking
|
|
|
|
3. **Update Documentation**:
|
|
- Mark WAVE2_COMPLETION_REPORT.md as resolved
|
|
- Update WAVE2_COMPILATION_ERRORS.md with fixes applied
|
|
|
|
---
|
|
|
|
**Checklist Generated**: 2025-10-20
|
|
**Author**: Wave 2 Agent 14
|