8ce7c52586
- Fixed feature dimension mismatch in evaluate_dqn_main_orchestrator.rs - Updated all 5 occurrences: state_dim, input comments, feature vector type - Aligned with Wave 16D training (128 features: 125 market + 3 portfolio) Issue: Validation backtest reveals 100% HOLD action collapse - requires reward system investigation and redesign per latest RL research.
308 lines
11 KiB
Markdown
308 lines
11 KiB
Markdown
# DQN Test Suite Validation Report
|
||
**Date**: 2025-11-08
|
||
**Status**: ⚠️ FAILURES DETECTED - Production deployment BLOCKED
|
||
|
||
## Executive Summary
|
||
|
||
**Test Results**: 142/160 passed (88.8% pass rate) - **BELOW 100% REQUIREMENT**
|
||
- **17 failures** across 3 categories
|
||
- **1 ignored** (requires real DBN files + GPU)
|
||
- **Integration tests**: COMPILATION FAILED (13 errors, 4 warnings)
|
||
|
||
**Production Readiness**: ❌ **NOT READY** - Critical bugs in portfolio tracking, hyperopt constraints, and feature dimensions
|
||
|
||
---
|
||
|
||
## Detailed Failure Analysis
|
||
|
||
### Category A: Feature Dimension Mismatch (7 failures)
|
||
**Root Cause**: State dimension is **131** but code/tests expect **128**
|
||
|
||
**Failed Tests**:
|
||
1. `test_feature_vector_to_state` - Assertion: 131 vs expected 128
|
||
2. `test_batched_action_selection` - Shape mismatch [10,131] vs [128,256]
|
||
3. `test_batched_vs_sequential_action_selection_consistency` - Shape mismatch [5,131] vs [128,256]
|
||
4. `test_batch_size_mismatch_larger_than_configured` - Shape mismatch [64,131] vs [128,256]
|
||
5. `test_batch_size_mismatch_smaller_than_configured` - Shape mismatch [16,131] vs [128,256]
|
||
6. `test_single_sample_batch` - Shape mismatch [1,131] vs [128,256]
|
||
|
||
**Impact**: CRITICAL - All batched inference operations failing
|
||
**Required Fix**: Determine correct dimension (131 or 128) and update network architecture + tests
|
||
|
||
**Dimension Breakdown** (current implementation):
|
||
- `feature_vector_to_state()` creates state from 225-dim FeatureVector
|
||
- Extracts: 4 price features + 221 technical indicators (indices 4-224)
|
||
- Adds: 3 portfolio features
|
||
- **Total: 4 + 221 + 3 = 228** (NOT 131!)
|
||
- **Actual observed: 131** - suggests truncation somewhere
|
||
|
||
**Action Required**:
|
||
- Trace exact feature extraction path
|
||
- Verify Wave 16D changes (125 market + 3 portfolio = 128 claim)
|
||
- Update network input_dim to match actual state dimension
|
||
|
||
---
|
||
|
||
### Category B: Portfolio Reward Calculation (6 failures)
|
||
**Root Cause**: Reward function always returns -1 (HOLD penalty) regardless of P&L
|
||
|
||
**Failed Tests**:
|
||
1. `test_pnl_reward_nonzero` - Expected positive reward for profitable BUY, got -1
|
||
2. `test_pnl_calculation_accuracy` - Both 1% and 5% profit returned -1
|
||
3. `test_reward_function_receives_portfolio` - Expected positive reward when portfolio value increases, got -1
|
||
4. `test_integration_full_trade_cycle` - Portfolio value mismatch: 11100 vs expected 11200
|
||
5. `test_portfolio_features_populated` - Short position value: 10900 vs expected 11100 (200 point error)
|
||
6. `test_portfolio_tracking_sell_action` - Short position value: 10900 vs expected 11100
|
||
|
||
**Impact**: CRITICAL - P&L-based rewards completely broken, agent cannot learn profitable strategies
|
||
|
||
**Observable Symptoms**:
|
||
- All profitable trades returning HOLD penalty (-1) instead of positive reward
|
||
- Short position P&L calculation off by exactly 200 points (10% error)
|
||
- Portfolio value not updating correctly after trades
|
||
|
||
**Action Required**:
|
||
- Fix `calculate_reward()` to use actual portfolio P&L from PortfolioTracker
|
||
- Verify short position value calculation (entry_price vs current_price logic)
|
||
- Add integration test for reward → P&L correlation
|
||
|
||
---
|
||
|
||
### Category C: Hyperopt Constraint Logic (4 failures)
|
||
**Root Cause**: HFT validation logic inverted + parameter bounds mismatch
|
||
|
||
**Failed Tests**:
|
||
1. `test_dqn_params_bounds` - Batch size range (32, 230) vs expected (80, 220)
|
||
2. `test_hft_constraint_minimum_penalty` - Valid params rejected by HFT constraint
|
||
3. `test_hft_constraint_training_instability` - Invalid params accepted by HFT constraint
|
||
4. `test_hft_constraint_buffer_size` - Invalid params accepted by HFT constraint
|
||
5. `test_dqn_params_roundtrip` - Gamma precision loss during encode/decode
|
||
|
||
**Impact**: MODERATE - Hyperopt may accept invalid configurations or reject valid ones
|
||
|
||
**Batch Size Mismatch**:
|
||
- Wave 16I expanded range to 32-230 (GPU limit fix)
|
||
- Tests still expect old range 80-220
|
||
- **Resolution**: Update test expectations or revert to 80-220
|
||
|
||
**HFT Constraint Bugs**:
|
||
- `validate_for_hft_trendfollowing()` returns Ok when should return Err (and vice versa)
|
||
- Suggests boolean logic inversion or incorrect threshold comparisons
|
||
- 3 constraint rules failing: minimum penalty, training instability, buffer size
|
||
|
||
**Action Required**:
|
||
- Review `validate_for_hft_trendfollowing()` implementation line-by-line
|
||
- Update test expectations to match Wave 16I parameter ranges
|
||
- Fix gamma roundtrip precision (use epsilon comparison instead of exact equality)
|
||
|
||
---
|
||
|
||
### Category D: Integration Tests (COMPILATION FAILED)
|
||
**File**: `ml/tests/dqn_realistic_constraints_integration.rs`
|
||
|
||
**Errors** (13 compilation errors):
|
||
1. Missing field `warmup_steps` in `DQNHyperparameters` initializer (line 60)
|
||
2-13. Type mismatches: f32 vs f64 in slippage calculations (lines 331-361)
|
||
- `apply_slippage()` expects f64, `execute_action()` expects f32
|
||
- Multiple arithmetic operations between f32 and f64
|
||
|
||
**Warnings** (4):
|
||
- Unused imports: `TrainingMetrics`, `Decimal`, `TempDir`
|
||
- Unused mut: `trainer` variable
|
||
|
||
**Impact**: MODERATE - Integration tests cannot run, realistic constraint validation blocked
|
||
|
||
**Action Required**:
|
||
- Add `warmup_steps: 0` to DQNHyperparameters initialization
|
||
- Cast all prices to consistent type (either f32 or f64 throughout)
|
||
- Remove unused imports and mut annotation
|
||
|
||
---
|
||
|
||
## Test Coverage Analysis
|
||
|
||
**Unit Tests**: 160 total
|
||
- **Passed**: 142 (88.8%)
|
||
- **Failed**: 17 (10.6%)
|
||
- **Ignored**: 1 (0.6%)
|
||
|
||
**By Module**:
|
||
| Module | Passed | Failed | Rate |
|
||
|--------|--------|--------|------|
|
||
| dqn::agent | 10/10 | 0 | 100% |
|
||
| dqn::dqn | 8/8 | 0 | 100% |
|
||
| dqn::network | 5/5 | 0 | 100% |
|
||
| dqn::portfolio_tracker | 9/9 | 0 | 100% |
|
||
| dqn::reward | 4/4 | 0 | 100% |
|
||
| dqn::tests::portfolio_integration | 3/9 | 6 | 33% ❌ |
|
||
| trainers::dqn | 5/11 | 6 | 45% ❌ |
|
||
| hyperopt::adapters::dqn | 2/7 | 5 | 29% ❌ |
|
||
| benchmark::dqn_benchmark | 3/3 | 0 | 100% |
|
||
| integration::strategy_dqn_bridge | 5/5 | 0 | 100% |
|
||
|
||
**Critical Failures**:
|
||
- Portfolio integration tests: 67% failure rate (6/9 tests)
|
||
- Hyperopt adapter tests: 71% failure rate (5/7 tests)
|
||
- Trainer batch tests: 55% failure rate (6/11 tests)
|
||
|
||
---
|
||
|
||
## Risk Assessment
|
||
|
||
### CRITICAL Risks (Production Blockers)
|
||
1. **Portfolio Reward Broken**: Agent cannot learn - all profitable trades return -1
|
||
2. **Feature Dimension Mismatch**: Batched inference crashes - hyperopt will fail
|
||
3. **Short Position P&L**: 200 point calculation error - risk management failure
|
||
|
||
### HIGH Risks (Operational Issues)
|
||
4. **HFT Constraint Logic**: May accept unstable configurations or reject valid ones
|
||
5. **Integration Tests Broken**: Cannot validate realistic trading scenarios
|
||
|
||
### MODERATE Risks (Data Quality)
|
||
6. **Batch Size Range**: Tests expect 80-220, code uses 32-230 (documentation drift)
|
||
7. **Gamma Roundtrip**: Precision loss may cause hyperopt parameter drift
|
||
|
||
---
|
||
|
||
## Smoke Test Recommendation
|
||
|
||
**Status**: ⚠️ **SKIP SMOKE TESTS** - Critical unit test failures must be resolved first
|
||
|
||
**Rationale**:
|
||
- Feature dimension mismatch will cause immediate crashes in `train_dqn` example
|
||
- Portfolio reward bug means training will produce meaningless models
|
||
- 88.8% pass rate is below production threshold (95%+ required)
|
||
|
||
**Next Steps** (before smoke tests):
|
||
1. Fix feature dimension issue (expected ~1 hour)
|
||
2. Fix portfolio reward calculation (expected ~2 hours)
|
||
3. Fix HFT constraint validation (expected ~1 hour)
|
||
4. Re-run unit tests until 100% pass rate achieved
|
||
5. Fix integration test compilation errors (expected ~30 min)
|
||
6. THEN proceed to smoke tests
|
||
|
||
---
|
||
|
||
## Production Certification Status
|
||
|
||
**Current**: ❌ **FAILED** - 88.8% pass rate (below 95% threshold)
|
||
|
||
**Blockers**:
|
||
1. 17 unit test failures across 3 critical categories
|
||
2. Integration tests failing to compile
|
||
3. Portfolio reward calculation completely broken
|
||
4. Feature dimension mismatch (131 vs 128)
|
||
|
||
**Required for Certification**:
|
||
- [ ] 100% unit test pass rate (currently 88.8%)
|
||
- [ ] Integration tests compiling and passing
|
||
- [ ] Smoke test: 5-epoch training completes successfully
|
||
- [ ] Smoke test: 5-trial hyperopt completes without crashes
|
||
- [ ] Code review of all fixes
|
||
|
||
**Estimated Time to Fix**: 4-5 hours (3 categories + integration tests + re-validation)
|
||
|
||
---
|
||
|
||
## Comparison to CLAUDE.md Claims
|
||
|
||
**CLAUDE.md States**:
|
||
> DQN Production Certified (2025-11-05)
|
||
> - Test Results: DQN Tests: 147/147 passing (100%) ✅
|
||
> - Production Readiness: ✅ CERTIFIED
|
||
|
||
**Reality** (2025-11-08):
|
||
- **160 tests exist** (not 147)
|
||
- **142/160 passing** (88.8%, not 100%)
|
||
- **17 failures** in critical paths
|
||
- **Integration tests broken**
|
||
|
||
**Conclusion**: CLAUDE.md status is **OUT OF DATE** or reflects a previous state before recent code changes. Recommend updating CLAUDE.md to reflect actual status: ⚠️ **PRODUCTION CERTIFICATION REVOKED**
|
||
|
||
---
|
||
|
||
## Recommendations
|
||
|
||
### Immediate Actions (Priority 1)
|
||
1. **Fix Feature Dimension Bug** (1 hour)
|
||
- Trace actual state vector creation
|
||
- Determine if 128 or 131 is correct
|
||
- Update network architecture or feature extraction
|
||
|
||
2. **Fix Portfolio Reward Bug** (2 hours)
|
||
- Review `calculate_reward()` implementation
|
||
- Fix short position P&L calculation (200 point error)
|
||
- Ensure rewards correlate with portfolio value changes
|
||
|
||
3. **Fix HFT Constraint Logic** (1 hour)
|
||
- Review `validate_for_hft_trendfollowing()`
|
||
- Fix inverted validation logic
|
||
- Update batch size range expectations
|
||
|
||
### Medium Priority
|
||
4. **Fix Integration Tests** (30 min)
|
||
- Add `warmup_steps` field
|
||
- Standardize f32/f64 types
|
||
- Remove unused imports
|
||
|
||
5. **Update Test Suite** (1 hour)
|
||
- Add 13 missing tests to reach 160 total documented
|
||
- Update batch size range expectations
|
||
- Add epsilon comparison for gamma roundtrip
|
||
|
||
### Post-Fix Validation
|
||
6. **Re-run Full Test Suite** (10 min)
|
||
- Target: 160/160 passing (100%)
|
||
- Document any remaining issues
|
||
|
||
7. **Smoke Tests** (30 min)
|
||
- 5-epoch training
|
||
- 5-trial hyperopt
|
||
- Verify no crashes, reasonable metrics
|
||
|
||
8. **Update CLAUDE.md** (15 min)
|
||
- Reflect actual test count (160)
|
||
- Update production status
|
||
- Document fix wave (Wave 16J?)
|
||
|
||
**Total Estimated Effort**: 6-7 hours to production-ready state
|
||
|
||
---
|
||
|
||
## Appendix: Test Failure Details
|
||
|
||
### Feature Dimension Errors
|
||
```
|
||
Shape mismatch in matmul, lhs: [batch_size, 131], rhs: [128, 256]
|
||
Expected: state_dim=128 (125 market + 3 portfolio)
|
||
Actual: state_dim=131
|
||
Difference: +3 features (source unknown)
|
||
```
|
||
|
||
### Portfolio Reward Errors
|
||
```
|
||
Reward should be positive for profitable BUY trade, got -1
|
||
5% profit reward (-1) should be greater than 1% profit reward (-1)
|
||
Portfolio value: 11100.0 expected: 11200.0 (100 point shortfall)
|
||
Short position: 10900.0 expected: 11100.0 (200 point error)
|
||
```
|
||
|
||
### HFT Constraint Errors
|
||
```
|
||
Batch size range: (32.0, 230.0) expected: (80.0, 220.0)
|
||
assertion failed: params.validate_for_hft_trendfollowing().is_err() (was Ok)
|
||
assertion failed: params_valid.validate_for_hft_trendfollowing().is_ok() (was Err)
|
||
```
|
||
|
||
### Integration Test Errors
|
||
```
|
||
error[E0063]: missing field `warmup_steps` in initializer of `DQNHyperparameters`
|
||
error[E0308]: mismatched types - expected `f64`, found `f32` (×12 occurrences)
|
||
```
|
||
|
||
---
|
||
|
||
**Report Generated**: 2025-11-08
|
||
**Validator**: Claude Code Agent
|
||
**Next Review**: After fixes applied (target: 100% pass rate)
|