71 lines
2.9 KiB
Markdown
71 lines
2.9 KiB
Markdown
# Hyperopt Degenerate Trial Fix
|
|
|
|
## Problem
|
|
|
|
7/9 DQN hyperopt trials produce degenerate policies (all-buy or all-hold) that generate 0-1 trades in backtest evaluation. All degenerates receive identical objective=1.25, creating a flat plateau that gives PSO zero gradient signal.
|
|
|
|
Root cause chain:
|
|
1. 45-action factored space collapses to 3 legacy actions (Buy/Sell/Hold) at eval time
|
|
2. EvaluationEngine treats consecutive same-direction actions as no-ops
|
|
3. Objective function can't distinguish between failure modes (all-buy vs all-hold vs insufficient trades)
|
|
|
|
## Solution: Three-Part Fix
|
|
|
|
### Part 1: Exposure-Aware Evaluation Engine
|
|
|
|
Add `process_bar_factored(&FactoredAction)` to EvaluationEngine that tracks continuous position exposure (-1.0 to +1.0):
|
|
|
|
- Long100→Long50 generates a partial close trade (delta=-0.5)
|
|
- Long50→Short100 generates a reversal trade (delta=-1.5)
|
|
- Long100→Long100 = no trade (delta=0)
|
|
- Transaction cost uses FactoredAction's order type (Market=0.15%, Limit=0.05%, IoC=0.10%)
|
|
- Position sizing: `effective_size = target_exposure * kelly_fraction` (multiplicative)
|
|
|
|
File: `crates/ml/src/evaluation/engine.rs` (~50 lines)
|
|
|
|
### Part 2: Graduated Trade Insufficiency Penalty
|
|
|
|
Replace flat 1.25 plateau with smooth, differentiated penalties in `extract_objective()`:
|
|
|
|
```
|
|
min_expected_trades = total_bars / 500 (~448 for 224K bars)
|
|
|
|
penalty = match total_trades {
|
|
0 => 10.0, // Model does nothing
|
|
1..10 => 5.0 + 5.0 * (1.0 - trades/10.0), // Near-degenerate
|
|
10..min => 2.0 * (1.0 - trades/min_expected), // Insufficient activity
|
|
_ => 0.0, // Rely on Sharpe/Sortino
|
|
}
|
|
```
|
|
|
|
Additive on objective (higher=worse for PSO minimization). Each failure mode gets a unique value → PSO gets gradient signal.
|
|
|
|
File: `crates/ml/src/hyperopt/adapters/dqn.rs` (~25 lines in `extract_objective()`)
|
|
|
|
### Part 3: Wire Factored Eval into Hyperopt Backtest
|
|
|
|
Replace legacy action collapse in backtest loop:
|
|
```rust
|
|
// Before: factored → legacy → Action::Buy/Sell/Hold → engine.process_bar()
|
|
// After: factored → engine.process_bar_factored()
|
|
```
|
|
|
|
File: `crates/ml/src/hyperopt/adapters/dqn.rs` (~3 lines in backtest loop)
|
|
|
|
## Expected Impact
|
|
|
|
- Degenerate trials now produce different objectives (10.0 vs 5.0 vs 2.0 vs 1.25) → PSO can navigate
|
|
- Factored eval generates more trades from partial position changes → richer performance metrics
|
|
- Models that learn Long100/Long50 switching get credit instead of being penalized
|
|
- Combined: PSO should find viable configs in 5-8 initial trials instead of relying on luck
|
|
|
|
## Files Changed
|
|
|
|
| File | Change | ~Lines |
|
|
|------|--------|--------|
|
|
| evaluation/engine.rs | process_bar_factored() | 50 |
|
|
| hyperopt/adapters/dqn.rs | extract_objective() penalty | 25 |
|
|
| hyperopt/adapters/dqn.rs | Backtest loop wiring | 3 |
|
|
| Tests | Unit tests | 80 |
|
|
| **Total** | | **~160** |
|