49 lines
1.8 KiB
Markdown
49 lines
1.8 KiB
Markdown
# DQN Generalization Gap — Action Plan
|
|
|
|
## Problem
|
|
In-sample Sharpe +0.57, OOS Sharpe -1.30 on H100. Model performs worse than random OOS.
|
|
100K+ IS trades vs 211 OOS trades — model learned IS-specific trading patterns.
|
|
|
|
## Root Causes (from codebase analysis)
|
|
1. **Deterministic episodes** — same start points every epoch, enables memorization
|
|
2. **CQL alpha=0.1** — effectively disabled (1.5% of gradient budget)
|
|
3. **Fixed simulation params** — spread=1bp, tx_cost=1.0, fill_prob=0.85 never vary
|
|
4. **Single-fold validation** — 80/20 time split, no purging/embargo
|
|
5. **No data augmentation** — pixel-perfect price sequences
|
|
|
|
## Execution Order
|
|
|
|
### Phase 1: Quick Wins (1-2 days)
|
|
|
|
**1a. Domain Randomization (Rust only, per-epoch)**
|
|
- `training_loop.rs` lines 900-990: randomize before kernel launch
|
|
- spread: U[0.5bp, 3.0bp], tx_cost: U[0.5, 2.5], fill probs: randomized
|
|
- Episode starts: add ±25% stride jitter (line 935)
|
|
- Variable episode length: U[250, 750] timesteps
|
|
|
|
**1b. CQL Alpha + Gradient Budget**
|
|
- `config.rs` line 1509: `cql_alpha: 0.1 → 1.0`
|
|
- `fused_training.rs` line 51: `CQL_GRAD_BUDGET: 0.15 → 0.25`
|
|
- C51 budget: 0.70 → 0.60
|
|
- Add `cql_alpha` warmup synchronized with c51_warmup
|
|
|
|
### Phase 2: Walk-Forward (3-5 days)
|
|
|
|
**2a. Enable GPU walk-forward by default**
|
|
- `config.rs`: `enable_gpu_walk_forward: true`
|
|
- Configure 6-8 folds with purge gap = n_steps bars
|
|
- Embargo gap = 100 bars (DSR EMA window)
|
|
- Model selection: median OOS Sharpe across folds
|
|
|
|
### Phase 3: Multi-Instrument (2-3 weeks)
|
|
|
|
**3a. ES + NQ dual training**
|
|
- Shared trunk, instrument-specific advantage heads
|
|
- Normalized rewards (vol-adjusted, instrument-agnostic)
|
|
- Data loading for multiple .dbn.zst files
|
|
|
|
## Expected Impact
|
|
- Phase 1: 50-70% generalization gap reduction
|
|
- Phase 2: 30-50% PBO reduction
|
|
- Phase 3: 8-12% OOS improvement
|