spec: Component 6 early termination — PLATEAU_EXHAUSTED after 12 attempts
3 rewinds × 4 routes = 12 max attempts. Failed = improvement rate below threshold over 10 epochs. On exhaustion: save best model, exit cleanly, log "structural change needed". Saves ~3 hours H100 vs frozen training. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -293,10 +293,33 @@ Each rewind attempt uses a different strategy, cycling through:
|
||||
- Liquid tau EMAs reset on rewind (stale velocity from pre-plateau)
|
||||
- Compatible with CUDA graph (params_buf pointer unchanged, only data changes via HtoD)
|
||||
|
||||
### Early Termination (PLATEAU_EXHAUSTED)
|
||||
When all rewinds × all routes are exhausted without breaking through:
|
||||
|
||||
```
|
||||
rewind 1 (best checkpoint) → 4 routes → all failed (no improvement > threshold)
|
||||
rewind 2 (2nd best) → 4 routes → all failed
|
||||
rewind 3 (3rd best) → 4 routes → all failed
|
||||
→ STOP. Save best model. Exit with status PLATEAU_EXHAUSTED.
|
||||
```
|
||||
|
||||
**Exit criteria:** `rewind_count >= max_rewinds (3) && route_idx >= max_routes (4)` for the current rewind.
|
||||
|
||||
**"Failed" route definition:** improvement rate over M=10 epochs is below `min_improvement_rate` threshold (e.g. 0.1 val_Sharpe/epoch). A route that improves but slowly still counts as success.
|
||||
|
||||
**On termination:**
|
||||
1. Save the best checkpoint across ALL attempts (highest absolute val_Sharpe)
|
||||
2. Log: `"PLATEAU_EXHAUSTED: {rewinds}×{routes}={total} attempts. Best val_Sharpe={best} at epoch {ep}. Structural change needed."`
|
||||
3. Exit training loop cleanly (not a crash — the Argo workflow sees success with the best model saved)
|
||||
|
||||
**Benefit:** train-w6qfd ran 200 epochs frozen from epoch 22 = 178 wasted epochs (~3 hours H100). With backtracking: worst case ~142 epochs with a definitive answer. Best case: breaks through and continues improving.
|
||||
|
||||
### Implementation
|
||||
- `BacktrackingState` struct in `crates/ml/src/trainers/dqn/trainer/mod.rs`
|
||||
- `TrajectoryCheckpoint` struct for checkpoint data
|
||||
- Checkpoint save/restore methods in `training_loop.rs`
|
||||
- Plateau detection reads liquid tau velocity (already computed)
|
||||
- Early termination returns `Ok(TrainingResult::PlateauExhausted { best_epoch, best_sharpe })`
|
||||
- ~150 lines total, no new files
|
||||
|
||||
---
|
||||
|
||||
Reference in New Issue
Block a user