gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fix(ml): disable early stopping for short hyperopt trials

Browse Source 
Early stopping with 8-epoch trials returns penalty metrics (no backtest),
causing ALL trials to hit FALLBACK OBJECTIVE = 44.6 regardless of actual
model quality. The Sharpe was 1.36-1.61 but natural fluctuation triggered
"Sharpe worsening" at epoch 5, killing the backtest evaluation.

Fix: disable early stopping when epochs ≤ 12. With ~90s per trial, running
all 8 epochs is cheap. Long training runs (50 epochs) still use it.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-03-06 18:49:01 +01:00
parent e412f12e33
commit 1950ba009d
1 changed files with 6 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
crates/ml/src/hyperopt/adapters/dqn.rs
View File

@@ -2346,10 +2346,12 @@ impl HyperparameterOptimizable for DQNTrainer {
min_replay_size: params.batch_size * 2, // Need at least 2x batch size
epochs: self.epochs,
checkpoint_frequency: (self.epochs / 5).max(1), // Save 5 checkpoints per trial, min 1
// C4 FIX: Re-enable early stopping with adaptive plateau window.
// Previous issue: plateau_window=5 with 8 epochs triggered too often.
// Fix: adaptive window = max(epochs / 2, 3), giving enough learning time.
early_stopping_enabled: true,
// Disable early stopping for short hyperopt trials (≤12 epochs).
// Reason: early stopping returns penalty metrics + no backtest, causing
// ALL trials to hit FALLBACK OBJECTIVE. With 8 epochs (~90s per trial),
// it's cheaper to run all epochs and get real backtest metrics.
// Long training runs (train-best, 50 epochs) still use early stopping.
early_stopping_enabled: self.epochs > 12,
q_value_floor: -5.0,
min_loss_improvement_pct: 2.0,
plateau_window: (self.epochs / 2).max(5),