fix(dqn): Fix HFT constraint handling to prune trials instead of crashing hyperopt
## Problem
HFT constraint violations (e.g., "Low LR + very high penalty causes training instability")
terminated the entire hyperopt campaign with an error instead of pruning the offending trial.
**Before**:
```
Error: Failed to convert parameters
Caused by:
Configuration error: Low LR + very high penalty causes training instability
```
Result: Entire hyperopt run crashed after Trial 2
## Solution
Moved HFT constraint validation from `from_continuous` (parameter conversion) to
`train_with_params` (objective evaluation), allowing graceful pruning of invalid trials.
**Changes**:
1. Removed validation from `from_continuous` (lines 139-141)
2. Added validation to `train_with_params` (lines 952-977)
3. Return heavily penalized metrics instead of error on constraint violation
**After**:
```
WARN ⚠️ Trial 1 PRUNED (HFT constraint): Low LR + very high penalty...
```
Result: Trial pruned with objective=+1.08e308, hyperopt continues successfully
## Validation
5-trial dry-run completed successfully:
- Trial 0: Trained (gradient explosion pruning - different constraint)
- Trial 1: ✅ PRUNED for HFT constraint (LR=4.38e-5 < 5e-5 AND hold_penalty=4.35 > 4.0)
- Trial 1 logged with WARN level (matches gradient explosion pattern)
- Hyperopt continued without crashing
## HFT Constraints (3 rules)
1. **Minimum penalty**: hold_penalty_weight ≥ 0.5 (force active trading)
2. **Training stability**: Low LR (<5e-5) + very high penalty (>4.0) rejected
3. **Buffer capacity**: Small buffer (<30K) + high penalty (>3.0) rejected
## Impact
- Hyperopt can now explore parameter space without crashing on constraint violations
- Invalid parameter combinations are pruned with penalty metrics
- Allows full 100-trial hyperopt campaigns to complete successfully
- Production-ready constraint enforcement for HFT trend-following strategies
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -136,9 +136,8 @@ impl ParameterSpace for DQNParams {
|
||||
hold_penalty_weight,
|
||||
};
|
||||
|
||||
// Validate HFT constraints
|
||||
params.validate_for_hft_trendfollowing()
|
||||
.map_err(|e| MLError::ConfigError { reason: e })?;
|
||||
// Note: HFT constraint validation moved to evaluate_objective (train_with_params)
|
||||
// to allow pruning instead of crashing the entire hyperopt run
|
||||
|
||||
Ok(params)
|
||||
}
|
||||
@@ -950,6 +949,33 @@ impl HyperparameterOptimizable for DQNTrainer {
|
||||
info!(" Gamma: {:.3}", params.gamma);
|
||||
info!(" Buffer size: {} (requested: {})", clamped_buffer_size, params.buffer_size);
|
||||
|
||||
// HFT constraint validation - return penalized objective on violation
|
||||
if let Err(constraint_msg) = params.validate_for_hft_trendfollowing() {
|
||||
tracing::warn!("⚠️ Trial {} PRUNED (HFT constraint): {}", current_trial, constraint_msg);
|
||||
|
||||
// Log constraint violation (ensure directory exists first)
|
||||
std::fs::create_dir_all(self.training_paths.logs_dir()).ok();
|
||||
write_training_log_dqn(
|
||||
&self.training_paths.logs_dir(),
|
||||
&format!("Trial PRUNED (HFT constraint): {}", constraint_msg)
|
||||
).ok();
|
||||
|
||||
// Return heavily penalized metrics to prune this trial
|
||||
return Ok(DQNMetrics {
|
||||
train_loss: 1000.0,
|
||||
val_loss: 1000.0,
|
||||
avg_q_value: 0.0,
|
||||
final_epsilon: 1.0,
|
||||
epochs_completed: 0,
|
||||
avg_episode_reward: -1000.0, // Heavy penalty (will give objective = +1000)
|
||||
buy_action_pct: 0.0,
|
||||
sell_action_pct: 0.0,
|
||||
hold_action_pct: 1.0, // Assume worst case (100% HOLD)
|
||||
gradient_norm: f64::MAX, // Maximum penalty
|
||||
q_value_std: f64::MAX, // Maximum penalty
|
||||
});
|
||||
}
|
||||
|
||||
// Log trial start (ensure directory exists first)
|
||||
std::fs::create_dir_all(self.training_paths.logs_dir()).ok();
|
||||
write_training_log_dqn(
|
||||
|
||||
Reference in New Issue
Block a user