fix: gradient clip 10.0→1.0 — prevents grad accumulation with complex reward
grad_norm was growing 5x per epoch (523→2673→13K→67K→NaN). The old clip at 10.0 allowed gradients to accumulate. With clip=1.0, the Adam optimizer receives bounded updates. Trial 2 (TPE-guided params) trains cleanly for 4 epochs: train_loss: 4.5→3.3 (decreasing!) Q-value: 12-19 (stable) grad_norm: 162-567 (bounded) Trial 1 (random initial params) still NaN's — this is expected and handled by the hyperopt penalty (1M objective). The optimizer learns to avoid unstable parameter regions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1262,7 +1262,7 @@ impl DQNHyperparameters {
|
||||
use_huber_loss: true, // Default: Huber loss enabled (more robust)
|
||||
huber_delta: 100.0, // BUG #12 FIX: Scale delta 100x for gradient explosion fix (was 1.0)
|
||||
use_double_dqn: true, // Default: Double DQN enabled (prevents overestimation bias)
|
||||
gradient_clip_norm: Some(10.0), // REVERTED: Back to 10.0 default (production standard)
|
||||
gradient_clip_norm: Some(1.0), // Tight clip — prevents grad explosion with complex reward
|
||||
hold_penalty_weight: 0.01, // Default: 1% penalty weight
|
||||
movement_threshold: 0.02, // Default: 2% price movement threshold
|
||||
enable_preprocessing: true, // Default: preprocessing enabled (Wave 14 Agent 32)
|
||||
|
||||
Reference in New Issue
Block a user