fix: set c51_alpha_max=0.5 — prevent C51 gradient convergence on H100
H100 fold 2 showed grad_norm→0 when C51 fully replaced MSE (alpha=1.0). MSE provides a non-vanishing gradient floor. Default 0.5 = 50/50 blend. Hyperopt can search [0.3, 0.9]. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -82,7 +82,7 @@ q_clip_max = 50.0
|
||||
n_steps = 5
|
||||
tau = 0.005
|
||||
c51_warmup_epochs = 5
|
||||
c51_alpha_max = 1.0
|
||||
c51_alpha_max = 0.5
|
||||
her_ratio = 0.2
|
||||
cql_alpha = 1.0
|
||||
curiosity_weight = 0.1
|
||||
|
||||
@@ -1865,7 +1865,7 @@ impl DQNHyperparameters {
|
||||
|
||||
// C51 warmup: use MSE loss for first 5 epochs (stronger gradient than C51 cross-entropy)
|
||||
c51_warmup_epochs: 5,
|
||||
c51_alpha_max: 1.0, // Default: pure C51 after warmup. Hyperopt can search [0.3, 0.9].
|
||||
c51_alpha_max: 0.5, // MSE always contributes 50%. Hyperopt searches [0.3, 0.9].
|
||||
|
||||
// Decision Transformer pre-training: disabled by default
|
||||
dt_pretrain_epochs: 0, // 0 = disabled
|
||||
|
||||
Reference in New Issue
Block a user