fix: set c51_alpha_max=0.5 — prevent C51 gradient convergence on H100

H100 fold 2 showed grad_norm→0 when C51 fully replaced MSE (alpha=1.0).
MSE provides a non-vanishing gradient floor. Default 0.5 = 50/50 blend.
Hyperopt can search [0.3, 0.9].

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-04-09 20:02:48 +02:00
parent 8cbabcef5e
commit 796fbd01cf
2 changed files with 2 additions and 2 deletions

View File

@@ -82,7 +82,7 @@ q_clip_max = 50.0
n_steps = 5
tau = 0.005
c51_warmup_epochs = 5
c51_alpha_max = 1.0
c51_alpha_max = 0.5
her_ratio = 0.2
cql_alpha = 1.0
curiosity_weight = 0.1

View File

@@ -1865,7 +1865,7 @@ impl DQNHyperparameters {
// C51 warmup: use MSE loss for first 5 epochs (stronger gradient than C51 cross-entropy)
c51_warmup_epochs: 5,
c51_alpha_max: 1.0, // Default: pure C51 after warmup. Hyperopt can search [0.3, 0.9].
c51_alpha_max: 0.5, // MSE always contributes 50%. Hyperopt searches [0.3, 0.9].
// Decision Transformer pre-training: disabled by default
dt_pretrain_epochs: 0, // 0 = disabled