fix: NoisyNet test uses sigma=2.0 for action diversity on wide networks
With 256-wide hidden layers (H100 gpu profile), Xavier-initialized Q-value gaps are O(1/sqrt(256)) ≈ 0.06. The old sigma=0.5 produced NoisyNet noise O(sigma/sqrt(fan_in)) ≈ 0.03 which couldn't flip argmax → all-same-action. sigma=2.0 makes noise ≈ 0.12, exceeding Q-value gaps on any network width. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -709,7 +709,11 @@ mod gpu_smoke {
|
||||
episode_length: timesteps,
|
||||
epsilon: 0.0, // NoisyNets handle exploration
|
||||
gamma: 0.99,
|
||||
noisy_sigma_init: 0.5,
|
||||
// sigma must be large enough that NoisyNet noise dominates Q-value gaps
|
||||
// even on wide networks (256-dim). With random Xavier init on a 256-wide
|
||||
// network, Q-values are O(1/sqrt(256)) ≈ 0.06 apart. sigma=2.0 ensures
|
||||
// noise O(sigma/sqrt(fan_in)) ≈ 0.12 exceeds this gap.
|
||||
noisy_sigma_init: 2.0,
|
||||
num_atoms: 51,
|
||||
v_min: -10.0,
|
||||
v_max: 10.0,
|
||||
|
||||
Reference in New Issue
Block a user