fix: NoisyNet test uses sigma=2.0 for action diversity on wide networks

With 256-wide hidden layers (H100 gpu profile), Xavier-initialized Q-value
gaps are O(1/sqrt(256)) ≈ 0.06. The old sigma=0.5 produced NoisyNet noise
O(sigma/sqrt(fan_in)) ≈ 0.03 which couldn't flip argmax → all-same-action.
sigma=2.0 makes noise ≈ 0.12, exceeding Q-value gaps on any network width.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-03-22 11:19:27 +01:00
parent adce841e6b
commit 3d46dd87e8

View File

@@ -709,7 +709,11 @@ mod gpu_smoke {
episode_length: timesteps,
epsilon: 0.0, // NoisyNets handle exploration
gamma: 0.99,
noisy_sigma_init: 0.5,
// sigma must be large enough that NoisyNet noise dominates Q-value gaps
// even on wide networks (256-dim). With random Xavier init on a 256-wide
// network, Q-values are O(1/sqrt(256)) ≈ 0.06 apart. sigma=2.0 ensures
// noise O(sigma/sqrt(fan_in)) ≈ 0.12 exceeds this gap.
noisy_sigma_init: 2.0,
num_atoms: 51,
v_min: -10.0,
v_max: 10.0,