NoisyLinear was completely dead after cuBLAS migration — neither training
nor experience collection applied ANY exploration noise. With identical
Q-values across actions, Boltzmann selection was uniform, making the
dueling mean-subtraction cancel ALL advantage gradients. The advantage
heads could NEVER learn (Q-gap permanently 0.0000).
Fix: add_advantage_noise CUDA kernel injects per-action Gaussian noise
into Q-values AFTER compute_expected_q, BEFORE action selection.
- Philox PRNG + Box-Muller for GPU-native Gaussian sampling
- Per-sample, per-action noise (breaks within-batch symmetry)
- Only during experience collection (not training targets)
- noise_sigma=0.1 (configurable, TOML + hyperopt tunable)
The noise creates non-uniform Boltzmann selection → different actions
have different frequencies → advantage gradient survives the dueling
mean-subtraction → Q-gap can grow.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
21bd05c9a2