Files
foxhunt/crates
jgrusewski a6ca9fba4f feat(mamba): wire real GPU Bernoulli dropout in supervised Mamba2
Supervised Mamba2's dropout_rate field was configured and stored but
never applied — comments said "simulated" but the arithmetic was
absent. An operator tuning dropout_rate upward got zero regularisation.

New dropout_kernel.cu with a forward-only Bernoulli dropout (Philox-
seeded, deterministic, in-place). New build.rs matching ml-dqn's
pattern. Applied in forward_with_gradients (training path) only; the
`forward()` path used by validate/predict/SPSA stays deterministic so
SPSA's ±ε finite-difference estimator is not destabilised.

NOT a DQN fix: DQN has its own regime_dropout kernel in
cuda_pipeline/experience_kernels.cu and its own native mamba2_step
in gpu_dqn_trainer.rs; DQN does not call into Mamba2SSM. This commit
affects only the supervised Mamba2 trainer and its hyperopt adapter.

Tests: determinism, eval-mode identity, ctr-increment divergence.
DQN smoke tests (magnitude_distribution, multi_fold_convergence) are
unaffected by this change — ran them for regression assurance only.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 10:07:06 +02:00
..