feat(dqn): add moe_lambda hyperparameter (default 0.01)
Anti-collapse load-balancing aux loss weight for the upcoming Mixture-of- Experts redesign. Configurable via hyperopt; not a kernel constant. Spec: docs/superpowers/specs/2026-04-27-moe-regime-redesign-design.md §4.3. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -915,6 +915,11 @@ pub struct DQNHyperparameters {
|
||||
/// L_total = L_c51 + iqn_lambda * L_iqn
|
||||
/// Range [0.0, 2.0]: 0.0 = C51 only, 0.5 = balanced, 1.0 = equal weight
|
||||
pub iqn_lambda: f32,
|
||||
/// Mixture-of-Experts load-balancing aux loss weight.
|
||||
/// Default: 0.01 (anti-collapse only — prevents init-noise-dominated
|
||||
/// single-expert lock-in without forcing uniform utilization).
|
||||
/// Hyperopt search range: [0.001, 0.1].
|
||||
pub moe_lambda: Option<f32>,
|
||||
/// Spectral norm σ_max — constrains ||W||_σ ≤ σ_max.
|
||||
/// Range [1.0, 10.0]. Default 3.0 (permits Xavier scaling, prevents Q-explosion).
|
||||
pub spectral_norm_sigma_max: f32,
|
||||
@@ -1629,6 +1634,9 @@ impl DQNHyperparameters {
|
||||
// Plan 3 Task 8 B.3: GPU-only seeded warm-start.
|
||||
// 100k samples ≈ first 5 production-scale collection epochs.
|
||||
replay_seed_steps: 100_000,
|
||||
|
||||
// MoE load-balancing aux loss weight (Phase 1: additive field only).
|
||||
moe_lambda: Some(0.01),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -745,3 +745,5 @@ The 3 remaining Orphan rows are:
|
||||
|
||||
<!-- 2026-04-27: distributional_q_tests.rs — code-review fixups (u32::div_ceil for grid-dim; tightened OOB validation against B0_SIZE on Thompson + argmax read paths). No semantic change. -->
|
||||
|
||||
<!-- 2026-04-27: config.rs — Phase 1 Task 1.1: add `moe_lambda: Option<f32>` hyperparameter (default Some(0.01)) to DQNHyperparameters. Pure additive field; no kernel, no ISV slot, no hot-path consumer in this commit. Wiring into GpuDqnTrainConfig happens in a later Phase 1 task. -->
|
||||
|
||||
|
||||
Reference in New Issue
Block a user