gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

feat(dqn): add moe_lambda hyperparameter (default 0.01)

Browse Source 
Anti-collapse load-balancing aux loss weight for the upcoming Mixture-of-
Experts redesign. Configurable via hyperopt; not a kernel constant.

Spec: docs/superpowers/specs/2026-04-27-moe-regime-redesign-design.md §4.3.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-04-27 18:02:04 +02:00
parent 8de65265ee
commit db4dbf6599
2 changed files with 10 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/dqn-wire-up-audit.md
View File

@@ -745,3 +745,5 @@ The 3 remaining Orphan rows are:
<!-- 2026-04-27: distributional_q_tests.rs — code-review fixups (u32::div_ceil for grid-dim; tightened OOB validation against B0_SIZE on Thompson + argmax read paths). No semantic change. -->
<!-- 2026-04-27: config.rs — Phase 1 Task 1.1: add `moe_lambda: Option<f32>` hyperparameter (default Some(0.01)) to DQNHyperparameters. Pure additive field; no kernel, no ISV slot, no hot-path consumer in this commit. Wiring into GpuDqnTrainConfig happens in a later Phase 1 task. -->