Files
foxhunt/docs
jgrusewski 13773375b2 feat(moe): MoeGate + MoeExpert skeletons + DQN field plumbing
Module crates/ml-dqn/src/moe.rs introduces:
- MoeGate: 42→64→8 softmax gating network, zero-init for uniform start.
- MoeExpert: 256→64→256 bottleneck, Xavier init per-expert seed.
- Constants: MOE_NUM_EXPERTS=8, MOE_EXPERT_BOTTLENECK=64, MOE_GATE_HIDDEN=64.

DQN struct gains optional moe_gate / moe_experts fields, initialized to
None for now — actual wire-up into the forward graph in Phase 3.

Spec: docs/superpowers/specs/2026-04-27-moe-regime-redesign-design.md §4.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 18:14:22 +02:00
..