Replace static moe_lambda=0.01 with a GPU-driven controller that scales
λ inversely with observed gate-entropy: λ_eff = λ_floor + λ_max_extra ×
max(0, ent_target − ent_ema)/ent_target, where ent_ema is the existing
ISV[126] producer (Phase 2 task 2.4 wire-up).
Motivation: the L40S validation run train-multi-seed-fgg9x exposed
single-expert specialization in fold 1 (expert 4 → 81% util, 7/8
experts < 5%) under static λ=0.01. The load-balance push was too weak
to prevent winner-take-all once the gate had a strong preference.
Adaptive λ rises proportional to the entropy deficit, restoring
diversity pressure when the gate concentrates.
Per pearl_blend_formulas_must_have_permanent_floor: λ_floor remains a
permanent minimum (= old static 0.01) so the controller never weakens
below baseline. Per feedback_isv_for_adaptive_bounds: signal flows
through ISV[128] (new MOE_LAMBDA_EFF_INDEX); consumer kernel reads
the slot at runtime; no DtoH on hot path.
Wiring (feedback_no_partial_refactor):
• new kernel moe_lambda_eff_update (single-block cold-path cadence,
matches kelly_cap_update / cql_alpha_seed_update precedent)
• new ISV slot 128, ISV_TOTAL_DIM 127 → 129
• layout_fingerprint_seed updated (schema_hash bumps; PVC fxcache will
auto-regen on next deploy)
• moe_load_balance_loss kernel takes (isv_signals, isv_lambda_eff_idx)
instead of float lambda — matches mag_concat_qdir's ISV-read pattern
• config: pub moe_lambda field replaced with moe_lambda_floor (0.01),
moe_lambda_max_extra (0.09), moe_entropy_target_frac (0.7)
• state-reset registry entry — ISV[128] resets to floor at fold boundary
• HEALTH_DIAG aux_moe line gains λ_eff field for observability
• hyperopt adapter (DQNHyperparameters) migrated to new knobs
• constructor bootstraps ISV[128]=floor + ISV[118..127)=uniform 1/K +
ISV[126]=ln(K) so cold-start matches legacy byte-for-byte
Smoke validates: magnitude_distribution still passes/fails identically
to HEAD (eq=0.586 same as pre-change eq=0.592 — unrelated Kelly cap
behaviour per project_magnitude_eval_collapse_kelly_capped.md).
HEALTH_DIAG fold 3 confirms controller live: ent decays 1.381 → 0.746
across epochs 7-19, λ_eff rises 0.0146 → 0.0539 monotonically.
Unit test moe_lambda_eff_update_writes_correct_values exercises 5
regimes (above-target / at-target / half-collapse / full-collapse /
deeply-above-target) on the GPU kernel — all pass within 1e-5.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Single-thread cold-path-cadence kernel writes 8 per-expert utilization
EMAs + 1 gate-entropy EMA into ISV slots [118..127), α=0.05. Same shape
as h_s2_rms_ema_update / aux_heads_loss_ema_update.
GPU-resident, CPU read-only per pearl_cold_path_no_exception_to_gpu_drives.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two-step design (per-k contribution then K-element reduce) avoids
atomicAdd per feedback_no_atomicadd.md. Tests verify CPU reference
match and uniform-gate minimum (loss = λ).
Backward gradient lands in Phase 3 (wire-up).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two-stage backward: moe_mixture_backward computes de_k = g · dh_s2 in
one kernel; moe_dgate_reduce computes dg via shmem reduction over c.
FD test verifies analytic backward matches numerical for B=2, K=4, C=32
within 1e-3 tolerance (perturbation 1e-3).
All test data flows via mapped pinned buffers (no HtoD/HtoH).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
