End-to-end investigation (2026-04-27) confirmed RegimeConditionalDQN is
vestigial decoration — 3 heads constructed at training start but only
trending_head ever receives gradient updates. GpuDqnTrainer (the actual
production GPU trainer) has zero references to RegimeType/regime
routing; experience replay inserts go to trending_head.memory only;
ranging_head and volatile_head stay at random init for the entire
training run. Several support APIs (get_count_bonuses_branched, config,
get_state_dim) hardcode-delegate to trending_head, ignoring the regime
split entirely. Per `feedback_no_hiding.md` (wire up or delete) and the
user's preference to fix not delete: the design wires regime
conditioning properly via Mixture-of-Experts replacing the vestigial
3-head architecture.
Pearl introduced and saved as `pearl_learned_gate_subsumes_handcoded.md`:
when the network already sees the heuristic's inputs, a learned gate
strictly subsumes any hand-coded discretization. This is the
load-bearing rationale — ADX/CUSUM are already at state indices 40/41,
so threshold-based regime classification is a strict information
bottleneck the gate can recover and improve on.
Design summary:
- Architecture: shared GRN trunk -> K=8 small expert MLPs (256->64->256
bottleneck per expert, ~33k params each) -> learned gating network
(state[42]->64->8 softmax) -> mixed h_s2 -> existing branching heads
+ C51 + IQN dual head. Soft full mixture (no top-k hardcoding); gate
emerges peaky or flat from data. Anti-collapse load-balancing aux
loss with default lambda=0.01 (configurable hyperparameter, not a
kernel constant) prevents init-noise-dominated single-expert lock-in
without forcing uniform utilization. User-confirmed signal:
"collapses don't recover well in this codebase".
- 9 new ISV slots (118-126: per-expert utilization EMA + gate entropy
EMA), GPU-driven producer per
`pearl_cold_path_no_exception_to_gpu_drives.md`.
- 3 new small CUDA kernels (moe_mixture_forward/backward,
moe_load_balance_loss) + 1 ISV producer; everything else is cuBLAS-
reusable. CUDA Graph capture compatible.
- Atomic deletion (no fallback): regime_conditional.rs (~700 LOC),
RegimeType enum, classify_from_features, RegimeMetrics,
RegimeClassConfig, 4 DQNConfig regime threshold fields, per-regime
3-file checkpoint format. DQNAgentType becomes thin wrapper over
single DQN. Old checkpoints fail loudly with layout-fingerprint
mismatch.
- 5-layer testing strategy (unit kernels, smoke, gate-differentiation
validation, L40S production validation with explicit kill criteria,
architecture-hash backward-incompat).
Out of scope (explicit): top-k routing, per-expert action heads,
hierarchical MoE, regime-conditional CountBonus/NoisySigma broadcast,
expert warm-start from existing trending checkpoint, CVaR action
selection on mixed C51 distribution.
Precondition: the in-progress use_* flag cleanup + count_bonus
[f32; N] refactor lands as its own commit before MoE implementation
begins, per `feedback_no_partial_refactor.md`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
8629a9e7c2