spec+plan(moe): correct gate input dim 42 → STATE_DIM=128

T1.6 implementer correctly identified that the gate input dim is
ml_core::state_layout::STATE_DIM=128, not the literal 42 the spec/plan
incorrectly stated. The 42-dim figure was the bar-feature subset; the
actual state vector is 128-dim (42 features + portfolio + MTF + OFI
padded to 128 for cuBLAS alignment).

Updated spec §3 architecture diagram, §4.1 gate subnetwork description
+ parameter count (3,272 → 8,776), and plan header architecture line.
Implementation in commit 28c707f6a is correct; this commit just makes
the spec match the implementation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-04-27 18:22:40 +02:00
parent 28c707f6ab
commit fc4addaabf
2 changed files with 5 additions and 5 deletions

View File

@@ -4,7 +4,7 @@
**Goal:** Replace vestigial `RegimeConditionalDQN` (3 hardcoded heads, only `trending_head` trains) with a Mixture-of-Experts policy network (K=8 experts, learned gate) integrated into the existing single-DQN production training path. Atomic deletion of all regime-conditional infrastructure with no fallback paths.
**Architecture:** Shared GRN trunk → 8 small expert MLPs (256→64→256 bottleneck) → learned gating network (state[42]→64→8 softmax) → mixed `h_s2` → existing 4 branching heads + C51 + IQN dual head. Soft full mixture (no top-k hardcoding); anti-collapse load-balancing aux loss `λ·K·Σ_k(mean_b g[b,k])²` with `λ=0.01` configurable.
**Architecture:** Shared GRN trunk → 8 small expert MLPs (256→64→256 bottleneck) → learned gating network (state[STATE_DIM=128]→64→8 softmax) → mixed `h_s2` → existing 4 branching heads + C51 + IQN dual head. Soft full mixture (no top-k hardcoding); anti-collapse load-balancing aux loss `λ·K·Σ_k(mean_b g[b,k])²` with `λ=0.01` configurable.
**Tech Stack:** Rust 1.85+, CUDA 12.4, cuBLAS, cudarc, safetensors, GPU PER replay, CUDA Graph capture, Argo Workflows on Scaleway L40S/H100 K8s.

View File

@@ -30,17 +30,17 @@ The legacy 3-head infrastructure, per-regime checkpoint format, regime classifie
## 3. Architecture overview
```
state[42] ─── shared GRN trunk ─── h_s1[256] ──┬── expert_MLP_0 ──┐
state[STATE_DIM=128] ─── shared GRN trunk ─── h_s1[256] ──┬── expert_MLP_0 ──┐
├── expert_MLP_1 ──┤
├── ... ├── (g · {expert_k}) ── h_s2[256] ── 4 branching heads ── C51 atoms ── existing IQN/CQL/MSE/Ensemble losses
└── expert_MLP_7 ──┘
gate: state[42] ── small_MLP(42→64→8) ── softmax ── g[8]
gate: state[STATE_DIM=128] ── small_MLP(42→64→8) ── softmax ── g[8]
```
- **Shared early trunk**: existing GRN block (unchanged) computes `h_s1`.
- **K=8 expert MLPs**: each is a small bottleneck projection `h_s1[256] → 64 → h_s2[256]`. Learns a regime-conditioned policy-relevant representation.
- **Gating network**: small MLP `state[42] → 64 → 8` followed by softmax produces `g[B, 8]` per state. Gate has access to the same ADX/CUSUM features the legacy threshold classifier used, plus the rest of the state.
- **Gating network**: small MLP `state[STATE_DIM=128] → 64 → 8` followed by softmax produces `g[B, 8]` per state. Gate consumes the full state vector (`ml_core::state_layout::STATE_DIM=128`, which includes 42 bar features + portfolio + MTF + OFI features padded to 128 for cuBLAS alignment). The 42-dim subset matters because that's where ADX (40) and CUSUM (41) live — the legacy threshold classifier's inputs — but the gate sees the full state including position/risk context.
- **Mixture**: `h_s2[b, :] = Σ_k g[b, k] * expert_k(h_s1[b, :])`. Soft full mixture (no top-k hardcoding); the gate's softmax distribution emerges peaky or flat from data, no architectural sparsity constraint.
- **Downstream**: existing 4 branching heads (direction × magnitude × order × urgency), C51 distributional Q, IQN dual head — all unchanged, consume `h_s2` exactly as before.
- **Load-balancing aux loss**: `λ · K · Σ_k (mean_b g[b, k])²` with default λ=0.01. **λ is a configurable hyperparameter** (lives in `DQNHyperparameters` alongside `cql_alpha`, `iqn_lambda`, etc.) — not a hardcoded kernel constant — so L40S validation can adjust it (e.g., raise to 0.05) without re-compilation if anti-collapse is too weak. Anti-collapse only — prevents init-noise-dominated single-expert lock-in (M1) without forcing uniform utilization (M2). User-confirmed empirical signal: collapses don't recover well in this codebase, so anti-collapse insurance is non-negotiable.
@@ -51,7 +51,7 @@ state[42] ─── shared GRN trunk ─── h_s1[256] ──┬── expert_
- Input: full 42-dim state vector.
- Architecture: `Linear(42 → 64) → LeakyReLU(α=0.01) → Linear(64 → 8) → softmax`.
- Parameters: 42·64 + 64 + 64·8 + 8 = 3,272 weights + biases.
- Parameters: 128·64 + 64 + 64·8 + 8 = 8,776 weights + biases (using `STATE_DIM=128` for the gate input — see §3 architecture note on the 42 vs 128 distinction).
- Output: `g [B, 8]`, rows sum to 1.
- Initialization: zero weights + zero bias on both linear layers, so initial `g(s) = 1/8` uniform for every state. No expert is favored by init noise; specialization emerges from data + the load-balancing aux.