diff --git a/docs/superpowers/specs/2026-05-01-sp5-magnitude-differentiation-and-eval-collapse-design.md b/docs/superpowers/specs/2026-05-01-sp5-magnitude-differentiation-and-eval-collapse-design.md index c25c64b59..dd2b2e40a 100644 --- a/docs/superpowers/specs/2026-05-01-sp5-magnitude-differentiation-and-eval-collapse-design.md +++ b/docs/superpowers/specs/2026-05-01-sp5-magnitude-differentiation-and-eval-collapse-design.md @@ -73,9 +73,11 @@ Per-branch NoisyNet σ scales to per-branch Q-magnitude so noise/signal ratio st ### Pearl 4: `pearl_per_group_adam_hyperparams` -Per-param-group Adam β1, β2, ε ISV-driven (8 SP4 param groups: DqnTrunk/Value/Branches/Iqn/IqlHigh/IqlLow/Attn/Curiosity). Each group's Adam dynamics adapt to per-group gradient signal-to-noise. +**Per Q3=a (include as designed, accept theoretical risk):** per-param-group Adam β1, β2, ε ISV-driven (8 SP4 param groups: DqnTrunk/Value/Branches/Iqn/IqlHigh/IqlLow/Attn/Curiosity). Each group's Adam dynamics adapt to per-group gradient signal-to-noise. -**Producer signal:** per-group gradient direction-stability EMA (cosine similarity of consecutive gradients). +**Theoretical caveat:** Adam's original paper assumes constant β1/β2 across training. Per-group adaptive β1/β2 breaks this constant-β convergence guarantee. Mitigation: Pearls A+D smoothing prevents abrupt β-changes (Wiener α adapts gradually); EPS_CLAMP_FLOOR ensures β stays in [0.5, 0.999] structural envelope. Practical consequences monitored via Layer C smoke. If destabilization observed, fall back to Pearl 4 variant adapting ε only (β1, β2 stay constant). + +**Producer signal:** per-group gradient direction-stability EMA (cosine similarity of consecutive gradients). High stability → β2 closer to 0.999 (long memory); low stability → β2 closer to 0.9 (short memory). β1 derived analogously from per-group gradient-magnitude variance. **Slots (24):** ADAM_BETA1[8], ADAM_BETA2[8], ADAM_EPS[8] @@ -93,13 +95,13 @@ Kelly cap removes hardcoded `0.5` warmup_floor and `0.5 + 0.5 × h` health_safet **Slots (6):** KELLY_F_SMOOTH, CONVICTION_SMOOTH, TRADE_VAR_SMOOTH, KELLY_SAMPLE_COUNT, WIN_RATE_SMOOTH, LOSS_RATE_SMOOTH -### Pearl 7: `pearl_action_select_decomposition_audit_or_fix` +### Pearl 7: `pearl_action_select_decomposition_audit` (investigation-only in SP5) -**Phase 7.1 (investigation):** root-cause why intent_dist freezes at exact `Bin(2, 0.5)` ratio (0.25/0.50/0.25). Hypothesis: action_select_kernel may have hidden binary decomposition (e.g., 2 independent {0,1} bits sampled then summed for magnitude). Audit the action selection logic. +**Per Q2=a (investigate-first):** Pearl 7 is dispatched as an INVESTIGATION TASK after Pearls 1-3 land in Layer A and Layer B's first smoke run. Hypothesis: `intent_dist` freezing at exact `Bin(2, 0.5)` ratio (0.25/0.50/0.25) may be incidentally resolved once Pearls 1-3 fix Q-flatness + per-branch σ. If resolved, Pearl 7 closes with no code changes. -**Phase 7.2 (fix if found):** if hidden decomposition exists, replace with proper categorical sampling over 3 magnitude buckets, ISV-driven temperature. +**If pattern persists post-Layer-B smoke:** root-cause `action_select_kernel` for hidden binary decomposition (e.g., 2 independent {0,1} bits sampled then summed for magnitude). Fix logic in a SEPARATE follow-up commit/spec — NOT bundled into SP5 atomic Layer B. -**Slots:** 0-8 depending on Phase 7.1 outcome. +**Slots:** 0 in SP5 spec. Any fix lands as separate work post-SP5. ### Pearl 8: `pearl_trail_stop_signal_driven` @@ -128,7 +130,9 @@ Substantial: requires per-step trade-PnL aggregation kernel + multi-step composi ### Layer A — additive infrastructure -Multiple commits, no behavior change. Each commit adds: +**Commit granularity: 1 commit per pearl** (decision Q1=c). ~9 commits in Layer A — Pearls 1-3 separately (3 commits), Pearl 4, Pearl 5, Pearl 6, Pearl 8, Pearl 9, Pearl 1-ext, plus the host-EMA→GPU close-out as 1 final commit. Pearl 7 is investigation-only (deferred to a separate post-Layer-B task per Q2). + +Each commit adds: 1. ISV slot constants (per pearl) 2. Producer kernel(s) 3. Pearls A+D wire-up via existing `apply_pearls_ad_kernel` @@ -136,7 +140,7 @@ Multiple commits, no behavior change. Each commit adds: 5. Buffer growth + LAYOUT_FINGERPRINT_SEED bump (per commit accumulating) 6. GPU-gated unit tests -Estimated 15 commits in Layer A. Final state: ~120-128 ISV slots wired, all producers running, no consumers migrated yet (zero behavior change). +Final state: ~116-120 ISV slots wired (Pearl 7 investigation may add 0-8 in a follow-up), all producers running, no consumers migrated yet (zero behavior change). ### Layer B — atomic consumer migration @@ -211,11 +215,11 @@ After SP5, no known adaptive-parameter deferrals remain in the DQN training loop If anything surfaces post-SP5 (e.g., reward shaping issues exposed by improved magnitude differentiation), separate spec. -## Open question for user review +## Resolved decisions (post-brainstorming) -- 9 pearls + 1 layer-A close-out + 1 validation milestone in a single spec is large. Should the implementation plan use phased layers (A1, A2, A3 sub-layers per pearl, like SP4 had A1..A18 for individual tasks)? -- Pearl 7 Phase 7.1 may yield 0 slots if `Bin(2,0.5)` is downstream of Pearls 1-3 fixes (i.e., resolved without action_select changes). Should we prepare contingency: investigate first, then design Phase 7.2 IF needed? -- Pearl 4 (per-group Adam β1/β2/ε) introduces per-group hyperparameter dynamics. The Adam algorithm assumes constant β1/β2 over training; per-group adaptation breaks this assumption. Worth a separate pearl-design discussion? +- **Q1 → c**: Layer A commits = 1 per pearl (~9 commits total), no batching. Each pearl independently buildable + revertible. +- **Q2 → a**: Pearl 7 is investigation-only in SP5. Dispatched as task after Pearls 1-3 + Layer B smoke. If `Bin(2,0.5)` resolves incidentally, Pearl 7 closes with zero code. If persists, separate follow-up spec/commit. +- **Q3 → a**: Pearl 4 includes per-group β1/β2/ε with theoretical caveat documented. Mitigation: Pearls A+D smoothing + EPS_CLAMP_FLOOR β envelope. Layer C smoke monitors for destabilization; fall-back to ε-only if observed. ---