docs(sp5): resolve open questions Q1=c, Q2=a, Q3=a
Per user direction:
Q1 (Layer A granularity): per-pearl commits (~9 commits, decision c)
Q2 (Pearl 7 timing): investigate-only in SP5; fix in follow-up
if Bin(2,0.5) persists post-Pearls-1-3 (decision a)
Q3 (Pearl 4 Adam β): include as designed; accept theoretical risk
with Pearls A+D + EPS_CLAMP_FLOOR mitigation;
Layer C smoke monitors for destabilization;
fall-back to ε-only if observed (decision a)
Spec section updated:
- Layer A description: per-pearl commit structure
- Pearl 7 framed as investigation-only with conditional follow-up
- Pearl 4 documents theoretical caveat + mitigation + fallback
User final review pending before invoking writing-plans skill.
This commit is contained in:
@@ -73,9 +73,11 @@ Per-branch NoisyNet σ scales to per-branch Q-magnitude so noise/signal ratio st
|
||||
|
||||
### Pearl 4: `pearl_per_group_adam_hyperparams`
|
||||
|
||||
Per-param-group Adam β1, β2, ε ISV-driven (8 SP4 param groups: DqnTrunk/Value/Branches/Iqn/IqlHigh/IqlLow/Attn/Curiosity). Each group's Adam dynamics adapt to per-group gradient signal-to-noise.
|
||||
**Per Q3=a (include as designed, accept theoretical risk):** per-param-group Adam β1, β2, ε ISV-driven (8 SP4 param groups: DqnTrunk/Value/Branches/Iqn/IqlHigh/IqlLow/Attn/Curiosity). Each group's Adam dynamics adapt to per-group gradient signal-to-noise.
|
||||
|
||||
**Producer signal:** per-group gradient direction-stability EMA (cosine similarity of consecutive gradients).
|
||||
**Theoretical caveat:** Adam's original paper assumes constant β1/β2 across training. Per-group adaptive β1/β2 breaks this constant-β convergence guarantee. Mitigation: Pearls A+D smoothing prevents abrupt β-changes (Wiener α adapts gradually); EPS_CLAMP_FLOOR ensures β stays in [0.5, 0.999] structural envelope. Practical consequences monitored via Layer C smoke. If destabilization observed, fall back to Pearl 4 variant adapting ε only (β1, β2 stay constant).
|
||||
|
||||
**Producer signal:** per-group gradient direction-stability EMA (cosine similarity of consecutive gradients). High stability → β2 closer to 0.999 (long memory); low stability → β2 closer to 0.9 (short memory). β1 derived analogously from per-group gradient-magnitude variance.
|
||||
|
||||
**Slots (24):** ADAM_BETA1[8], ADAM_BETA2[8], ADAM_EPS[8]
|
||||
|
||||
@@ -93,13 +95,13 @@ Kelly cap removes hardcoded `0.5` warmup_floor and `0.5 + 0.5 × h` health_safet
|
||||
|
||||
**Slots (6):** KELLY_F_SMOOTH, CONVICTION_SMOOTH, TRADE_VAR_SMOOTH, KELLY_SAMPLE_COUNT, WIN_RATE_SMOOTH, LOSS_RATE_SMOOTH
|
||||
|
||||
### Pearl 7: `pearl_action_select_decomposition_audit_or_fix`
|
||||
### Pearl 7: `pearl_action_select_decomposition_audit` (investigation-only in SP5)
|
||||
|
||||
**Phase 7.1 (investigation):** root-cause why intent_dist freezes at exact `Bin(2, 0.5)` ratio (0.25/0.50/0.25). Hypothesis: action_select_kernel may have hidden binary decomposition (e.g., 2 independent {0,1} bits sampled then summed for magnitude). Audit the action selection logic.
|
||||
**Per Q2=a (investigate-first):** Pearl 7 is dispatched as an INVESTIGATION TASK after Pearls 1-3 land in Layer A and Layer B's first smoke run. Hypothesis: `intent_dist` freezing at exact `Bin(2, 0.5)` ratio (0.25/0.50/0.25) may be incidentally resolved once Pearls 1-3 fix Q-flatness + per-branch σ. If resolved, Pearl 7 closes with no code changes.
|
||||
|
||||
**Phase 7.2 (fix if found):** if hidden decomposition exists, replace with proper categorical sampling over 3 magnitude buckets, ISV-driven temperature.
|
||||
**If pattern persists post-Layer-B smoke:** root-cause `action_select_kernel` for hidden binary decomposition (e.g., 2 independent {0,1} bits sampled then summed for magnitude). Fix logic in a SEPARATE follow-up commit/spec — NOT bundled into SP5 atomic Layer B.
|
||||
|
||||
**Slots:** 0-8 depending on Phase 7.1 outcome.
|
||||
**Slots:** 0 in SP5 spec. Any fix lands as separate work post-SP5.
|
||||
|
||||
### Pearl 8: `pearl_trail_stop_signal_driven`
|
||||
|
||||
@@ -128,7 +130,9 @@ Substantial: requires per-step trade-PnL aggregation kernel + multi-step composi
|
||||
|
||||
### Layer A — additive infrastructure
|
||||
|
||||
Multiple commits, no behavior change. Each commit adds:
|
||||
**Commit granularity: 1 commit per pearl** (decision Q1=c). ~9 commits in Layer A — Pearls 1-3 separately (3 commits), Pearl 4, Pearl 5, Pearl 6, Pearl 8, Pearl 9, Pearl 1-ext, plus the host-EMA→GPU close-out as 1 final commit. Pearl 7 is investigation-only (deferred to a separate post-Layer-B task per Q2).
|
||||
|
||||
Each commit adds:
|
||||
1. ISV slot constants (per pearl)
|
||||
2. Producer kernel(s)
|
||||
3. Pearls A+D wire-up via existing `apply_pearls_ad_kernel`
|
||||
@@ -136,7 +140,7 @@ Multiple commits, no behavior change. Each commit adds:
|
||||
5. Buffer growth + LAYOUT_FINGERPRINT_SEED bump (per commit accumulating)
|
||||
6. GPU-gated unit tests
|
||||
|
||||
Estimated 15 commits in Layer A. Final state: ~120-128 ISV slots wired, all producers running, no consumers migrated yet (zero behavior change).
|
||||
Final state: ~116-120 ISV slots wired (Pearl 7 investigation may add 0-8 in a follow-up), all producers running, no consumers migrated yet (zero behavior change).
|
||||
|
||||
### Layer B — atomic consumer migration
|
||||
|
||||
@@ -211,11 +215,11 @@ After SP5, no known adaptive-parameter deferrals remain in the DQN training loop
|
||||
|
||||
If anything surfaces post-SP5 (e.g., reward shaping issues exposed by improved magnitude differentiation), separate spec.
|
||||
|
||||
## Open question for user review
|
||||
## Resolved decisions (post-brainstorming)
|
||||
|
||||
- 9 pearls + 1 layer-A close-out + 1 validation milestone in a single spec is large. Should the implementation plan use phased layers (A1, A2, A3 sub-layers per pearl, like SP4 had A1..A18 for individual tasks)?
|
||||
- Pearl 7 Phase 7.1 may yield 0 slots if `Bin(2,0.5)` is downstream of Pearls 1-3 fixes (i.e., resolved without action_select changes). Should we prepare contingency: investigate first, then design Phase 7.2 IF needed?
|
||||
- Pearl 4 (per-group Adam β1/β2/ε) introduces per-group hyperparameter dynamics. The Adam algorithm assumes constant β1/β2 over training; per-group adaptation breaks this assumption. Worth a separate pearl-design discussion?
|
||||
- **Q1 → c**: Layer A commits = 1 per pearl (~9 commits total), no batching. Each pearl independently buildable + revertible.
|
||||
- **Q2 → a**: Pearl 7 is investigation-only in SP5. Dispatched as task after Pearls 1-3 + Layer B smoke. If `Bin(2,0.5)` resolves incidentally, Pearl 7 closes with zero code. If persists, separate follow-up spec/commit.
|
||||
- **Q3 → a**: Pearl 4 includes per-group β1/β2/ε with theoretical caveat documented. Mitigation: Pearls A+D smoothing + EPS_CLAMP_FLOOR β envelope. Layer C smoke monitors for destabilization; fall-back to ε-only if observed.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user