# SP5: Comprehensive Per-Branch + Per-Group Adaptation Layer — Close All Known Hardcoded-Multiplier Deferrals

**Date:** 2026-05-01
**Status:** Design — awaiting user review before plan
**Branch:** to be created from main (HEAD `a60e7b092`)
**Triggering data:** `train-multi-seed-cv2mw` 50-epoch L40S baseline, terminated F0 ep10 after diagnostic-quality data captured. Plus all known deferrals from SP4 close-out, sweep audit, and project-history memory pearls.

## Goal

Close every known hardcoded-multiplier deferral in the DQN training loop in a single coherent project. Where SP4 made magnitude bounds adaptive, SP5 makes the entire learning machinery — atom resolution, loss-component budgets, exploration noise, optimizer hyperparameters, distributional quantile schedule, position-sizing safety floors, trail-stop thresholds, Thompson temperature — all per-branch / per-group ISV-driven via Pearls A+D.

After SP5: zero hardcoded multipliers in any adaptive code path; every adaptive value lives on ISV with sentinel-bootstrap and Wiener-optimal smoothing.

## Triggering findings

From the 50-epoch baseline (commit `a60e7b092`, captured F0 epochs 4-9):

| Symptom | Root cause |
|---|---|
| Magnitude head Q-values flat at O(1e-3), variances at O(1e-9 to 1e-8) | C51 atom span [-15, 15] with width 0.6 is 600× larger than mag Q-diff signal |
| `dist_q/h/f` freezes at exact 0.225/0.280/0.495 across 6+ epochs | NoisyNet σ=0.0226 dominates Q-diff=0.005 by 4-5× → action selection noise-dominated |
| `intent_dist` freezes at exact 0.250/0.500/0.250 (Bin(2,0.5)) | Suspicious binary decomposition in action_select kernel — needs investigation |
| Eval-mode argmax: trade_count=1 over 214,654 val bars | Q-variance below noise floor → argmax deterministically picks one action |
| `Sharpe=67` from one lucky trade | No kill-criterion catches `active_frac=1.0 with trade_count=1` degeneracy |

Plus all SP4 close-out and sweep deferrals (see [Scope](#scope) below).

## Scope

| # | Component | Status | New ISV slots |
|---|---|---|---|
| 1-3 | Per-branch atom span / loss budget / NoisyNet σ | core SP5 | 52 |
| 4 | Per-param-group Adam β1/β2/ε ISV-driven | from SP4 close-out memory | 24 (3 × 8 param groups) |
| 5 | Per-branch IQN τ schedule | from SP5 out-of-scope | 20 (5 × 4 branches) |
| 6 | Kelly cap signal-driven floors (cross-fold persistent — see [Pearl 6](#pearl-6-pearl_kelly_cap_signal_driven_floors)) | from project_magnitude_eval_collapse_kelly_capped.md | 6 |
| 7 | `dist_q/h/f Bin(2,0.5)` investigation only — fix in follow-up if needed | per Q2=a | **0** |
| 8 | Trail stop `base_trail_distance` signal-driven (per-direction) | new (Kelly-adjacent) | 4 |
| 9 | Thompson direction-branch temperature — **achieved via Pearl 1** (no separate slots) | per code audit | 0 |
| 1-ext | Per-branch C51 num_atoms | extension of Pearl 1 | 4 |
| Layer D close-out | 5 host-EMA host→GPU migrations (PnL aggregation pipeline → GPU) | from sweep audit, separate Layer | 5 sites |
| Validation | 3-seed × 50-epoch harness as exit gate | from SP5 out-of-scope | (no code) |

**Total: 110 new ISV slots** (52 + 24 + 20 + 6 + 0 + 4 + 0 + 4 = 110). LOC ~5500-7500 across Layers A+B+D. 9-11 producer kernels. 11 consumer migrations.

After SP5: zero deferrals on hardcoded multipliers in DQN training loop. Every adaptive value ISV-driven.

## Pearl design summaries

### Pearl 1: `pearl_per_branch_c51_atom_span`

Per-branch C51 atom positions (`v_center`, `v_half`) adapt to per-branch Q-magnitude. Magnitude head atoms shrink to ~[-0.025, 0.025] when Q-scale is 1e-3; direction head keeps wider span. Headroom factor itself ISV-driven via per-branch atom-clip rate (Pearl C-style 1% target rate).

**Slots (16):** ATOM_V_CENTER[4], ATOM_V_HALF[4], ATOM_HEADROOM[4], ATOM_CLIP_RATE[4]

### Pearl 1-ext: `pearl_per_branch_c51_num_atoms`

Per-branch atom COUNT adapts to per-branch Q-resolution requirement. Direction branch can use fewer atoms with wider span; magnitude branch may need more atoms at narrow span.

**Slots (4):** ATOM_NUM_ATOMS[4]
**Constraint:** total atom-tile shared memory bounded; sum across branches ≤ HW limit.

### Pearl 2: `pearl_per_branch_loss_budget`

Per-branch C51/IQN/CQL/Ens budgets driven by per-branch flatness signal `var(Q[b]) / σ[b]²`. IQN dominates when flat, C51 yields proportionally.

**Slots (20):** BUDGET_{C51,IQN,CQL,ENS}[4], FLATNESS[4]

### Pearl 3: `pearl_per_branch_noisy_sigma`

Per-branch NoisyNet σ scales to per-branch Q-magnitude so noise/signal ratio stays constant. Sigma fraction itself ISV-driven via per-branch action-entropy gap (target 70% of max entropy as Invariant 1 anchor — only "structural" constant remaining at this layer).

**Slots (12):** NOISY_SIGMA[4], SIGMA_FRACTION[4], BRANCH_ENTROPY[4]

### Pearl 4: `pearl_per_group_adam_hyperparams`

**Per Q3=a (include as designed, accept theoretical risk):** per-param-group Adam β1, β2, ε ISV-driven (8 SP4 param groups: DqnTrunk/Value/Branches/Iqn/IqlHigh/IqlLow/Attn/Curiosity). Each group's Adam dynamics adapt to per-group gradient signal-to-noise.

**Theoretical caveat (3 concrete risks):**

1. **Adam's convergence proof requires constant β1, β2 across training.** Per-group adaptive β breaks this guarantee. Combined with Pearl 2 (per-branch loss budget), Adam's update direction becomes a moving target — could oscillate or saturate. No theoretical guarantee that the system converges.

2. **β2 = 0.999 means ~1000-step memory of squared gradients.** Reducing β2 mid-training discards that memory; the variance estimate effectively resets, which can spike `m̂/√v̂` to large values and destabilize. Pearls A+D smoothing limits the rate of change but doesn't eliminate the discontinuity.

3. **Per-group ε pushed too small (e.g. 1e-12) → numerical instability** (division-by-near-zero). Pushed too large (e.g. 1e-3) → loss of precision on small gradients. Auto-adapting could push outside the safe range.

**Mitigations:**
- Pearls A+D Wiener α adapts gradually — no abrupt β changes step-to-step
- Structural β envelopes: β1 ∈ [0.85, 0.95], β2 ∈ [0.99, 0.9995], ε ∈ [1e-10, 1e-6] (Invariant 1 carve-outs)
- ALPHA_META in Pearls A+D set to **5e-4** (half the SP4 default 1e-3) to limit Pearl 4's β change rate
- Layer C smoke MUST monitor for destabilization (Q-explosion kill-criterion + per-group Adam-state checks)
- Fall-back: if destabilization observed in Layer C, revert Pearl 4 commit, replace with **ε-only adaptive** variant (β1=0.9, β2=0.999 stay Invariant 1 anchors; only ε adapts per-group). 8 slots instead of 24.

**Producer signal:** per-group gradient direction-stability EMA (cosine similarity of consecutive gradients). High stability → β2 closer to 0.999 (long memory); low stability → β2 closer to 0.99 (short memory, narrower envelope). β1 derived analogously from per-group gradient-magnitude variance. ε derived from per-group gradient L2 norm scale.

**Slots (24):** ADAM_BETA1[8], ADAM_BETA2[8], ADAM_EPS[8]

### Pearl 5: `pearl_per_branch_iqn_tau_schedule`

Per-branch IQN τ quantiles `{τ_05, τ_25, τ_50, τ_75, τ_95}` adapt to per-branch tail-thickness signal. Symmetric distributions can use uniform τ; skewed distributions concentrate τ in dense regions.

**Producer signal:** per-branch Q-distribution skew + kurtosis EMA.

**Slots (20):** IQN_TAU[4 branches × 5 quantiles]

### Pearl 6: `pearl_kelly_cap_signal_driven_floors`

Kelly cap removes hardcoded `0.5` warmup_floor and `0.5 + 0.5 × h` health_safety formula. Replaces with Pearls A+D smoothed kelly_f + conviction + trade-variance signals. Sentinel bootstrap handles cold-start naturally.

**Cross-fold persistence carve-out:** Kelly cap's signals serve a cross-fold steady-state baseline purpose (see [`grad_norm_slow_ema_pinned` precedent in fold_warmup_factor_kernel docstring](../../crates/ml/src/trainers/dqn/state_reset_registry.rs)). The 6 Pearl 6 slots are **explicitly NOT in SP4's per-fold-reset registry** — they persist trade-history across fold boundaries by design. Documented as Invariant 1 architectural carve-out: cross-fold-persistent signals are exempt from sentinel-contract reset because their semantic purpose IS cross-fold continuity.

Mechanism: Pearl 6 slots are allocated in a separate cross-fold-persistent ISV slot range (slots 280..286, NOT 174..226). State-reset registry has no entries for these slots; they retain values across fold boundaries throughout training. Pearls A+D bootstrap on first observation (training step 0); subsequent folds resume from prior state.

**Slots (6, cross-fold persistent):** KELLY_F_SMOOTH, CONVICTION_SMOOTH, TRADE_VAR_SMOOTH, KELLY_SAMPLE_COUNT, WIN_RATE_SMOOTH, LOSS_RATE_SMOOTH

### Pearl 7: `pearl_action_select_decomposition_audit` (investigation-only in SP5)

**Per Q2=a (investigate-first):** Pearl 7 is dispatched as an INVESTIGATION TASK at Layer C step 4 — AFTER the full atomic Layer B commit lands AND the 3-seed × 50-epoch L40S validation completes. Hypothesis: `intent_dist` freezing at exact `Bin(2, 0.5)` ratio (0.25/0.50/0.25) may be incidentally resolved once Pearls 1-3 fix Q-flatness + per-branch σ. If resolved, Pearl 7 closes with no code changes.

**If pattern persists post-Layer-B smoke:** root-cause `action_select_kernel` for hidden binary decomposition (e.g., 2 independent {0,1} bits sampled then summed for magnitude). Fix logic in a SEPARATE follow-up commit/spec — NOT bundled into SP5 atomic Layer B.

**Slots:** 0 in SP5 spec. Any fix lands as separate work post-SP5.

### Pearl 8: `pearl_trail_stop_signal_driven`

`check_trailing_stop` (`trade_physics.cuh:486`) takes `base_trail_distance` as kernel arg, currently hardcoded `0.005` (0.5%) at call site. `vol_scale` (CUSUM-based) and `trend_scale` (ADX-based) are already feature-driven adaptive multipliers. Floor constants (`hold_time<1.0f`, `peak_equity<1.0f`) are Invariant 1 carve-outs.

Pearl 8 migrates the `base_trail_distance` from hardcoded constant to per-direction ISV-driven signal. Producer signal: per-trade ATR EMA (volatility regime). Consumer: `check_trailing_stop` reads `ISV[TRAIL_DIST_PER_DIR + dir]` per call.

**Slots (4):** TRAIL_DIST_PER_DIR[4 directions: Short, Hold, Long, Flat]

(Hold/Flat directions read floor from EPS_CLAMP_FLOOR — they don't actually trigger trail stop but slot symmetry simplifies kernel signature.)

### Pearl 9 — REMOVED (achieved via Pearl 1)

Code audit of `experience_action_select`'s Thompson direction-branch sampling: uses `sample_C51_inverse_cdf(probs_d, atoms_d, u)` where `atoms_d` IS the per-branch atom support. There is no explicit "temperature" parameter — the spread of the C51 distribution implicitly sets the exploration scale.

**Therefore Pearl 9 collapses to zero scope.** Once Pearl 1 makes atom positions per-branch adaptive (per Pearl 1's `ATOM_V_CENTER` + `ATOM_V_HALF` + `ATOM_HEADROOM` slots), the Thompson distribution is automatically per-branch adaptive. No additional Pearl 9 slots needed.

Documented here for completeness; the SP5 brainstorm flagged this as a question. Code inspection resolved it.

## Architecture (mirror SP4 layer structure)

### Layer A — additive infrastructure

**Commit granularity: 1 commit per pearl** (decision Q1=c). 8 commits in Layer A in this order (resolves Pearl 2's circular dependency on Pearls 1+3 — Pearl 2 lands after both):

1. Pearl 1 (atom span — provides Q-stats producer that Pearl 2 depends on)
2. Pearl 3 (NoisyNet σ — provides σ that Pearl 2 depends on)
3. Pearl 2 (loss budget — depends on Pearls 1+3 outputs)
4. Pearl 4 (Adam β/β/ε — independent, can land anytime after Layer A starts)
5. Pearl 5 (IQN τ schedule — independent)
6. Pearl 6 (Kelly cap — cross-fold persistent slot range, NOT in fold-reset registry)
7. Pearl 8 (trail distance — independent)
8. Pearl 1-ext (per-branch num_atoms — extends Pearl 1, lands last)

Each commit adds:
1. ISV slot constants (per pearl)
2. Producer kernel(s)
3. Pearls A+D wire-up via existing `apply_pearls_ad_kernel`
4. StateResetRegistry entries (sentinel contract: ISV slot AND Wiener triple together) — EXCEPT Pearl 6 (cross-fold persistent, no fold-reset entry)
5. Buffer growth + LAYOUT_FINGERPRINT_SEED bump (per commit accumulating)
6. GPU-gated unit tests

Final state: 110 ISV slots wired, all producers running, no consumers migrated yet (zero behavior change).

### Layer B — atomic consumer migration

Single coordinated commit per `feedback_no_partial_refactor`. Migrates 11 consumers:
1. `atoms_update_kernel.cu` — per-branch atom positions + counts from ISV
2. 4 `apply_*_saxpy` kernels — per-branch budget sub-launches (4 each = 16 sub-launches per step)
3. NoisyLinear forward kernel — per-branch σ scaling
4. 5 Adam launchers — per-group β1/β2/ε from ISV (Pearl 4)
5. IQL/IQN τ-quantile selection — per-branch τ from ISV (Pearl 5)
6. `apply_kelly_cap` — Pearl 6 ISV reads, hardcoded floors removed
7. `check_trailing_stop` callers — Pearl 8 ISV reads for `base_trail_distance`

(Pearl 9 collapsed to zero migrations — Thompson reads atoms via Pearl 1 already-migrated path.)
(Pearl 7 stays out of Layer B — investigation-only, follow-up if needed.)
(Layer D / host-EMA close-out is its own atomic commit, not in this Layer B.)

Single large commit, ~1500-2200 LOC delta. Validates with smoke + L40S regression test post-commit.

### Layer C — validation + cleanup

1. Local 16+ GPU unit test pass on RTX 3050 Ti
2. L40S `multi_fold_convergence` smoke (5-epoch, single fold) — basic stability
3. L40S `train-multi-seed-baseline` (3-seed × 50-epoch × 3-fold) — full acceptance gate (per acceptance criteria below)
4. Pearl 7 investigation: dispatched AFTER Layer C step 3 completes. If `intent_dist` no longer freezes at Bin(2,0.5), Pearl 7 closes with no code. If pattern persists, separate follow-up spec.
5. Audit doc append (single comprehensive Layer A close-out + Layer B atomic + Layer C validation entry)
6. 8 new memory pearls + indices (1-3, 4, 5, 6, 8, 1-ext, plus 1 cross-fold persistence pearl documenting Kelly carve-out)
7. Retire any superseded global slots / config fields

### Layer D — host-EMA → GPU close-out (SEPARATE atomic commit, not bundled with Layer B)

Migrate 5 remaining host-aggregated EMA sites to GPU per `feedback_no_cpu_compute_strict`:
- `training_sharpe_ema` — requires per-step PnL aggregation on GPU
- `max_dd_ema` — same
- `gamma_blend` — host-aggregated eval result, port to GPU
- `LearningHealth` composition pipeline — multi-step host arithmetic, port to GPU
- `HealthEmaTrackers` — q_gap_ema + q_var_ema + grad_norm_ema host-side EMAs

**Substantial scope:** requires per-step trade-PnL aggregation kernel + multi-step composition kernel. Estimated 2000-3000 LOC across PnL aggregation pipeline (trade_physics outputs) + reward computation + LearningHealth composition. Lands as **single Layer D atomic commit** after Layer B validates. Reduces Layer B's blast radius by separating concerns.

**Why separate Layer D, not within Layer A close-out:** PnL aggregation pipeline migration touches trade_physics + experience_collector + reward computation, which are distinct architectural concerns from per-branch adaptation. Bundling them into Layer A's "close-out" mis-scoped the work. Layer D as its own atomic commit allows independent validation + revert.

## Acceptance criteria

### Layer A acceptance (per-commit verification)

- `cargo check -p ml --offline` clean
- Layer A unit tests pass on local RTX 3050 Ti
- Each commit independently buildable + reverible

### Layer B acceptance (single commit)

**Correctness gates (must pass):**
- Multi_fold_convergence smoke succeeds (5 epochs F0, all 3 folds reach epoch 5)
- Zero graph capture failures
- All SP4-era smoke kill-criteria pass (no NaN, no Q-explosion, no fold-boundary-grad-explosion)
- New SP5 kill-criterion: `Q-flatness-kill` if `var(Q[b]) < 0.5 × σ[b]²` by F0 ep10

**Adaptation indicators (informative, NOT blocking):**
- `var(Q[branch=Mag])` shows growth from Layer A baseline (not stuck at 1e-9)
- `v_half_mag` < `v_half_dir` (per-branch atom adaptation visible)
- Per-fold trade_count > 50 (not the trivial 1-trade case from pre-SP5)

**Layer B does NOT gate on absolute Sharpe** — performance characterization comes at Layer C exit gate.

### Layer C exit gate (full 3-seed × 50-epoch validation)

3-seed × 50-epoch × 3-fold L40S run completes:

**Correctness gates (must pass — no NaN, basic stability):**
- All 9 fold-runs (3 seeds × 3 folds) reach epoch 50 without NaN/explosion
- intent_dist drifts epoch-to-epoch (not frozen at exact ratio across 3+ epochs)
- Per-fold trade_count > 100 (no eval-collapse degeneracy)
- Per-branch HEALTH_DIAG shows coherent adaptation (variance grows, atoms shrink for narrow-Q branches)
- Memory pearls + audit doc landed

**Performance gates (reasonable bar — does NOT require strict positivity):**
- Mean Best Sharpe across seeds is **finite and not catastrophically negative** (defined as: > -50, equivalent to "not actively losing money at training-end-equity")
- Mean Best Sharpe across seeds is within **2× standard deviation** of pre-SP5 baseline (e.g., if pre-SP5 baseline mean was 25 ± 30, SP5 ≥ -35 is acceptable)
- val Sharpe variance across folds is finite (no degenerate single-trade Sharpe=67 cases — measured by `trade_count > 50`)

**Performance is informative, not blocking:** SP5's primary goal is architectural correctness (every adaptive value ISV-driven). Performance improvements are expected secondary effect but not gate-able.

## Architectural rules carryover from SP4

All `feedback_*` rules apply unchanged. Strict zero-tolerance:
- No CPU compute (`feedback_no_cpu_compute_strict`) — every producer is GPU; Pearls A+D applied via `apply_pearls_ad_kernel`
- No atomicAdd (`feedback_no_atomicadd`) — block tree-reduce only
- Mapped-pinned for HtoD/HtoH only
- No partial refactor — Layer B atomic commit
- No hardcoded multipliers (`feedback_isv_for_adaptive_bounds`) — only Invariant-1 numerical-stability anchors remain (EPS_CLAMP_FLOOR=1.0, ε_div=1e-8, base_iqn=0.11 anchor, entropy 70%-of-max anchor)
- Wire-up audit (`feedback_wire_everything_up`) — all 120-128 slots wired in Layer B; no orphan code
- No CPU forwards / test fallbacks — GPU oracle tests only
- No TODO/FIXME — declarative doc-comments only

## Out of scope

After SP5, no known adaptive-parameter deferrals remain in the DQN training loop.

If anything surfaces post-SP5 (e.g., reward shaping issues exposed by improved magnitude differentiation), separate spec.

## Resolved decisions (post-brainstorming + critical review)

- **Q1 → c**: Layer A commits = 1 per pearl (8 commits total), ordering Pearls 1 → 3 → 2 → 4 → 5 → 6 → 8 → 1-ext to resolve Pearl 2's circular dependency on Pearls 1+3.
- **Q2 → a**: Pearl 7 is investigation-only. Dispatched as Layer C step 4 (AFTER full atomic Layer B + 3-seed × 50-epoch validation). If `Bin(2,0.5)` resolves incidentally, Pearl 7 closes with zero code. If persists, separate follow-up spec/commit.
- **Q3 → a**: Pearl 4 includes per-group β1/β2/ε with 3 concrete risks documented + structural envelopes + ALPHA_META halved + ε-only fall-back path defined.
- **Critical review fixes applied:**
  - Pearl 6 cross-fold persistence carve-out (slots 280..286, NOT in fold-reset registry)
  - Layer A close-out (host-EMA→GPU) split into separate Layer D (atomic, post-Layer-B)
  - Pearl 9 collapsed to zero scope (achieved via Pearl 1)
  - Pearl 7 timing clarified (Layer C step 4, post-validation)
  - Layer B acceptance loosened: correctness gates required, performance gates loosened to "not catastrophically negative" + "within 2σ of pre-SP5 baseline"
  - Pearl 4 mitigations + fall-back path concrete
  - Pearl 8 enumeration concretized (4 slots: TRAIL_DIST_PER_DIR[4])
  - Total slot count corrected to 110 (was 120-128 inconsistent)

---

End of design. Implementation plan to be written next via `superpowers:writing-plans`. User review pending before plan dispatch.
