Files
foxhunt/crates/ml/tests
jgrusewski 67dd414cb3 feat(sp5): Task A4 — Pearl 4 per-group Adam β1/β2/ε
Per-param-group Adam hyperparameters from per-group gradient
direction-stability EMA + L2 norm. 8 SP4 param groups × 3 hparams = 24
ISV slots [226..250).

Two-kernel chain:
  grad_cosine_sim_update — per-group cosine_sim of curr vs prev grads
                          + L2 norm; writes back grad_curr → grad_prev
                          for next step's comparison.
  pearl_4_adam_hparams_update — β1/β2/ε from cosine + l2_norm.

Structural envelopes (Invariant 1 anchors per spec line 88):
  β1 ∈ [0.85, 0.95]; β2 ∈ [0.99, 0.9995]; ε ∈ [1e-10, 1e-6]

ALPHA_META migration (Option A — atomic shared-contract migration per
feedback_no_partial_refactor): apply_pearls_ad_kernel.cu and the
launch_apply_pearls Rust wrapper now take alpha_meta as a parameter.
All 45 existing call sites (SP4 + SP5 producers) pass the prior default
1.0e-3 explicitly via crate::cuda_pipeline::sp4_wiener_ema::ALPHA_META.
Pearl 4's apply_pearls calls pass 5.0e-4 (half the default per spec
line 89) to limit β change rate.

Theoretical caveat: adaptive β breaks Adam's constant-β convergence
proof. Mitigations: structural envelopes + halved ALPHA_META + Pearls
A+D smoothing. Fall-back path defined: revert + ε-only adaptive
variant if Layer C destabilization observed.

New mapped-pinned buffer: grad_prev_buf_per_group [TOTAL_PARAMS] for
cosine-sim previous-step direction storage. Mapped-pinned i32 mirror
of grad_buf layout.

producer_step_scratch_buf grew 131 → 171 (40 new outputs: 8 cosine_sim
+ 8 l2_norm + 24 β1/β2/ε). wiener_state_buf already at 543 (A1 sized
for entire SP5 block).

StateResetRegistry: 4 new FoldReset entries: sp5_adam_beta1,
sp5_adam_beta2, sp5_adam_eps, sp5_grad_prev_buf. All sentinel 0 →
bootstrap to envelope midpoint via Pearls A+D + cosine_sim=0 fall-back
on first step of new fold.

Adam-launcher consumer migration deferred to Layer B.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-01 22:18:10 +02:00
..