Files
foxhunt/docs/superpowers/specs
jgrusewski 24accea774 fix(sp4): migrate fast/slow grad-norm EMA update to GPU per feedback_no_cpu_compute_strict
Layer C close-out C1 redesigned. Original plan claimed
grad_norm_slow_ema_pinned was orphan post-Mech-6 migration; verification
surfaced a SECOND live consumer (fold_warmup_factor_update, commit
4ef1d8ebb) that legitimately reads the slow EMA as cross-fold
steady-state baseline.

The actual defect: the EMA UPDATE at update_adaptive_clip:22720-22737
was host-side `(1-α)*prev + α*obs` arithmetic — exactly the pattern
feedback_no_cpu_compute_strict (saved 2026-05-01) strictly forbids.

Migrated:
  - New `update_grad_norm_emas_kernel.cu` — single-thread fused fast+slow
    EMA update kernel. Reads `grad_norm_buf[0]`, updates two mapped-pinned
    EMA scalars via dev_ptr. `__threadfence_system()` ensures the
    `fold_warmup_factor_kernel` consumer sees freshly-written values.
  - `launch_update_grad_norm_emas` Rust launcher chained on the
    producer's stream — graph-capture-compatible, no host sync.
  - update_adaptive_clip's host-side `unsafe { ... }` block replaced
    with the GPU launcher call (warn-and-continue on launch failure
    mirroring the launch_h_s2_rms_ema / launch_fold_warmup_factor
    per-step ISV producer pattern at training_loop.rs:3450/3464).

Preserved:
  - grad_norm_slow_ema_pinned mapped-pinned buffer (cross-fold persistent;
    legitimate consumer is fold_warmup_factor_update).
  - Fixed-α design (FAST_ALPHA=0.1, SLOW_ALPHA=0.001) — Pearls A+D
    adaptive α would defeat the cross-fold-baseline semantic the warmup
    factor depends on.
  - Cold-start sentinel (`prev ≤ 0.0` ⇒ assign obs directly) — same
    formula as the deleted host code.
  - Host-side update_adaptive_clip early-return guard — kernel only
    launches when observed_grad_norm is finite and > 0.
  - grad_norm_emas_step_count host counter — scalar control-flow
    metadata for warmup-window gating, not compute.

Plan/spec docs updated to remove stale "orphan" claim. State-reset
registry doc-comment + field doc-comments updated to reflect GPU-only
update path. fold_warmup_factor_kernel docstring no longer describes
its grad-norm EMA inputs as host-side-fed.

Build clean, sp4 + state_reset_registry lib tests pass (11/11), 16/16
SP4 producer GPU tests pass on RTX 3050 Ti. No behavior change — pure
architectural fix.

Refs: SP4 Layer C C1 redesigned (was: retire). Original plan
docs/superpowers/plans/2026-04-30-sp4-signal-driven-magnitude-control.md
lines 2184-2221.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 14:07:10 +02:00
..