The pre-spec balancer used `cap = 4 × median` to suppress outlier
branches. On L40S train-5wb4n (epoch 9 norms `[dir=130, mag=20000,
ord=18000, urg=16000]`), that made cap = 68000 — exceeding every
branch's norm. No cap ever fired; the kernel was a geometric no-op
for 11 consecutive epochs while direction starved at 1/150 the
magnitude gradient. The outlier-cap algorithm can only handle "one
branch above the pack", not "three co-elevated, one starved".
Replace with ISV-driven equalisation:
- 4 new ISV slots (31..34): per-branch grad-norm target
- 1 new ISV slot (35): scale clamp limit
- ISV_TOTAL_DIM 31 → 36
- New on-GPU producer kernel `grad_balance_isv_update` runs after
`branch_grad_norm_reduce`, before `branch_grad_rescale`. Computes
current-step median + max/min spread, applies adaptive-rate EMA
to the ISV slots. Single-threaded (1 warp), graph-capture safe,
no atomicAdd, no host syncs.
- Rescale kernel now reads ISV target + limit, computes per-branch
`scale[b] = clamp(target[b] / norm[b], 1/limit, limit)`. Every
branch equalises toward the shared median target over time;
starved branches boost, elevated branches suppress. Limit is the
observed max/min spread EMA, hard-bounded [2.0, 1e4] as a
numerical-safety cap (not tuning).
Adaptive-rate EMA: `alpha = clamp(err / (err + baseline), 0.01, 0.3)`.
Fast response when error is large relative to current value, slow
drift when stable. No fixed alpha constant.
Bootstrap at construction + fold reset: targets = 1.0, limit = 2.0.
Pre-first-EMA-update behaviour: scale clamped to [0.5, 2.0], effectively
the original identity on warm-up, until real observations flow in.
Complies with feedback_isv_for_adaptive_bounds.md (2026-04-23):
adaptive bounds always live in ISV, never hardcoded. No partial
refactor — all consumers of the balancer contract (launch, kernels,
rescale algorithm) migrate together.
Still pending from the task-#92 triad: (a) C51 backward kernel must
replicate the forward advantage-standardization for d==1 (chain-rule
violation produces wrong gradient; separate commit), (b) IQL
branch_scales floor for direction-branch Hold/Flat starvation
(separate commit). This commit is the safety net; the two root-cause
fixes come next.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
d61aefe2b8