Connects the per-horizon lambda computed by horizon_ema_and_lambda
(landed in 37c3a8f4d) to the actual trunk gradient flow. This is the
model-behavior change. mhzs7 (3a196382f) hit mean_auc=0.726 but with
the long-horizon h6000 stuck at 0.694 — exactly the horizon the
auto-horizon-weights formula `min(1, K/h)` over-weights down (h6000
weight = 0.0053). ISV detects "this horizon is hard, give it more
trunk-gradient pull" and lambda[h] scales the per-horizon `d_z`
contribution into the trunk in heads_bwd.
Kernel change (cuda/multi_horizon_heads.cu):
multi_horizon_heads_backward_batched(): new arg
`const float* __restrict__ lambda` (5 elements, between
grad_h_carry and n_batch in arg order).
- lambda is broadcast into __shared__ float s_lambda[5] once per
block so every thread reads the 5 floats without repeated
global loads.
- Sentinel: zero buffer (init state, EMA hasn't run yet) is read
as 1.0 so the kernel reduces to pre-ISV behavior. After step 1
every entry is clamped to [0.5, 2.0] by horizon_lambda.cu and
the > 0 check is always true.
- lambda[k] multiplies the `acc += s_lambda[k] * sd_z * w[k]`
term that produces grad_h (trunk gradient).
- grad_w and grad_b updates are UNCHANGED. The horizon heads keep
learning their own weight/bias normally; lambda only biases how
much each horizon's error signal leaks into the shared trunk.
Per `pearl_adam_normalizes_loss_weights.md`: scaling the
effective gradient bypasses Adam's m/sqrt(v) normalization that
would otherwise cancel a loss-weight lift.
Trainer change (trainer/perception.rs):
- heads_bwd_batched launch now passes `&self.lambda_d` between
grad_h_carry_d and n_batch_i. lambda_d is refreshed each step
by the horizon_ema_and_lambda kernel that ran just after BCE.
- Whole chain lives inside the captured CUDA Graph.
Validation:
- All 7 perception_overfit tests pass.
- Synthetic overfit converges marginally FASTER than pre-Phase-3
(final loss 0.0007 → 0.0005). Plausible explanation: in a
constant-signal smoke, the per-horizon BCE values are similar
enough that lambda mostly stays near 1.0, but the EMA
bootstrap on step 1 still produces non-uniform initial lambdas
that nudge the trunk toward whichever horizon converges
slowest. Either way, no regression.
- horizon_ema_and_lambda_track_after_training test still passes
with the lambda values now flowing through heads_bwd:
ema = [0.50, 0.87, 0.64, 0.49, 0.60]
lambda = [0.80, 1.41, 1.03, 0.79, 0.97]
→ h100 hardest (BCE 0.87) gets 1.41× trunk grad; h300 easiest
(BCE 0.49) gets 0.79× — exactly the intended ISV semantics.
- 26 lib + 23 integration ml-alpha tests pass.
Honors:
- feedback_isv_for_adaptive_bounds.md (no hardcoded constants;
lambda derives from runtime BCE signal).
- pearl_adam_normalizes_loss_weights.md (scaling trunk gradient,
not BCE coefficient, to bypass Adam normalization).
- pearl_audit_unboundedness_for_implicit_asymmetry.md (lambda is
bounded by horizon_lambda.cu's clamp [0.5, 2.0]).
- pearl_no_deferrals_for_complementary_fixes.md — Phase 3 wires
up the infrastructure from Phase 1+2 in the same logical
delivery rather than waiting a CV cycle.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
5d42ab0e98