mjzfk + pdgxn diags showed `advantage_var_ratio` and `td_kurtosis`
identically 0 for 100% of every 50k-step smoke. Root cause: the
per-batch `rl_var_over_abs_mean_b` and `rl_kurtosis_b` kernels are
mathematically undefined at b_size=1 (variance of a single sample is
zero; kurtosis of a single sample is 0/0). The kernels correctly
returned 0 in that case but the downstream `rl_rollout_steps` and
`rl_per_alpha` controllers then never saw signal and pegged at MIN
(2048 / 0.4) for the entire run.
## Fix: time-axis Welford-EMA streaming
Replace per-batch reduction with per-step EMA-streaming moments
maintained in ISV slots:
rl_var_over_abs_mean_streaming.cu — maintains streaming mean + M2,
emits var/|mean| each step. Welford-EMA on the batch-mean of
advantages_d (one value at b_size=1, or a single batch reduction
at b_size>1) folded into the time-axis estimator.
rl_kurtosis_streaming.cu — maintains streaming mean + M2 + M4,
emits M4/M2² (Pearson kurtosis) each step. Same Welford-EMA shape
applied to td_per_sample_d batch mean.
Both kernels use STREAM_ALPHA = 0.05 (matches LR_LOSS_EMA_ALPHA —
half-life ≈ 14 steps) so the time estimator smooths over noisy
per-step batch-mean observations. The kernel writes the smoothed
estimate DIRECTLY to the controller-input ISV slot
(RL_ADVANTAGE_VAR_RATIO_EMA_INDEX = 421,
RL_TD_KURTOSIS_EMA_INDEX = 422); the prior downstream
ema_update_per_step calls for these two signals are REMOVED — the
streaming kernel IS the EMA.
## ISV slot allocation
5 new state slots holding the streaming-mean / M2 / M4 per-stream
state. RL_SLOTS_END: 442 → 447.
RL_ADV_VAR_STREAM_MEAN_INDEX = 442 (streaming mean of advantages)
RL_ADV_VAR_STREAM_M2_INDEX = 443 (streaming M2 of advantages)
RL_TD_KURT_STREAM_MEAN_INDEX = 444 (streaming mean of TD-CE)
RL_TD_KURT_STREAM_M2_INDEX = 445
RL_TD_KURT_STREAM_M4_INDEX = 446
Per `pearl_first_observation_bootstrap`: sentinel-zero state
triggers replace-direct first-observation bootstrap (the first
step seeds μ = batch_mean, M2 = 0, M4 = 0 — subsequent steps blend).
Per `pearl_blend_formulas_must_have_permanent_floor`: var/|mean|
denominator floored at 1e-6, M2² denominator floored at 1e-12 —
prevents div-by-zero blow-up when streaming mean / variance is
genuinely zero (cold-start or quiet regime).
## Files
* crates/ml-alpha/cuda/rl_var_over_abs_mean_streaming.cu — new
* crates/ml-alpha/cuda/rl_kurtosis_streaming.cu — new
* crates/ml-alpha/cuda/rl_var_over_abs_mean_b.cu — deleted
* crates/ml-alpha/cuda/rl_kurtosis_b.cu — deleted
* crates/ml-alpha/src/rl/isv_slots.rs — +5 slots
* crates/ml-alpha/src/trainer/integrated.rs — rewired
launchers,
dropped
redundant
ema_update
calls
* crates/ml-alpha/build.rs — swapped
cubin
entries
## Verified gates (local sm_86)
G1 isv_bootstrap ✅
G3 controllers ✅
G4 target_update ✅
integrated_smoke ✅
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
39f90f3723