For seq_len K and horizon h with h ≫ K, the K position-supervised
labels in a single sequence are near-identical (sequential positions'
forward windows overlap by ~(h-1)/h). Per-position BCE therefore
treats ~K highly-correlated labels as independent samples, inflating
gradient pressure on long horizons by a factor of K.
Concretely at K=96:
h=30 → ~3 effective samples per seq (forward windows overlap ~97%)
h=100 → ~1 (~99%)
h=6000 → ~1 (~99.98%)
Per-position supervision was paying 96× the natural signal density on
h=6000, pulling the model toward fitting noise at long horizons.
Fix: the fused BCE kernel now accepts an optional
`loss_weights[N_HORIZONS]` (nullptr → uniform = no-op). Each (k, h)
loss + grad contribution is multiplied by w_h; the normaliser is the
sum of weighted valid entries instead of the raw valid count.
`auto_horizon_weights(K, horizons)` computes `w_h = min(1.0, K/h)` so
short horizons stay at full weight and long horizons collapse to
their independent-sample density. Exposed via CLI:
--auto-horizon-weights # K/h auto-derived
--horizon-weights "1,1,0.5,0.1,0.02" # explicit floats
Default behaviour is uniform (1.0) — apples-to-apples with the
in-flight qf5mj baseline. Synthetic overfit still 0.6268 → 0.1144 in
250 steps (82% drop). 77 tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
85ce295773