tune(aux-bce): lower POS_WEIGHT_MAX 50→10 (gentler class-weight clamp)

A+B Smoke 1 (alpha-perception-79rbn) showed pos_weight=50 caused
overshooting: BCE=0.83 (worse than chance ln(2)=0.69) AND sub-chance
dir_acc (0.28-0.45) at all horizons. The 50× cap pushed the model
toward the rare positive class hard enough to blow up loss on 99.99%
of negative samples.

Lower MAX clamp to 10× lets per-horizon imbalance still scale the loss
but caps the runaway gradient on extreme-rare positive classes (like
K=100's 0.01% positive rate where pos_weight would otherwise be ~9999).

Synthetic test still passes (clamp doesn't matter on constant signal
where pos_fraction=1.0).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-05-22 12:19:31 +02:00
parent 0f5d5c7b4a
commit 24289f40a5
2 changed files with 9 additions and 2 deletions

View File

@@ -96,7 +96,12 @@ pub const DEFAULT_HUBER_DELTA: f32 = 1.0;
/// runaway logit gradients while still letting >5:1 class imbalance
/// scale the positive-class loss proportionally.
pub const POS_WEIGHT_MIN: f32 = 1.0;
pub const POS_WEIGHT_MAX: f32 = 50.0;
/// Lowered 50→10 after A+B Smoke 1 (alpha-perception-79rbn, 2026-05-22)
/// showed pw=50 caused class-weight overshooting: BCE=0.83 (worse than
/// chance ln(2)=0.69) AND sub-chance dir_acc at all horizons. The 50× cap
/// pushed the model toward the rare positive class hard enough to
/// blow up loss on 99.99% of negative samples.
pub const POS_WEIGHT_MAX: f32 = 10.0;
/// Host-side weight storage for [`AuxHeads`] (checkpoint round-trip).
#[derive(Clone, Debug)]

View File

@@ -97,7 +97,9 @@ const AUX_VEC_ADD_CUBIN: &[u8] = include_bytes!(
/// (would down-weight the rarer class) and pw > 50 produces unstable
/// gradient spikes.
const POS_WEIGHT_MIN: f32 = 1.0;
const POS_WEIGHT_MAX: f32 = 50.0;
/// Mirrors `aux_heads::POS_WEIGHT_MAX`; lowered 50→10 after A+B Smoke 1
/// (79rbn) showed pw=50 caused class-weight overshooting (BCE > ln(2)).
const POS_WEIGHT_MAX: f32 = 10.0;
/// CB5 (SDD-3): Wiener-α floor for the aux BCE / Huber EMAs
/// (`pearl_wiener_alpha_floor_for_nonstationary`). The aux trunk is
/// non-stationary: its loss trajectory drifts as the encoder