tune(aux-bce): lower POS_WEIGHT_MAX 50→10 (gentler class-weight clamp)
A+B Smoke 1 (alpha-perception-79rbn) showed pos_weight=50 caused overshooting: BCE=0.83 (worse than chance ln(2)=0.69) AND sub-chance dir_acc (0.28-0.45) at all horizons. The 50× cap pushed the model toward the rare positive class hard enough to blow up loss on 99.99% of negative samples. Lower MAX clamp to 10× lets per-horizon imbalance still scale the loss but caps the runaway gradient on extreme-rare positive classes (like K=100's 0.01% positive rate where pos_weight would otherwise be ~9999). Synthetic test still passes (clamp doesn't matter on constant signal where pos_fraction=1.0). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -96,7 +96,12 @@ pub const DEFAULT_HUBER_DELTA: f32 = 1.0;
|
||||
/// runaway logit gradients while still letting >5:1 class imbalance
|
||||
/// scale the positive-class loss proportionally.
|
||||
pub const POS_WEIGHT_MIN: f32 = 1.0;
|
||||
pub const POS_WEIGHT_MAX: f32 = 50.0;
|
||||
/// Lowered 50→10 after A+B Smoke 1 (alpha-perception-79rbn, 2026-05-22)
|
||||
/// showed pw=50 caused class-weight overshooting: BCE=0.83 (worse than
|
||||
/// chance ln(2)=0.69) AND sub-chance dir_acc at all horizons. The 50× cap
|
||||
/// pushed the model toward the rare positive class hard enough to
|
||||
/// blow up loss on 99.99% of negative samples.
|
||||
pub const POS_WEIGHT_MAX: f32 = 10.0;
|
||||
|
||||
/// Host-side weight storage for [`AuxHeads`] (checkpoint round-trip).
|
||||
#[derive(Clone, Debug)]
|
||||
|
||||
@@ -97,7 +97,9 @@ const AUX_VEC_ADD_CUBIN: &[u8] = include_bytes!(
|
||||
/// (would down-weight the rarer class) and pw > 50 produces unstable
|
||||
/// gradient spikes.
|
||||
const POS_WEIGHT_MIN: f32 = 1.0;
|
||||
const POS_WEIGHT_MAX: f32 = 50.0;
|
||||
/// Mirrors `aux_heads::POS_WEIGHT_MAX`; lowered 50→10 after A+B Smoke 1
|
||||
/// (79rbn) showed pw=50 caused class-weight overshooting (BCE > ln(2)).
|
||||
const POS_WEIGHT_MAX: f32 = 10.0;
|
||||
/// CB5 (SDD-3): Wiener-α floor for the aux BCE / Huber EMAs
|
||||
/// (`pearl_wiener_alpha_floor_for_nonstationary`). The aux trunk is
|
||||
/// non-stationary: its loss trajectory drifts as the encoder
|
||||
|
||||
Reference in New Issue
Block a user