Files
foxhunt/crates/ml-core
jgrusewski 30f46c04c1 fix(critical): mean-reduce gradients + fix ExposureLevel 4-branch mismatch
Three root causes found and fixed:

1. SUM-reduced gradients without 1/N: all CUDA loss gradient kernels
   (C51, MSE, IQN backward, CQL) accumulated per-sample gradients as
   raw SUM. At batch=16384 (H100) the raw norm was 282x larger than
   batch=58, causing gradient clipping to destroy signal-to-noise ratio
   and collapse training at epoch 2-3. Now all kernels multiply by
   1/batch_size, making gradient scale batch-invariant.

2. ExposureLevel::target_exposure() used a flat 9-level scale that did
   not match the 4-branch dir*mag encoding. The Rust backtest evaluator
   computed wrong position sizes (e.g. 4x oversize for Short+Small).
   Now uses dir x mag formula. Also fixed is_buy/is_sell/is_hold and
   from_trading_action for 4-branch semantics.

3. Rust epsilon-greedy only explored 5/9 exposure combos (0..5 instead
   of dir*3+mag), ignored the magnitude branch on greedy, and used wrong
   indices for order/urgency (get(1)/get(2) instead of get(2)/get(3)).

LR recalibrated: old gradient_clip_norm was accidentally a batch-size-
dependent LR reducer (~60x at batch=58, ~16000x at batch=16384). With
mean-reduced gradients the clip rarely fires, so LR is now the sole
training speed control. Smoketest 1e-4 -> 2e-6, production 1e-4 -> 1e-5,
hyperopt range [1e-5,3e-4] -> [1e-7,1e-4].

Diagnostics: FOXHUNT_GRAD_DIAG=1 enables per-stage gradient norm logging.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 14:18:36 +02:00
..