jgrusewski 58ff8db406 fix: aux GEMM gradient explosion — scale 0.5→0.01 + clip after backward ...

The exposure aux backward GEMM amplifies gradients through the
hidden activation matrix (d_logits × h_b0), producing grad_norm=5000+.
With max_grad_norm=10, this throttles ALL gradients by 500x (lr/500).

Fix: reduce aux_weight 0.5→0.01 (50x) + clip_grad_buf_inplace after
aux GEMM to cap combined gradient at max_grad_norm.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-04-08 10:03:54 +02:00

..

infra

feat(infra): GPU training launcher with local/cloud routing

2026-02-23 11:19:58 +01:00

plans

feat(dqn): enable Branching DQN with 45 factored actions (5×3×3)

2026-03-11 22:00:13 +01:00

superpowers

fix: aux GEMM gradient explosion — scale 0.5→0.01 + clip after backward

2026-04-08 10:03:54 +02:00