Files
foxhunt/crates
jgrusewski ca7dfa9f08 fix(critical): c51_loss_reduce missing threadIdx guard — 32-way write race
SECOND race condition from block_dim (1,1,1)→(32,1,1) change:
c51_loss_reduce had 32 threads all computing sum and writing to
total_loss[0] simultaneously. On sm_90 (H100) → corrupted loss →
NaN in C51 gradient → graph_mega replay hangs.

This kernel runs INSIDE graph_mega (submit_forward_ops_main),
called twice (MSE reduce + C51 reduce). Both corrupted.

Also fixed stochastic_depth_rng in previous commit (same issue,
runs in pre-replay).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-17 13:19:39 +02:00
..