Files
foxhunt/crates
jgrusewski 020e9ba460 fix: rewrite curiosity training to per-block reduce — eliminates non-deterministic atomicAdd
Replace warp-reduced atomicAdd gradient accumulation in the curiosity
forward model with a two-kernel deterministic pipeline:

1. curiosity_fwd_bwd_per_block: each block reduces its threads' gradient
   contributions via shared memory tree reduction, writes one partial
   gradient vector [CUR_TOTAL_PARAMS] per block.

2. curiosity_grad_reduce: one thread per parameter sums block partials
   in fixed order (block 0, 1, 2, ...). Fully deterministic.

Deleted kernels: curiosity_forward_backward (atomicAdd path),
curiosity_fused_zero_fwd_bwd_adam (grid-wide atomic barrier),
curiosity_adam_step_fused (dead code), warp_sum_cur helper.

Rust side: removed block_counter, adam_fused_func,
fused_zero_fwd_bwd_adam_func. Added partial_grads buffer
[max_blocks * CUR_TOTAL_PARAMS] (~2.8 MB for 64 blocks).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-18 09:59:11 +02:00
..