jgrusewski
09f5f9fb25
feat(bf16): f32 d_logits buffers — native atomicAdd, zero NaN from gradients
d_value_logits, d_adv_logits (+ MSE/CQL scratch): bf16 → f32
- Native atomicAdd(float*) replaces atomicAddBF16 CAS loop
- Eliminates bf16 accumulation overflow in gradient kernels
- Gradient value clamping ±100 removed (unnecessary with f32)
- NaN guards removed from loss kernels
Architecture:
- f32 d_logits for gradient accumulation (atomicAdd-safe)
- bf16 staging buffers (d_value_logits_bf16, d_adv_logits_bf16)
cast via f32_to_bf16_kernel before backward dW GemmEx
- dqn_saxpy_f32_kernel for gradient blending (MSE+C51 alpha)
- CQL backward uses bf16 staging after f32→bf16 cast
Remaining intermittent NaN (~1/2000 steps on long runs):
- Source: bf16 params_buf weight precision loss → forward pass
- Fix: f32 master weights (next commit)
895/895 unit + 359/359 ml-dqn tests pass.
9-11/11 smoke tests (intermittent NaN on 50-epoch runs).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-29 10:47:55 +02:00
..
2026-03-13 10:18:35 +01:00
2026-03-27 19:52:44 +01:00
2026-03-12 19:10:18 +01:00
2026-03-14 11:35:15 +01:00
2026-03-13 10:18:35 +01:00
2026-03-13 10:18:35 +01:00
2026-03-29 10:47:55 +02:00
2026-03-13 10:18:35 +01:00
2026-03-13 10:18:35 +01:00
2026-03-13 10:18:35 +01:00
2026-03-28 12:13:45 +01:00
2026-03-13 10:18:35 +01:00
2026-03-29 09:48:44 +02:00
2026-03-28 01:51:45 +01:00
2026-03-28 01:51:45 +01:00
2026-03-15 11:59:31 +01:00
2026-03-27 00:33:05 +01:00
2026-03-19 00:39:03 +01:00
2026-03-13 10:18:35 +01:00
2026-03-13 10:18:35 +01:00
2026-03-28 13:11:47 +01:00
2026-03-16 21:01:28 +01:00
2026-03-13 10:18:35 +01:00
2026-03-13 10:18:35 +01:00
2026-03-13 10:18:35 +01:00
2026-03-10 13:18:57 +01:00
2026-03-28 01:51:45 +01:00
2026-03-13 10:18:35 +01:00
2026-03-13 10:18:35 +01:00
2026-03-13 10:18:35 +01:00
2026-03-18 00:53:47 +01:00
2026-03-14 11:35:15 +01:00
2026-03-15 11:59:31 +01:00
2026-03-03 22:16:35 +01:00