Found while auditing the remaining atomicAdds in the codebase:
ensemble_diversity_kernel computed pairwise KL divergence between K
ensemble heads and wrote the result to ensemble_diversity_loss_buf.
A readback function (`readback_diversity_loss`) was defined to consume
this scalar — but **nothing in the codebase ever called it**. The
kernel ran every training step, allocated a buffer per step, and the
result was discarded.
Removed (-195 LOC net):
- ensemble_diversity_kernel CUDA kernel (~100 LOC C++)
- kernel load + cubin pin in compile_ensemble_kernels
- ensemble_diversity_kernel field on FusedTrainingCtx
- ensemble_diversity_loss_buf field + alloc
- pending_diversity_loss_ptr + pending_diversity_normalizer fields
- readback_diversity_loss() function (the would-be consumer)
- launch site in run_ensemble_step (zero+launch+pending_ptr setup)
Kept (these ARE used):
- ensemble_aggregate_kernel (Q-value mean/variance for exploration bonus)
- ensemble_kl_gradient_kernel (computes diversity gradient → SAXPY into
grad_buf → adam — this is the actual training-path mechanism)
- apply_ensemble_diversity_backward (calls kl_gradient_kernel)
- ensemble_diversity_weight (scales the gradient)
Net effect on training: zero (the deleted code's output was unused).
Net effect on per-step cost: small but non-zero — saves one kernel
launch + memset per step + a CudaSlice<f32> alloc per training context.
On L40S/H100 this is microseconds; on RTX 3050 Ti slightly more.
Effect on determinism: zero. The atomicAdd in the deleted kernel was
in a code path whose output didn't feed training, so removing it
doesn't change the training trajectory. The remaining 9 atomicAdds
in the codebase break down as: 5 in monitoring_kernel (diagnostic
stats only), 3 in experience_kernels (atom_stats / penalty_out —
diagnostic-ish), 1 in dqn_utility (sensitivity_out feature attribution),
1 in trade_stats. None are in the gradient hot path.
Files touched:
crates/ml/src/cuda_pipeline/ensemble_kernels.cu (-100 lines)
crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs (-13 lines)
crates/ml/src/trainers/dqn/fused_training.rs (-100 lines)
Verified: SQLX_OFFLINE=true cargo check -p ml --lib --tests passes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
25ceb3b8d5