Root cause: confidence gate decouples policy from actions. Policy
softmax stays high-entropy (~2.4) while the gate forces Hold, producing
action entropy ~0.7. SAC read policy entropy EMA (slot 420), saw
"above target", and kept LOWERING τ — the exact opposite of what was
needed.
New kernel `action_entropy_per_step` computes H(action_histogram) from
the POST-gate actions buffer and writes to ISV slot 583
(RL_ACTION_ENTROPY_EMA_INDEX). SAC co-tuning in rl_q_pi_distill_grad
now reads this slot. Launched OUTSIDE CUDA Graph capture (after
confidence gate + FRD gate) in both training and prefill paths.
Kernel design: 11 threads (N_ACTIONS), each thread counts its action
across all b_size elements. Thread 0 computes entropy from the
histogram and updates the EMA. No atomics.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
69d8038a80