Add KL bonus to reward: r += 0.02 × log(π_ref(a_taken)). This makes
Q learn the same KL-regularized objective as π, aligning their action
rankings (qpa positive when both value Hold). Reward written back to
rewards_d so PER replay also sees the augmented signal.
Decoupled from the adaptive gradient β: reward uses fixed 0.02,
gradient uses adaptive β from the hold_frac controller. Prevents
compounding that caused Hold=100% when both used the same β.
Removed dead code: normalize_advantages kernel (#if 0 block),
RL_HOLD_PRIOR_INDEX references.
Results at reward_kl_β=0.02: Hold 25-88% (oscillating around target),
entropy 1.19-1.45 (healthy), wr=0.35, qpa oscillates ±0.4.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
7894585ff4