jgrusewski a7ec00bcab feat(rl): ISV-driven reward KL β + gentle gradient β=0.003 ...

Reward augmentation now ISV-driven (slot 579, bootstrap 0.005).
Gradient KL β reduced to 0.003 (gentle nudge, not straitjacket).
Adaptive β controller disabled — overshoots consistently.

At b=16 (1 done/step), KL dominates sparse PPO signal → Hold=100%.
At b=1024 (60 dones/step), the done-gated PPO signal is 60× denser
and should balance with the gentle KL without collapse. The b=1024
run without KL (done-gated only) showed qpa=0.84, ent=2.04, wr=0.34
stable at 25k steps — the best metrics yet.

Removed dead code from normalize_advantages.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

2026-05-27 09:50:28 +02:00

..

cuda

feat(rl): ISV-driven reward KL β + gentle gradient β=0.003

2026-05-27 09:50:28 +02:00

examples

fix(rl): wire BCE + aux losses into GPU loader path + event-based sync

2026-05-26 22:18:30 +02:00

src

feat(rl): ISV-driven reward KL β + gentle gradient β=0.003

2026-05-27 09:50:28 +02:00

tests

perf(rl): fuse adamw_increment into adamw_step + tau into iqn_forward

2026-05-26 16:46:10 +02:00

build.rs

feat(rl): KL reference policy — proven RLHF pattern for Hold preservation

2026-05-27 01:58:01 +02:00

Cargo.toml

feat(rl): R6 — replace LobEnv with kernel-driven GPU-pure env step

2026-05-23 10:39:25 +02:00