fix(v8): increase micro_reward_scale 0.001→0.01 (10x stronger bootstrap signal)

At 0.001, the dense micro-reward was 1/1000th the magnitude of the
sparse exit reward (±10). Too weak to meaningfully bootstrap Q-values.
At 0.01, it's 1/100th — strong enough to provide directional feedback
while still subordinate to the exit signal.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-04-08 01:12:24 +02:00
parent b6527ca1db
commit 6fa3505e12

View File

@@ -1656,7 +1656,7 @@ impl DQNHyperparameters {
exposure_aux_weight: 0.5,
exposure_aux_warmup_epochs: 5,
// v8: Dense micro-reward, TD(λ), PopArt, curriculum, hindsight
micro_reward_scale: 0.001,
micro_reward_scale: 0.01,
td_lambda: 0.9,
max_trace_length: 7,
popart_enabled: false, // disabled by default until wired