fix(v8): increase micro_reward_scale 0.001→0.01 (10x stronger bootstrap signal)
At 0.001, the dense micro-reward was 1/1000th the magnitude of the sparse exit reward (±10). Too weak to meaningfully bootstrap Q-values. At 0.01, it's 1/100th — strong enough to provide directional feedback while still subordinate to the exit signal. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1656,7 +1656,7 @@ impl DQNHyperparameters {
|
||||
exposure_aux_weight: 0.5,
|
||||
exposure_aux_warmup_epochs: 5,
|
||||
// v8: Dense micro-reward, TD(λ), PopArt, curriculum, hindsight
|
||||
micro_reward_scale: 0.001,
|
||||
micro_reward_scale: 0.01,
|
||||
td_lambda: 0.9,
|
||||
max_trace_length: 7,
|
||||
popart_enabled: false, // disabled by default until wired
|
||||
|
||||
Reference in New Issue
Block a user