Files
foxhunt/crates
jgrusewski 4085831452 fix(dqn): asymmetric anti-LR — fast response to good, slow to bad
Symmetric 5-window mean (f1359f3dc) filtered noise but also filtered
exploration. RL Sharpe is plentiful-bad and rare-good in early training,
so the mean always sees the plentiful side — controller locked at 0.3×
LR and the model couldn't escape its initial bad minimum
(train-v82b2: sharpe stuck at −8 to −21 throughout Fold 0, never
peaked positive like prior runs had).

Root fix: the two decisions have different evidence requirements.
  * Boost LR: low cost if wrong (clamp caps runaway), high value if
    right (kicks out of overfit). Accept weak evidence — ANY of the
    last short_window epochs clearly positive fires the boost.
  * Dampen LR: high cost if wrong (stuck model), low value if right
    (stability we didn't need). Demand strong evidence — MEAN over
    long_window must be clearly negative.

Both windows derive from anti_lr_warmup (one knob):
  long_window  = anti_lr_warmup       (full window for sustained-bad)
  short_window = anti_lr_warmup / 2   (half window for recent-good)

No new hyperparameter. Asymmetry is structural (max vs mean over
differently-sized windows), not tuned.

Verified: multi-trial smoke 5/5 pass, median_q_gap=2.77, mean_sharpe_ema=12.24.
Compared to symmetric smoothing (2.13 / 11.03) and raw-signal (1.13 / 2.94),
the asymmetric version is the best on all three multi-trial metrics.

Tie-breaking: "good wins" when both signals fire in the same step —
matches the controller's original intent (exploration over dampening)
and our diagnosis (model needs LR headroom to escape bad starting
states).
2026-04-21 19:40:41 +02:00
..