Files
foxhunt/docs
jgrusewski 250cee124e fix(dqn): Winsorize adaptive_clip EMA input against single-sample outliers (N)
Smoke smoke-test-xw4c6 showed F1 ep2 grad_norm=8.4B polluting the
adaptive_clip EMA -> next adaptive_clip threshold became huge ->
subsequent extreme grads passed unclipped -> NaN propagation -> grad
collapse -> fold 1 fails despite K+warmup fixing the first-step problem.

Add Winsorized clamp on raw_grad_norm before the adaptive_clip EMA
update: clamped_sample = min(raw, K * previous_adaptive_clip) where
K = 100 (numerical-stability bound, not a tuned constant — explicit
"single sample can grow EMA at most 100x in one step" semantics).

Companion observability log emits GRAD_CLIP_OUTLIER warning per clamp
event so we can see when this fires in subsequent smokes.

Fast/slow grad_norm EMAs (driving warmup factor) intentionally NOT
winsorized — they're a stability signal that SHOULD respond to
outliers, providing extra warmup damping when the system is unstable.

Predicted impact on Plan C smoke F1 ep2: 8.4B grad -> clamped to
~3000 for EMA -> next adaptive_clip caps at ~1000 -> subsequent F1
batches get bounded gradients passed to Adam -> no NaN propagation.
F1+F2 should now train through, exposing whether the underlying
Q-target inflation requires further structural work or whether
clipping alone suffices.

Composes with K+warmup (4ef1d8ebb), A.2, A.1/A.3, F+H — fourth layer
of the adaptive gradient-stability scaffolding (now five total:
A.2 target-sync drift dampening, K+warmup post-reset transient
dampening, N persistent clip-EMA outlier defense).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 20:51:50 +02:00
..