The default DQN agent (RegimeConditionalDQN with 3 regime heads) was
silently failing all gradient accumulation calls because only the
Standard DQN variant was supported. All training steps errored out and
loss was recorded as 0.0.
Changes:
- Add compute_gradients() to RegimeConditionalDQN: classifies batch by
regime, routes sub-batches to heads, merges gradient stores (no key
collisions since heads have independent parameters)
- Add apply_accumulated_gradients(): applies to heads with initialized
optimizers, skips uninitialized heads (no training data for that regime)
- Add optimizer_vars(): returns combined vars from all heads
- Update DQNAgentType dispatch to delegate to RegimeConditional
- Re-export GradientResult from dqn module
- Fix batch size validation tests for AutoBatchSizer clamping behavior
Verified: convergence test shows 4.8% relative difference between
accumulated (16x4) and direct (64x1) training paths (threshold: 30%).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
878510ba6d