Root cause of q_pi_agree collapse: the symmetric Schulman controller
decayed λ from 0.5 to 0.001 in 34 steps (÷1.2/step), killing Q→π
coupling. Once decoupled, Q and π learned independent policies,
making q_pi_agree drop to 1e-22.
Three fixes:
1. MIN_LAMBDA 0.001 → 0.05: Q pull never drops below 5% strength
2. Asymmetric rates: ramp 1.2×/step (4 steps to double), decay
0.998×/step (347 steps to halve). Coupling establishes fast and
persists for thousands of steps.
3. Dead zone tolerance 1.5× → 3.0×: natural KL fluctuation stays
in-band instead of triggering constant ramp/decay cycles.
Seed λ raised to 0.1 so distillation is active from step 0.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
96abc364b5