Files
foxhunt/docs
jgrusewski fe2b77388d fix(safety): adaptive Q-drift kill threshold from ISV |Q| reference (no tuned constant)
The previous absolute floor (`|q_mean| > 1.5`) in the Q-drift kill
criterion was a tuned constant in violation of
`feedback_adaptive_not_tuned.md` and
`feedback_isv_for_adaptive_bounds.md`. Replace with an ISV-driven
adaptive threshold:

    kill_floor = max(0.5, 3.0 × max(ISV[Q_ABS_REF_INDEX=16],
                                    ISV[Q_DIR_ABS_REF_INDEX=21]))

Both ISV slots are per-branch EMAs of `max(|Q_mean|)` already
maintained on-GPU by `q_stats_kernel.cu` and consumed by
`c51_loss_kernel`/`c51_grad_kernel`. The kill floor now anchors on
the same recently-observed healthy Q scale that the loss kernels
already use to normalise their collapse-fraction signals.

The 3.0× multiplier is architectural ("RL Q-divergence shows up at
2-4× healthy scale"); the 0.5 cold-start floor is an Invariant-1
numerical-stability bound active only while both ISV slots are still
≤ 1e-6 in fold-0 epoch-1, then dominated by the runtime formula. The
2× ratio gate is unchanged — already an architectural rate-of-change
bound.

ISV reads use the existing pinned/device-mapped path
(`fused.trainer().read_isv_signal_at`) — no HtoD/DtoH per
`feedback_no_htod_htoh_only_mapped_pinned.md`.

The bug-signature trajectory that motivated the original 1.5 floor
(F1 ep2 Q=+2.23 from F1 ep1 Q=+0.82) still trips: ISV[16] would have
been ~0.6 with α=0.05 EMA tracking, giving kill_floor ≈ 1.8, and
Q=+2.23 > 1.8 with ratio 2.7× trips both gates.

Touched: `training_loop.rs` (+70 LOC, two new use-list imports),
`docs/dqn-wire-up-audit.md` (Invariant 7 audit entry).

cargo check clean at 13 warnings (workspace baseline). No
fingerprint change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 20:22:41 +02:00
..