fix(dqn): Bug #7 — cql_alpha regime gate handles null ISV without silent fallback (Task 2.5)

Track 3 triage §C5 identified as an error-hiding case per
feedback_no_hiding.md: when `isv_signals_pinned` is null (smoke-scale
runs without ISV warmup), the previous code silently fell back to
(health=0.5, regime_stability=0.5), yielding
`cql_alpha_eff = base × 0.5 × 0.5 = 0.25 × base` by degenerate math,
not by design. The hide made the smoke cql_alpha path near-zero for
reasons unrelated to the intended regime-gated behaviour.

Fix (option b per plan): emit a one-shot `tracing::warn!` and gate
the regime multiplier OFF when the pointer is null — cql_alpha falls
back to the scheduled base value (`base × 1.0 × 1.0`). Real ISV path
unchanged. The `std::sync::Once` bounds log spam to once per trainer
lifetime (this function runs every training step).

Option (a) — wiring ISV warmup at smoke scale — is a follow-up task;
it requires an upstream ISV pipeline change that is out of scope for
this bug-fix sweep.
This commit is contained in:
jgrusewski
2026-04-22 16:09:28 +02:00
parent 0cac3c84ce
commit 90e1e3dbb2

View File

@@ -5203,20 +5203,42 @@ impl GpuDqnTrainer {
let d_adv_ptr = self.cql_d_adv_logits.raw_ptr();
// B1/G2: Adaptive cql_alpha.
// - health from ISV[LEARNING_HEALTH_INDEX] (0.5 fallback if pinned ptr null)
// - health from ISV[LEARNING_HEALTH_INDEX]
// - regime_stability from ISV[11]
// - cql_alpha_eff = base × (1 regime_stability) × health
// Collapse → 0 (CQL off); volatile+healthy → full; stable+healthy → 0.
let (health, regime_stability) = if self.isv_signals_pinned.is_null() {
(0.5_f32, 0.5_f32)
//
// Task 2.5 Bug #7 — when `isv_signals_pinned` is null (smoke scale
// without ISV warmup), the previous silent fallback of (0.5, 0.5)
// gave `cql_alpha_eff = base × 0.5 × 0.5 = 0.25 × base` by degenerate
// math, not by design — violating feedback_no_hiding.md. Fix: emit a
// tracing::warn! once and gate the regime multiplier OFF (health=1.0,
// regime=0.0 → cql_alpha = base × 1.0 × 1.0 = base, the scheduled
// value). Full ISV warmup at smoke scale is a follow-up task.
let cql_alpha = if self.isv_signals_pinned.is_null() {
// Log once per-ctor lifetime so spam is bounded (this function
// runs every step). The std::sync::Once pattern here is local
// to this formula — constructors still populate isv_signals_pinned
// at full scale so the warn only fires under smoke-scoped runs
// that legitimately skip ISV warmup.
static WARN_ONCE: std::sync::Once = std::sync::Once::new();
WARN_ONCE.call_once(|| {
tracing::warn!(
"cql_alpha regime gate: isv_signals_pinned is null — formula \
disabled, using scheduled cql_alpha (base × 1.0 × 1.0). \
Smoke-scale runs without ISV warmup expected; full-scale \
should never hit this path."
);
});
self.config.cql_alpha
} else {
unsafe {
let (health, regime_stability) = unsafe {
let h = (*self.isv_signals_pinned.add(LEARNING_HEALTH_INDEX)).clamp(0.0, 1.0);
let s = (*self.isv_signals_pinned.add(11)).clamp(0.0, 1.0);
(h, s)
}
};
self.config.cql_alpha * (1.0 - regime_stability) * health
};
let cql_alpha = self.config.cql_alpha * (1.0 - regime_stability) * health;
// Cache for logging via FusedTrainingCtx / DQNTrainer.
self.last_cql_alpha_eff = cql_alpha;