fix(dqn): Bug #7 — cql_alpha regime gate handles null ISV without silent fallback (Task 2.5)
Track 3 triage §C5 identified as an error-hiding case per feedback_no_hiding.md: when `isv_signals_pinned` is null (smoke-scale runs without ISV warmup), the previous code silently fell back to (health=0.5, regime_stability=0.5), yielding `cql_alpha_eff = base × 0.5 × 0.5 = 0.25 × base` by degenerate math, not by design. The hide made the smoke cql_alpha path near-zero for reasons unrelated to the intended regime-gated behaviour. Fix (option b per plan): emit a one-shot `tracing::warn!` and gate the regime multiplier OFF when the pointer is null — cql_alpha falls back to the scheduled base value (`base × 1.0 × 1.0`). Real ISV path unchanged. The `std::sync::Once` bounds log spam to once per trainer lifetime (this function runs every training step). Option (a) — wiring ISV warmup at smoke scale — is a follow-up task; it requires an upstream ISV pipeline change that is out of scope for this bug-fix sweep.
This commit is contained in:
@@ -5203,20 +5203,42 @@ impl GpuDqnTrainer {
|
||||
let d_adv_ptr = self.cql_d_adv_logits.raw_ptr();
|
||||
|
||||
// B1/G2: Adaptive cql_alpha.
|
||||
// - health from ISV[LEARNING_HEALTH_INDEX] (0.5 fallback if pinned ptr null)
|
||||
// - health from ISV[LEARNING_HEALTH_INDEX]
|
||||
// - regime_stability from ISV[11]
|
||||
// - cql_alpha_eff = base × (1 − regime_stability) × health
|
||||
// Collapse → 0 (CQL off); volatile+healthy → full; stable+healthy → 0.
|
||||
let (health, regime_stability) = if self.isv_signals_pinned.is_null() {
|
||||
(0.5_f32, 0.5_f32)
|
||||
//
|
||||
// Task 2.5 Bug #7 — when `isv_signals_pinned` is null (smoke scale
|
||||
// without ISV warmup), the previous silent fallback of (0.5, 0.5)
|
||||
// gave `cql_alpha_eff = base × 0.5 × 0.5 = 0.25 × base` by degenerate
|
||||
// math, not by design — violating feedback_no_hiding.md. Fix: emit a
|
||||
// tracing::warn! once and gate the regime multiplier OFF (health=1.0,
|
||||
// regime=0.0 → cql_alpha = base × 1.0 × 1.0 = base, the scheduled
|
||||
// value). Full ISV warmup at smoke scale is a follow-up task.
|
||||
let cql_alpha = if self.isv_signals_pinned.is_null() {
|
||||
// Log once per-ctor lifetime so spam is bounded (this function
|
||||
// runs every step). The std::sync::Once pattern here is local
|
||||
// to this formula — constructors still populate isv_signals_pinned
|
||||
// at full scale so the warn only fires under smoke-scoped runs
|
||||
// that legitimately skip ISV warmup.
|
||||
static WARN_ONCE: std::sync::Once = std::sync::Once::new();
|
||||
WARN_ONCE.call_once(|| {
|
||||
tracing::warn!(
|
||||
"cql_alpha regime gate: isv_signals_pinned is null — formula \
|
||||
disabled, using scheduled cql_alpha (base × 1.0 × 1.0). \
|
||||
Smoke-scale runs without ISV warmup expected; full-scale \
|
||||
should never hit this path."
|
||||
);
|
||||
});
|
||||
self.config.cql_alpha
|
||||
} else {
|
||||
unsafe {
|
||||
let (health, regime_stability) = unsafe {
|
||||
let h = (*self.isv_signals_pinned.add(LEARNING_HEALTH_INDEX)).clamp(0.0, 1.0);
|
||||
let s = (*self.isv_signals_pinned.add(11)).clamp(0.0, 1.0);
|
||||
(h, s)
|
||||
}
|
||||
};
|
||||
self.config.cql_alpha * (1.0 - regime_stability) * health
|
||||
};
|
||||
let cql_alpha = self.config.cql_alpha * (1.0 - regime_stability) * health;
|
||||
|
||||
// Cache for logging via FusedTrainingCtx / DQNTrainer.
|
||||
self.last_cql_alpha_eff = cql_alpha;
|
||||
|
||||
Reference in New Issue
Block a user