Files
foxhunt/docs
jgrusewski 96b76d9298 feat(sp20): c51_loss_batched aux_conf_at_state reward gate
Adds the Phase 5 consumer kernel-side gate. New kernel arg
`const float* __restrict__ aux_conf_at_state` appended to
`c51_loss_batched`'s signature. Gate computation runs once per sample
at the kernel-entry reward-setup site (after the #27 ensemble-
disagreement adjustment), then the gated `reward` propagates through
every branch's `block_bellman_project_f` call without per-branch changes.

Formula:
    gate    = sigmoid((aux_conf - threshold) / temp)
    reward  = gate * reward
where:
    threshold = ISV[AUX_CONF_THRESHOLD_INDEX=518]
    temp      = max(ISV[AUX_GATE_TEMP_INDEX=519], 1e-3)

Mathematical interpretation: at low aux confidence (gate→0),
`r_used → 0`, so the Bellman target becomes `gamma * Q(s', a')`. The
Q value at the current state collapses toward `gamma * Q(s', a')` —
model gets no reward feedback on uncertain transitions. Effectively
"don't update Q on uncertain transitions" — the "uncertain-state
neutralizer" semantic from the Phase 3 Task 3.4 audit doc spec §4.4.

NULL-tolerant: `aux_conf_at_state == NULL` OR `isv_signals == NULL`
⇒ gate skipped (identity, no-op = pre-Phase-5 behaviour). Test
scaffolds without a wired aux head still work.

Out of scope: `iqn_dual_head_kernel.cu` — IQN is the auxiliary loss,
C51 is production. Gating IQN is more complexity for marginal gain.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 14:54:41 +02:00
..