gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(rl): confidence gate LCB relative to atom span, not absolute

Browse Source 
The old formula conf = clamp((μ - λσ) / σ_norm, 0, 1) produced
conf=0 for ANY distribution with σ > μ — including uniform Q at
training start (μ=0, σ=0.577 → LCB=-0.577 → conf=0). This caused
permanent 100% block rate regardless of threshold setting.

New formula: conf = clamp((μ - V_MIN - λσ) / (V_MAX - V_MIN), 0, 1)
Shifts by V_MIN so conf measures "how far above worst case":
  Uniform: conf = 0.21 (passes threshold 0.01)
  Peaked V_MAX: conf ≈ 1.0 (high confidence)
  Peaked V_MIN: conf ≈ 0.0 (blocked — correct)

Also sets conf gate floor to 0.0 so the adaptive controller can
fully disable the gate when trades dry up.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-05-25 10:29:49 +02:00
parent 43c4bddd8e
commit d456ad3962
1 changed files with 9 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
crates/ml-alpha/cuda/rl_confidence_gate.cu
View File

@@ -89,9 +89,15 @@ extern "C" __global__ void rl_confidence_gate(
}
const float sigma = sqrtf(var + 1e-8f);
// Lower confidence bound.
const float lcb = mu - lambda * sigma;
const float conf = fmaxf(0.0f, fminf(lcb / (sigma_norm + 1e-8f), 1.0f));
// Lower confidence bound, shifted relative to atom span so conf=0
// means "worst possible" and conf=1 means "certain at V_MAX".
// Without the V_MIN shift, any distribution with σ > μ gives conf=0
// (permanent block for near-uniform Q at training start).
const float v_min = atom_supports[0];
const float v_max = atom_supports[Q_N_ATOMS - 1];
const float span = v_max - v_min + 1e-8f;
const float lcb = (mu - v_min) - lambda * sigma;
const float conf = fmaxf(0.0f, fminf(lcb / span, 1.0f));
if (conf < threshold) {
actions[b] = ACTION_HOLD;