gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fix: address all 3 precommit LOW findings

Browse Source 
1. D6 ensemble oracle no longer dormant — replaces hardcoded ens_disagreement=0.1
   with range of per-branch Q-gap EMAs (max - min over 4 branches). This gives a
   real signal that collapses to 0 when branches agree on uniform Q and expands
   to non-trivial values when branches preserve diverse action differentiation.
   D6's smoothstep(0.01, 0.1) window now actually moves.

2. HEALTH_DIAG defaults aligned with constructor inits — cql_budget default
   0.10→0.00, c51_budget 0.45→0.55. Only visible when fused_ctx is None
   (pre-first-step); eliminates the cosmetic log jump on the first real epoch.

3. reward_v8 smoke-test module changed from pub mod → mod, resolving the
   pre-existing unreachable_pub warning. No external consumers of the module.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-04-20 21:07:45 +02:00
parent d5fea00108
commit d9d35b6fad
2 changed files with 25 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
crates/ml/src/trainers/dqn/smoke_tests/mod.rs
View File

@@ -17,7 +17,7 @@ mod hyperopt;
#[cfg(test)]
mod regression;
#[cfg(test)]
pub mod reward_v8;
mod reward_v8;
#[cfg(test)]
mod generalization;
#[cfg(test)]

27
crates/ml/src/trainers/dqn/trainer/training_loop.rs
View File

@@ -1821,7 +1821,28 @@ impl DQNTrainer {
.unwrap_or(1.0);
let grad_norm = avg_grad.abs();
let ens_disagreement = 0.1f32; // placeholder — D6/N6 task will replace with real ensemble pairwise Q-gap
// Ensemble disagreement: dispersion across the 4 per-branch Q-gaps as a
// proxy for true per-head ensemble disagreement (per-head Q readback isn't
// wired in this codebase). When all branches collapse to similar Q-gap, the
// range shrinks → low disagreement → D6/N6 oracle score rises. When
// branches preserve diverse Q-gaps (healthy), the range is wider. Maps
// directly into the smoothstep(0.01, 0.1) thresholds in from_raw and the
// D6 oracle's own smoothstep with the same window.
let ens_disagreement = if let Some(ref fused) = self.fused_ctx {
let gaps = fused.per_branch_q_gap_ema();
let mut gmin = f32::INFINITY;
let mut gmax = f32::NEG_INFINITY;
for &g in &gaps {
if g.is_finite() {
if g < gmin { gmin = g; }
if g > gmax { gmax = g; }
}
}
if gmin.is_finite() && gmax.is_finite() { (gmax - gmin).max(0.0) } else { 0.1 }
} else {
0.1
};
// Update EMAs.
self.health_ema.update(q_gap_signal, q_var_signal, grad_norm);
@@ -2082,8 +2103,8 @@ impl DQNTrainer {
self.learning_health.components.spectral_gap_norm,
self.last_cql_alpha_eff.unwrap_or(0.0),
self.last_iqn_budget_eff.unwrap_or(0.40),
self.last_cql_budget_eff.unwrap_or(0.10),
self.last_c51_budget_eff.unwrap_or(0.45),
self.last_cql_budget_eff.unwrap_or(0.00),
self.last_c51_budget_eff.unwrap_or(0.55),
self.last_tau_eff.unwrap_or(0.005),
self.last_sarsa_tau_factor.unwrap_or(1.0),
self.last_gamma_eff.unwrap_or(0.99),