gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fix(dqn): Bug #2 — epsilon_greedy_action samples 4-branch factored space (Task 2.5)

Browse Source 
Stale 0..5 range from pre-2026-04-08 9-level code. Only test paths call
this cold-path fallback, but the stale range was a latent foot-gun and
would mislead anyone reading the code (per feedback_trust_code_not_docs).

Fix: sample dir ∈ [0,3), mag ∈ [0,3), ord ∈ [0,3), urg ∈ [0,3) and encode
as `dir*27 + mag*9 + ord*3 + urg`, matching MEMORY.md 4-branch DQN
architecture (81 factored actions).

Per Track 4 E1 triage tech-debt flag.
This commit is contained in:
jgrusewski
2026-04-22 15:51:18 +02:00
parent ef165e8961
commit 2a7005f29a
1 changed files with 12 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
crates/ml/src/trainers/dqn/trainer/action.rs
View File

@@ -164,8 +164,18 @@ impl DQNTrainer {
use rand::rngs::StdRng;
let epsilon = self.get_epsilon().await? as f32;
let mut rng = StdRng::seed_from_u64(0xAC7_DEF0);
if rng.gen::<f32>() < epsilon { Ok(rng.gen_range(0..5)) }
else {
if rng.gen::<f32>() < epsilon {
// Task 2.5 Bug #2: sample 4-branch factored action space
// (dir × mag × ord × urg = 3×3×3×3 = 81), matching MEMORY.md
// DQN architecture (updated 2026-04-08). The legacy 0..5 range
// predated the 4-branch refactor and was a latent foot-gun.
// Encoding: action = dir*27 + mag*9 + ord*3 + urg.
let dir = rng.gen_range(0..3usize);
let mag = rng.gen_range(0..3usize);
let ord = rng.gen_range(0..3usize);
let urg = rng.gen_range(0..3usize);
Ok(dir * 27 + mag * 9 + ord * 3 + urg)
} else {
let agent = self.agent.read().await;
let indices = agent.batch_greedy_actions(state)?;
let best_action = *indices.first().ok_or_else(|| anyhow::anyhow!("argmax on empty Q-values"))? as usize;