gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fix(ci): correct epsilon assertion for noisy nets + reduce pipeline epochs

Browse Source 
The epsilon assertion expected <0.01 (epsilon=0.0 with noisy nets), but
the codebase evolved: noisy_epsilon_floor (0.05) now provides a minimum
exploration rate to prevent action collapse while NoisyNets handle the
primary learned exploration. Updated assertions to match: epsilon < 0.10.

Also reduced pipeline test epochs (10→5, 20→10) to prevent GPU timeout
when 5 concurrent DQN trainers share one H100.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-03-13 18:12:43 +01:00
parent 45eec3f1f2
commit b87b7b4bea
2 changed files with 17 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

22
crates/ml/tests/dqn_training_pipeline_test.rs
View File

@@ -159,15 +159,15 @@ async fn test_dqn_trains_on_es_fut() -> Result<()> {
let checkpoint_dir = create_checkpoint_dir()?;
// Configure hyperparameters for fast test (10 epochs)
// Configure hyperparameters for fast test (5 epochs)
let mut hyperparams = DQNHyperparameters::conservative();
hyperparams.epochs = 10; // Fast test
hyperparams.epochs = 5; // Fast CI test
hyperparams.batch_size = 64;
hyperparams.learning_rate = 0.001;
hyperparams.epsilon_start = 0.5; // Reduced for faster training
hyperparams.epsilon_end = 0.05;
hyperparams.checkpoint_frequency = 5;
hyperparams.early_stopping_enabled = false; // Test all 10 epochs
hyperparams.early_stopping_enabled = false; // Test all epochs
println!(" ✅ Configuration ready");
println!(" 📂 Data directory: {}", data_dir);
@@ -295,9 +295,9 @@ async fn test_dqn_loss_decreases() -> Result<()> {
let checkpoint_dir = create_checkpoint_dir()?;
// Train for 20 epochs to measure convergence
// Train for 10 epochs to measure convergence
let mut hyperparams = DQNHyperparameters::conservative();
hyperparams.epochs = 20;
hyperparams.epochs = 10;
hyperparams.batch_size = 64;
hyperparams.learning_rate = 0.001;
hyperparams.early_stopping_enabled = false;
@@ -496,7 +496,7 @@ async fn test_dqn_epsilon_greedy() -> Result<()> {
// Configure with high epsilon decay
let mut hyperparams = DQNHyperparameters::conservative();
hyperparams.epochs = 10;
hyperparams.epochs = 5;
hyperparams.epsilon_start = 1.0;
hyperparams.epsilon_end = 0.01;
hyperparams.epsilon_decay = 0.9; // Fast decay
@@ -514,14 +514,16 @@ async fn test_dqn_epsilon_greedy() -> Result<()> {
.await?;
// BUG #40 VERIFIED: With noisy nets enabled (conservative() default),
// epsilon must be 0.0 — exploration comes from network noise, not random actions.
// epsilon is fixed at noisy_epsilon_floor (0.05) — not decayed, not 1.0.
// Before the fix, epsilon stayed at 1.0 (100% random actions throughout training).
// The noisy_epsilon_floor provides a minimum exploration rate to prevent action collapse
// while NoisyNets provide the primary learned exploration signal.
let final_epsilon = trainer.get_agent_epsilon().await;
println!(" Final epsilon: {:.4}", final_epsilon);
assert!(
final_epsilon < 0.01,
"BUG #40: Epsilon should be ~0.0 with noisy nets (got {:.4}). \
final_epsilon < 0.10,
"BUG #40: Epsilon should be at noisy_epsilon_floor (~0.05) with noisy nets (got {:.4}). \
If this fails, noisy net epsilon override is broken.",
final_epsilon
);
@@ -535,7 +537,7 @@ async fn test_dqn_epsilon_greedy() -> Result<()> {
);
println!(" Avg Q-value: {:.4}", avg_q);
println!(" ✅ Noisy nets exploration validated (epsilon=0, Q-values drive actions)");
println!(" ✅ Noisy nets exploration validated (epsilon=floor, Q-values+noise drive actions)");
Ok(())
}

8
crates/ml/tests/dqn_training_smoke_test.rs
View File

@@ -228,11 +228,13 @@ async fn test_dqn_training_smoke() -> Result<()> {
);
// === ASSERT 6: Epsilon correct for noisy nets (BUG #40 FIX) ===
// conservative() enables noisy nets → epsilon must be 0.0 (noise provides exploration)
// conservative() enables noisy nets → epsilon fixed at noisy_epsilon_floor (0.05).
// NoisyNets provide learned exploration; the floor prevents action collapse.
// Before the fix, epsilon stayed at 1.0 (100% random actions throughout training).
let final_epsilon = trainer.get_agent_epsilon().await;
assert!(
final_epsilon < 0.01,
"ASSERT 6 FAILED: Epsilon should be ~0.0 with noisy nets (got {:.4}). BUG #40 fix missing.",
final_epsilon < 0.10,
"ASSERT 6 FAILED: Epsilon should be at noisy_epsilon_floor (~0.05) with noisy nets (got {:.4}). BUG #40 fix missing.",
final_epsilon
);