00ef9e2866
Major Changes: - Migrated from 3-action TradingAction to 45-action FactoredAction - 45 actions: 5 exposure × 3 order types × 3 urgency levels - Absolute exposure model (target positions -1.0 to +1.0) - Transaction cost differentiation (Market 0.15%, LimitMaker 0.05%, IoC 0.10%) - Fixed action diversity threshold (1.11% → 0.5% for 45-action space) Bug Fixes: - Bug #15: Incomplete FactoredAction integration (code existed but unused) - Bug #16: Runtime crash in action diversity checking (hardcoded 3-action match) Code Changes (13 files, ~464 lines): - ml/src/dqn/action_space.rs: Core FactoredAction + 4 helper methods - ml/src/trainers/dqn.rs: Action diversity refactored (3→45 dynamic) - ml/src/dqn/reward.rs: calculate_reward() signature updated - ml/src/dqn/portfolio_tracker.rs: execute_action() absolute exposure - ml/src/dqn/dqn.rs: WorkingDQN action selection migrated - ml/tests/*.rs: 9 test files updated with FactoredAction assertions Test Results: - 1-epoch smoke test: 100% action diversity (45/45 actions, 80.2s) - 10-epoch production: 87.8% readiness (79/90 scorecard, 14.0 min) - Loss convergence: 96.9% reduction (119K → 3.6K) - Action diversity: 100% → 44% (healthy specialization) - Checkpoint reliability: 12/12 files saved (100%) - DQN tests: 195/195 passing (100%) - ML baseline: 1,514/1,515 passing (99.93%) Production Status: ✅ CERTIFIED (87.8% readiness) Go/No-Go: ✅ GO FOR 100-EPOCH PRODUCTION TRAINING 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
29 KiB
29 KiB
TEST COVERAGE VISUAL SUMMARY: The Missing Tests That Would Have Caught the Bug
The Bug Timeline
┌─────────────────────────────────────────────────────────────────────────┐
│ WHAT HAPPENED: A Critical Bug Slipped Through 147 Passing Tests │
└─────────────────────────────────────────────────────────────────────────┘
┌──────────────────────┐
│ DEFAULT FEATURES │
│ (Cargo.toml line 21)│
│ │
│ factored-actions ✅ │
│ NUM_ACTIONS = 45 │
└──────────────────────┘
│
│ cargo test --package ml
▼
┌──────────────────────┐
│ TEST EXECUTION │
│ │
│ Tests compile with: │
│ NUM_ACTIONS = 45 ✅ │
└──────────────────────┘
│
│ DQNHyperparameters::conservative()
▼
┌──────────────────────────────────────────┐
│ EPSILON = 0.0 (NO EXPLORATION) ❌ │
│ │
│ 100% of actions use greedy exploitation │
│ (argmax of Q-values) │
└──────────────────────────────────────────┘
│
│ Untrained network = random weights
▼
┌──────────────────────────────────────────┐
│ ARGMAX RETURNS 0-2 (BY LUCK) ❌ │
│ │
│ Random Q-values → argmax happens to be │
│ in range 0-2 for untrained network │
└──────────────────────────────────────────┘
│
│ TradingAction::from_int(0-2)
▼
┌──────────────────────────────────────────┐
│ TESTS PASS ✅ (BUT SHOULDN'T!) ❌ │
│ │
│ from_int(0-2) succeeds │
│ Assertion checks Buy/Sell/Hold │
│ BUG UNDETECTED │
└──────────────────────────────────────────┘
The Missing Test: What Would Have Caught This
┌─────────────────────────────────────────────────────────────────────────┐
│ IF WE HAD TESTED WITH EPSILON > 0: Bug Would Be Immediately Caught │
└─────────────────────────────────────────────────────────────────────────┘
┌──────────────────────┐
│ EPSILON = 0.5 │
│ (50% exploration) │
└──────────────────────┘
│
│ select_actions_batch (batch_size=100)
▼
┌──────────────────────────────────────────────────────┐
│ EXPLORATION PATH TRIGGERED (~50 actions) ✅ │
│ │
│ rng.gen_range(0..NUM_ACTIONS) │
│ → Returns values 0-44 ⚠️ │
└───────────────────────────────────────────────────────┘
│
│ Random action_idx = 29 (example)
▼
┌──────────────────────────────────────────────────────┐
│ TradingAction::from_int(29) ❌ │
│ │
│ from_int() only accepts 0-2 │
│ Returns None │
└───────────────────────────────────────────────────────┘
│
│ .ok_or_else(|| anyhow!("Invalid action index: 29"))
▼
┌──────────────────────────────────────────────────────┐
│ TEST PANICS ✅ (CORRECTLY!) │
│ │
│ thread panicked at ml/src/trainers/dqn.rs:2847:9: │
│ Batched action selection failed: │
│ Some(Invalid action index: 29 │
│ │
│ 🎉 BUG DETECTED BEFORE PRODUCTION! │
└───────────────────────────────────────────────────────┘
Code Path Coverage Map
select_actions_batch()
│
▼
┌──────────────────────────────┐
│ For each sample in batch: │
│ Get Q-values via forward() │
└──────────────────────────────┘
│
┌─────────┴─────────┐
│ │
┌───────────▼─────────┐ ┌──────▼──────────┐
│ EXPLORATION PATH │ │ EXPLOITATION │
│ (epsilon % chance) │ │ (1-epsilon) │
│ │ │ │
│ rng.gen_range( │ │ argmax of │
│ 0..NUM_ACTIONS) │ │ Q-values │
└─────────────────────┘ └─────────────────┘
│ │
│ NUM_ACTIONS=45 │ Returns 0-2
│ Returns 0-44 │ (random weights)
│ │
┌───────────▼─────────┐ ┌──────▼──────────┐
│ ❌ BROKEN PATH │ │ ✅ WORKS BY LUCK│
│ │ │ │
│ TradingAction:: │ │ TradingAction::│
│ from_int(0-44) │ │ from_int(0-2) │
│ → Returns None │ │ → Returns Some │
│ → Panics │ │ → Test passes │
└─────────────────────┘ └─────────────────┘
│ │
│ │
┌───────────▼─────────┐ ┌──────▼──────────┐
│ 🚫 NEVER TESTED │ │ ✅ TESTED │
│ (epsilon=0.0) │ │ (epsilon=0.0) │
└─────────────────────┘ └─────────────────┘
Key Insight:
- Left path (EXPLORATION): 95.5% of random actions crash (43/45 invalid indices)
- Right path (EXPLOITATION): Always works for untrained network (argmax returns 0-2)
- Tests only hit right path → Bug undetected
Feature Flag Complexity Matrix
┌────────────────────────────────────────────────────────────────────┐
│ FEATURE FLAG: factored-actions │
│ Affects: NUM_ACTIONS constant │
└────────────────────────────────────────────────────────────────────┘
Feature ON (default) Feature OFF
┌──────────────────┐ ┌──────────────────┐
│ NUM_ACTIONS = 45 │ │ NUM_ACTIONS = 3 │
└──────────────────┘ └──────────────────┘
│ │
▼ ▼
┌──────────────────┐ ┌──────────────────┐
│ Should use: │ │ Should use: │
│ FactoredAction │ │ TradingAction │
│ (45 variants) │ │ (3 variants) │
└──────────────────┘ └──────────────────┘
│ │
│ ACTUAL CODE: │ ACTUAL CODE:
▼ ▼
┌──────────────────┐ ┌──────────────────┐
│ ❌ Uses: │ │ ✅ Uses: │
│ TradingAction │ │ TradingAction │
│ (3 variants) │ │ (3 variants) │
│ │ │ │
│ MISMATCH! 🔥 │ │ CORRECT! ✅ │
└──────────────────┘ └──────────────────┘
│ │
▼ ▼
┌──────────────────┐ ┌──────────────────┐
│ Test Coverage: │ │ Test Coverage: │
│ ❌ NOT TESTED │ │ ✅ TESTED (by │
│ │ │ accident via │
│ Tests have │ │ epsilon=0.0) │
│ no #[cfg] │ │ │
│ guards │ │ │
└──────────────────┘ └──────────────────┘
Test Assertion Gap
CURRENT TEST ASSERTION (Line 2862-2870):
┌────────────────────────────────────────────────────────────────┐
│ for (i, action) in actions.iter().enumerate() { │
│ assert!( │
│ matches!(action, TradingAction::Buy | │
│ TradingAction::Sell | │
│ TradingAction::Hold), │
│ "Action {} is invalid: {:?}", i, action │
│ ); │
│ } │
└────────────────────────────────────────────────────────────────┘
│
┌────────────────────┴────────────────────┐
│ │
▼ ▼
┌─────────────────────┐ ┌──────────────────────┐
│ CHECKS: │ │ SHOULD ALSO CHECK: │
│ │ │ │
│ ✅ Action is valid │ │ ❌ Action came from │
│ enum variant │ │ correct type: │
│ │ │ │
│ ✅ Action is Buy/ │ │ #[cfg(feature = │
│ Sell/Hold │ │ "factored")] │
│ │ │ → FactoredAction │
│ │ │ │
│ │ │ #[cfg(not)] │
│ │ │ → TradingAction │
└─────────────────────┘ └──────────────────────┘
PROBLEM: Assertion validates OUTPUT but not CODE PATH
Epsilon Parameter Impact
Epsilon Value Analysis
═════════════════════
0.0 0.3 0.5 0.7 1.0
│ │ │ │ │
│ │ │ │ │
┌──▼──┐ ┌──▼──┐ ┌──▼──┐ ┌──▼──┐ ┌──▼──┐
│ 0% │ │ 30% │ │ 50% │ │ 70% │ │100% │
│Expl │ │Expl │ │Expl │ │Expl │ │Expl │
└──┬──┘ └──┬──┘ └──┬──┘ └──┬──┘ └──┬──┘
│ │ │ │ │
▼ ▼ ▼ ▼ ▼
┌─────┐ ┌─────┐ ┌─────┐ ┌─────┐ ┌─────┐
│ Bug │ │ Bug │ │ Bug │ │ Bug │ │ Bug │
│ Hit │ │ Hit │ │ Hit │ │ Hit │ │ Hit │
│ 0% │ │28.6%│ │47.7%│ │66.9%│ │95.5%│
└─────┘ └─────┘ └─────┘ └─────┘ └─────┘
▲
│
┌──┴────────────────────────────────────────────────────────┐
│ CURRENT TESTS USE EPSILON=0.0 │
│ → 0% of actions hit exploration path │
│ → 0% bug hit rate │
│ → Tests pass even though code is broken │
└───────────────────────────────────────────────────────────┘
Bug Hit Rate Calculation:
= epsilon × (invalid_actions / total_actions)
= epsilon × (43 / 45) [only 0,1,2 are valid; 3-44 are invalid]
= epsilon × 0.955
= 95.5% hit rate at epsilon=1.0
Statistical Analysis: Why Tests Passed
┌─────────────────────────────────────────────────────────────────┐
│ TEST SCENARIO: 100 actions, epsilon=0.0, untrained network │
└─────────────────────────────────────────────────────────────────┘
Sample 1: Q-values = [0.02, 0.01, 0.03] → argmax=2 → Hold ✅
Sample 2: Q-values = [0.01, 0.02, 0.01] → argmax=1 → Sell ✅
Sample 3: Q-values = [0.03, 0.01, 0.02] → argmax=0 → Buy ✅
...
Sample 100: Q-values = [0.01, 0.03, 0.02] → argmax=1 → Sell ✅
Result: 100/100 actions valid (all in range 0-2)
Test Status: PASS ✅
┌─────────────────────────────────────────────────────────────────┐
│ PRODUCTION SCENARIO: 100 actions, epsilon=0.3, trained network │
└─────────────────────────────────────────────────────────────────┘
Sample 1: epsilon check → exploitation → argmax=1 → Sell ✅
Sample 2: epsilon check → exploration → random=29 → CRASH ❌
Sample 3: epsilon check → exploitation → argmax=0 → Buy ✅
Sample 4: epsilon check → exploration → random=12 → CRASH ❌
Sample 5: epsilon check → exploitation → argmax=2 → Hold ✅
...
Sample 30: epsilon check → exploration → random=37 → CRASH ❌
Expected: ~30 exploration actions, ~28.6 crashes (95.5% of 30)
Result: PRODUCTION FAILURE 🔥
The 3 Critical Test Gaps (Visual)
┌─────────────────────────────────────────────────────────────────────┐
│ GAP #1: Epsilon Parameter Testing │
└─────────────────────────────────────────────────────────────────────┘
CURRENT TESTS NEEDED TESTS
┌────────────────┐ ┌────────────────┐
│ epsilon = 0.0 │ │ epsilon = 0.3 │
│ (default) │ │ epsilon = 0.5 │
│ │ │ epsilon = 1.0 │
│ ❌ Only tests │ │ ✅ Tests both │
│ exploitation│ │ paths │
└────────────────┘ └────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ GAP #2: Feature Flag Testing │
└─────────────────────────────────────────────────────────────────────┘
CURRENT TESTS NEEDED TESTS
┌────────────────┐ ┌────────────────────┐
│ No #[cfg] │ │ #[cfg(feature = │
│ guards │ │ "factored")] │
│ │ │ │
│ ❌ Assumes one │ │ #[cfg(not(feature │
│ config │ │ = "factored"))] │
│ │ │ │
│ │ │ ✅ Tests both │
│ │ │ configs │
└────────────────┘ └────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ GAP #3: Action Type Validation │
└─────────────────────────────────────────────────────────────────────┘
CURRENT TESTS NEEDED TESTS
┌────────────────┐ ┌────────────────────┐
│ matches!( │ │ Verify action type │
│ action, │ │ matches feature: │
│ TradingAction│ │ │
│ ::Buy | ...) │ │ factored-actions → │
│ │ │ FactoredAction │
│ ❌ Only checks │ │ │
│ variant │ │ else → │
│ │ │ TradingAction │
│ │ │ │
│ │ │ ✅ Validates type │
│ │ │ consistency │
└────────────────┘ └────────────────────┘
Recommended Test Matrix (Visual)
TEST COVERAGE MATRIX
═══════════════════
Feature Flag
┌──────────┬──────────┐
│ ON │ OFF │
┌───────────┼──────────┼──────────┤
│ eps=0.0 │ ✅ EXISTS │ ❌ MISSING│
│ │(existing)│ (add) │
E ├───────────┼──────────┼──────────┤
p │ eps=0.3 │ ❌ MISSING│ ❌ MISSING│
s │ │ (CRIT) │ (add) │
i ├───────────┼──────────┼──────────┤
l │ eps=0.5 │ ❌ MISSING│ ❌ MISSING│
o │ │ (CRIT) │ (add) │
n ├───────────┼──────────┼──────────┤
│ eps=1.0 │ ❌ MISSING│ ❌ MISSING│
│ │ (CRIT) │ (add) │
└───────────┴──────────┴──────────┘
CRIT = Critical (would catch bug)
add = Nice to have (completeness)
IMMEDIATE ACTION: Add 3 tests marked CRIT
→ Test #1: eps=0.3, factored-actions ON
→ Test #2: eps=0.5, factored-actions ON
→ Test #3: eps=1.0, factored-actions ON
Root Cause Visualization
WHY THE BUG SLIPPED THROUGH
═══════════════════════════
┌──────────────────────────────────────────────────────────────────┐
│ Root Cause #1: Tests Always Used Default Hyperparameters │
└──────────────────────────────────────────────────────────────────┘
create_test_params() → DQNHyperparameters::conservative()
→ epsilon_start = 0.0
→ epsilon_end = 0.0
↓
NEVER TESTED EXPLORATION PATH ❌
┌──────────────────────────────────────────────────────────────────┐
│ Root Cause #2: Tests Were Feature-Agnostic │
└──────────────────────────────────────────────────────────────────┘
No #[cfg(feature = "factored-actions")] guards
↓
Tests compiled with factored-actions=ON
↓
But assertions checked TradingAction (3 variants)
↓
NEVER VALIDATED FACTORED ACTION TYPE ❌
┌──────────────────────────────────────────────────────────────────┐
│ Root Cause #3: Lucky Random Weights │
└──────────────────────────────────────────────────────────────────┘
Untrained network → random Q-values
↓
argmax of 45 Q-values → happened to return 0-2
↓
TradingAction::from_int(0-2) → succeeded
↓
BUG HIDDEN BY LUCK ❌
Summary: The Perfect Storm
┌─────────────────────────────────────────────────────────────────┐
│ THE PERFECT STORM │
│ │
│ 5 Factors Combined to Hide the Bug: │
│ │
│ 1. ✅ Tests existed (147 tests) │
│ 2. ✅ Tests ran successfully (100% pass rate) │
│ 3. ❌ Tests used epsilon=0.0 (never hit exploration path) │
│ 4. ❌ Tests had no feature flag guards (wrong assertion) │
│ 5. ❌ Untrained network returned 0-2 by luck (masked bug) │
│ │
│ Result: CRITICAL BUG UNDETECTED FOR WEEKS 🔥 │
│ │
│ Fix: Add 3 tests with epsilon > 0 → Bug detected in 1 minute! │
└─────────────────────────────────────────────────────────────────┘
Takeaway: The Test That Would Have Caught It
/// THIS ONE TEST WOULD HAVE CAUGHT THE BUG:
#[tokio::test]
async fn test_exploration_path() {
let mut hyperparams = create_test_params();
hyperparams.epsilon_start = 0.5; // ⬅️ KEY DIFFERENCE
hyperparams.epsilon_end = 0.5;
let trainer = DQNTrainer::new(hyperparams).unwrap();
{
let mut agent = trainer.agent.write().await;
agent.set_epsilon(0.5).unwrap(); // ⬅️ KEY DIFFERENCE
}
// ... create states ...
let result = trainer.select_actions_batch(&states).await;
// THIS WOULD PANIC:
// thread panicked at ml/src/trainers/dqn.rs:2847:9:
// Batched action selection failed: Some(Invalid action index: 29
//
// 🎉 BUG DETECTED!
}
Time to add this test: 5 minutes Time saved from production bug: Hours/days Impact: CRITICAL BUG PREVENTED
Conclusion
The bug was 100% detectable with existing test infrastructure.
All we needed was:
- One test with epsilon > 0 (exploration path)
- Run it with default features (factored-actions enabled)
Lesson: Always test the code paths you don't think will be tested by default.