2df1ea92e1
BREAKING CHANGES: - Removed orphaned dqn.rs monolithic trainer (4,975 lines) - Removed orphaned dqn_ensemble.rs module (816 lines) - Removed orphaned tft.rs and tft_complete_int8_integration_test.rs - TFT trainer split into modular directory structure DQN Module Refactoring: - Split trainers/dqn.rs into modular structure (config.rs, statistics.rs, trainer.rs) - Fixed hyperopt 39D search space (continuous params only) - Boolean flags (use_dueling, use_double_dqn, use_per, use_noisy_nets) are now FIXED architectural decisions - use_distributional defaults to false (Candle BUG #36 - scatter_add gradient issues) Clean Module Structure: - ml/src/trainers/dqn/ directory with proper mod.rs exports - ml/src/trainers/tft/ directory with config.rs, types.rs, model.rs, trainer.rs, tests.rs - All P0 features validated: TD-error clamping, batch diversity, LR scheduler, priority staleness Documentation: - Added comprehensive docs in docs/codebase-cleanup/ - ADR-001 for DQN refactoring decisions - Rainbow DQN component matrix and quick reference guides Build Status: Compiles with zero errors 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
117 lines
13 KiB
Plaintext
117 lines
13 KiB
Plaintext
═══════════════════════════════════════════════════════════════════════════════
|
|
WAVE 26 P1.6: ADAPTIVE DROPOUT SCHEDULER - QUICK REFERENCE
|
|
═══════════════════════════════════════════════════════════════════════════════
|
|
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
│ WHAT IT DOES │
|
|
├─────────────────────────────────────────────────────────────────────────────┤
|
|
│ Linearly decreases dropout rate during training: │
|
|
│ • High dropout early (0.5) → prevents overfitting during exploration │
|
|
│ • Low dropout late (0.1) → allows fine-tuning during exploitation │
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
│ USAGE │
|
|
├─────────────────────────────────────────────────────────────────────────────┤
|
|
│ let config = QNetworkConfig { │
|
|
│ dropout_schedule: Some((0.5, 0.1, 100_000)), // initial, final, steps│
|
|
│ ..Default::default() │
|
|
│ }; │
|
|
│ │
|
|
│ // No changes needed: scheduler auto-updates on every forward() call │
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
│ TRAINING PROGRESSION EXAMPLE │
|
|
├─────────────────────────────────────────────────────────────────────────────┤
|
|
│ Step Dropout Phase │
|
|
│ ────────────────────────────────────────────────────────────── │
|
|
│ 0 0.500 Early: High regularization, robust learning │
|
|
│ 25,000 0.400 ↓ │
|
|
│ 50,000 0.300 Mid: Moderate dropout │
|
|
│ 75,000 0.200 ↓ │
|
|
│ 100,000+ 0.100 Late: Low dropout for fine-tuning │
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
│ IMPLEMENTATION DETAILS │
|
|
├─────────────────────────────────────────────────────────────────────────────┤
|
|
│ File: ml/src/dqn/network.rs │
|
|
│ │
|
|
│ Struct: DropoutScheduler │
|
|
│ • new(initial, final, decay_steps) → DropoutScheduler │
|
|
│ • get_rate() → f64 │
|
|
│ • step(steps: usize) │
|
|
│ │
|
|
│ Formula: │
|
|
│ progress = min(current_step / decay_steps, 1.0) │
|
|
│ rate = initial * (1 - progress) + final * progress │
|
|
│ │
|
|
│ Config: │
|
|
│ pub dropout_schedule: Option<(f64, f64, usize)> │
|
|
│ │
|
|
│ QNetwork: │
|
|
│ • dropout_scheduler: Mutex<Option<DropoutScheduler>> │
|
|
│ • get_dropout_rate() → f64 │
|
|
│ • Auto-steps on every forward() │
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
│ BACKWARD COMPATIBILITY │
|
|
├─────────────────────────────────────────────────────────────────────────────┤
|
|
│ ✅ 100% backward compatible │
|
|
│ dropout_schedule: None → uses static dropout_prob (existing behavior) │
|
|
│ │
|
|
│ ✅ Opt-in feature │
|
|
│ Only active when dropout_schedule is Some(...) │
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
│ TESTING │
|
|
├─────────────────────────────────────────────────────────────────────────────┤
|
|
│ Test Suite: ml/src/dqn/tests/dropout_scheduler_tests.rs │
|
|
│ • 11 comprehensive tests │
|
|
│ • Unit tests (scheduler logic) │
|
|
│ • Integration tests (QNetwork) │
|
|
│ • Edge cases (zero steps, constant rate) │
|
|
│ │
|
|
│ Standalone Verification: │
|
|
│ $ ./scripts/test_dropout_scheduler.sh │
|
|
│ ✅ All 12 tests passed! │
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
│ FILES CHANGED │
|
|
├─────────────────────────────────────────────────────────────────────────────┤
|
|
│ Modified: │
|
|
│ • ml/src/dqn/network.rs (DropoutScheduler + integration) │
|
|
│ • ml/src/dqn/tests/mod.rs (test module declaration) │
|
|
│ │
|
|
│ Created: │
|
|
│ • ml/src/dqn/tests/dropout_scheduler_tests.rs (test suite) │
|
|
│ • docs/codebase-cleanup/wave26_p1.6_adaptive_dropout_report.md │
|
|
│ • docs/codebase-cleanup/WAVE26_P1.6_SUMMARY.md │
|
|
│ • scripts/test_dropout_scheduler.sh │
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
│ BENEFITS │
|
|
├─────────────────────────────────────────────────────────────────────────────┤
|
|
│ Early Training (High Dropout): │
|
|
│ • Prevents overfitting to noisy experiences │
|
|
│ • Encourages robust feature learning │
|
|
│ • Regularizes exploration phase │
|
|
│ │
|
|
│ Late Training (Low Dropout): │
|
|
│ • Enables full network capacity │
|
|
│ • Allows precise fine-tuning │
|
|
│ • Improves final policy quality │
|
|
│ │
|
|
│ Smooth Transition: │
|
|
│ • Linear decay avoids sudden changes │
|
|
│ • Matches natural learning progression │
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
|
|
STATUS: ✅ COMPLETE - Implementation verified, tests passing
|
|
═══════════════════════════════════════════════════════════════════════════════
|