2df1ea92e1
BREAKING CHANGES: - Removed orphaned dqn.rs monolithic trainer (4,975 lines) - Removed orphaned dqn_ensemble.rs module (816 lines) - Removed orphaned tft.rs and tft_complete_int8_integration_test.rs - TFT trainer split into modular directory structure DQN Module Refactoring: - Split trainers/dqn.rs into modular structure (config.rs, statistics.rs, trainer.rs) - Fixed hyperopt 39D search space (continuous params only) - Boolean flags (use_dueling, use_double_dqn, use_per, use_noisy_nets) are now FIXED architectural decisions - use_distributional defaults to false (Candle BUG #36 - scatter_add gradient issues) Clean Module Structure: - ml/src/trainers/dqn/ directory with proper mod.rs exports - ml/src/trainers/tft/ directory with config.rs, types.rs, model.rs, trainer.rs, tests.rs - All P0 features validated: TD-error clamping, batch diversity, LR scheduler, priority staleness Documentation: - Added comprehensive docs in docs/codebase-cleanup/ - ADR-001 for DQN refactoring decisions - Rainbow DQN component matrix and quick reference guides Build Status: Compiles with zero errors 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
53 lines
1.5 KiB
Bash
Executable File
53 lines
1.5 KiB
Bash
Executable File
#!/bin/bash
|
|
LOG_FILE="/tmp/ml_training/hyperopt_full/hyperopt_full_run.log"
|
|
|
|
echo "=== DQN Hyperopt Campaign Analysis ==="
|
|
echo ""
|
|
|
|
# Start time
|
|
START_TIME=$(head -5 "$LOG_FILE" | grep -E "^\\[2m20" | head -1 | sed 's/\[2m\(.*\)Z\[0m.*/\1/')
|
|
echo "Start time: $START_TIME"
|
|
|
|
# End time
|
|
END_TIME=$(tail -5 "$LOG_FILE" | grep -E "^\\[2m20" | tail -1 | sed 's/\[2m\(.*\)Z\[0m.*/\1/')
|
|
echo "End time: $END_TIME"
|
|
|
|
echo ""
|
|
echo "=== TRIAL STATISTICS ==="
|
|
|
|
# Count completed trials (those with "✓ Trial N completed")
|
|
COMPLETED=$(grep "✓ Trial" "$LOG_FILE" | wc -l)
|
|
echo "Trials completed: $COMPLETED"
|
|
|
|
# Count pruned trials
|
|
PRUNED_GRAD=$(grep "gradient explosion" "$LOG_FILE" | wc -l)
|
|
PRUNED_Q=$(grep "Q-value collapse" "$LOG_FILE" | wc -l)
|
|
PRUNED_TOTAL=$((PRUNED_GRAD + PRUNED_Q))
|
|
|
|
echo "Pruned (gradient explosion): $PRUNED_GRAD"
|
|
echo "Pruned (Q-value collapse): $PRUNED_Q"
|
|
echo "Pruned (total): $PRUNED_TOTAL"
|
|
echo "Valid trials: $((COMPLETED - PRUNED_TOTAL))"
|
|
|
|
echo ""
|
|
echo "=== LAST 10 COMPLETED TRIALS ==="
|
|
grep "✓ Trial" "$LOG_FILE" | tail -10
|
|
|
|
echo ""
|
|
echo "=== ALL PRUNED TRIALS ==="
|
|
grep "PRUNED" "$LOG_FILE" | grep -E "(Trial [0-9]+)"
|
|
|
|
echo ""
|
|
echo "=== CAMPAIGN STATUS ==="
|
|
if grep -q "Optimization finished" "$LOG_FILE"; then
|
|
echo "Status: ✅ COMPLETED"
|
|
elif ps aux | grep -q "[h]yperopt_dqn_demo"; then
|
|
echo "Status: 🔄 RUNNING"
|
|
else
|
|
echo "Status: ❌ CRASHED or TERMINATED"
|
|
echo ""
|
|
echo "Last 10 log lines:"
|
|
tail -10 "$LOG_FILE" | sed 's/\[2m//g; s/\[0m//g; s/\[32m//g; s/\[33m//g'
|
|
fi
|
|
|