2df1ea92e1
BREAKING CHANGES: - Removed orphaned dqn.rs monolithic trainer (4,975 lines) - Removed orphaned dqn_ensemble.rs module (816 lines) - Removed orphaned tft.rs and tft_complete_int8_integration_test.rs - TFT trainer split into modular directory structure DQN Module Refactoring: - Split trainers/dqn.rs into modular structure (config.rs, statistics.rs, trainer.rs) - Fixed hyperopt 39D search space (continuous params only) - Boolean flags (use_dueling, use_double_dqn, use_per, use_noisy_nets) are now FIXED architectural decisions - use_distributional defaults to false (Candle BUG #36 - scatter_add gradient issues) Clean Module Structure: - ml/src/trainers/dqn/ directory with proper mod.rs exports - ml/src/trainers/tft/ directory with config.rs, types.rs, model.rs, trainer.rs, tests.rs - All P0 features validated: TD-error clamping, batch diversity, LR scheduler, priority staleness Documentation: - Added comprehensive docs in docs/codebase-cleanup/ - ADR-001 for DQN refactoring decisions - Rainbow DQN component matrix and quick reference guides Build Status: Compiles with zero errors 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
27 lines
983 B
Bash
Executable File
27 lines
983 B
Bash
Executable File
#!/bin/bash
|
|
# Extract diagnostic metrics from gamma 0.90 test logs
|
|
|
|
LOG_FILE="/tmp/ml_training/gamma_0.90_diagnostic.log"
|
|
OUTPUT_DIR="/home/jgrusewski/Work/foxhunt/diagnostic_data"
|
|
|
|
mkdir -p "$OUTPUT_DIR"
|
|
|
|
echo "Extracting Q-value progression..."
|
|
grep "Step [0-9]* Q-values:" "$LOG_FILE" | \
|
|
awk -F'Step | Q-values: BUY=|, SELL=|, HOLD=' '{print $2,$3,$4,$5}' | \
|
|
head -100 > "$OUTPUT_DIR/q_value_progression_gamma_0.90.txt"
|
|
|
|
echo "Extracting gradient metrics..."
|
|
grep "grad_norm=" "$LOG_FILE" | \
|
|
awk -F'grad_norm=|, train_steps=' '{print $1,$2}' | \
|
|
head -50 > "$OUTPUT_DIR/gradient_progression_gamma_0.90.txt"
|
|
|
|
echo "Extracting epoch-level metrics..."
|
|
grep "Epoch [0-9]*/10: train_loss=" "$LOG_FILE" > "$OUTPUT_DIR/epoch_metrics_gamma_0.90.txt"
|
|
|
|
echo "Extracting gradient collapse occurrences..."
|
|
grep "GRADIENT COLLAPSE" "$LOG_FILE" | wc -l > "$OUTPUT_DIR/gradient_collapse_count_gamma_0.90.txt"
|
|
|
|
echo "✅ Diagnostic data extracted to: $OUTPUT_DIR"
|
|
ls -lh "$OUTPUT_DIR"
|