2df1ea92e1
BREAKING CHANGES: - Removed orphaned dqn.rs monolithic trainer (4,975 lines) - Removed orphaned dqn_ensemble.rs module (816 lines) - Removed orphaned tft.rs and tft_complete_int8_integration_test.rs - TFT trainer split into modular directory structure DQN Module Refactoring: - Split trainers/dqn.rs into modular structure (config.rs, statistics.rs, trainer.rs) - Fixed hyperopt 39D search space (continuous params only) - Boolean flags (use_dueling, use_double_dqn, use_per, use_noisy_nets) are now FIXED architectural decisions - use_distributional defaults to false (Candle BUG #36 - scatter_add gradient issues) Clean Module Structure: - ml/src/trainers/dqn/ directory with proper mod.rs exports - ml/src/trainers/tft/ directory with config.rs, types.rs, model.rs, trainer.rs, tests.rs - All P0 features validated: TD-error clamping, batch diversity, LR scheduler, priority staleness Documentation: - Added comprehensive docs in docs/codebase-cleanup/ - ADR-001 for DQN refactoring decisions - Rainbow DQN component matrix and quick reference guides Build Status: Compiles with zero errors 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
66 lines
2.2 KiB
Bash
Executable File
66 lines
2.2 KiB
Bash
Executable File
#!/bin/bash
|
|
# Test script to verify DQN initialization is non-deterministic
|
|
# Runs 3 parallel training instances and extracts initial Q-values
|
|
|
|
set -e
|
|
|
|
echo "=== DQN Non-Deterministic Initialization Test ==="
|
|
echo "Starting 3 parallel training runs with 1 epoch each..."
|
|
echo ""
|
|
|
|
# Clean up old test outputs
|
|
rm -rf /tmp/init_test_* 2>/dev/null || true
|
|
|
|
# Run 3 training instances in parallel
|
|
cargo run --package ml --example train_dqn --release --features cuda -- \
|
|
--epochs 1 --output-dir /tmp/init_test_1 > /tmp/init_test_1.log 2>&1 &
|
|
PID1=$!
|
|
|
|
cargo run --package ml --example train_dqn --release --features cuda -- \
|
|
--epochs 1 --output-dir /tmp/init_test_2 > /tmp/init_test_2.log 2>&1 &
|
|
PID2=$!
|
|
|
|
cargo run --package ml --example train_dqn --release --features cuda -- \
|
|
--epochs 1 --output-dir /tmp/init_test_3 > /tmp/init_test_3.log 2>&1 &
|
|
PID3=$!
|
|
|
|
echo "Waiting for training runs to complete..."
|
|
echo " PID $PID1 (test 1)"
|
|
echo " PID $PID2 (test 2)"
|
|
echo " PID $PID3 (test 3)"
|
|
echo ""
|
|
|
|
wait $PID1 $PID2 $PID3
|
|
|
|
echo "All training runs completed. Extracting Q-values..."
|
|
echo ""
|
|
|
|
# Extract initial Q-values from logs
|
|
echo "=== Run 1 - Initial Q-Values ==="
|
|
grep -E "Step 0.*Q-values:" /tmp/init_test_1.log | head -1 || echo "No Q-values found in Run 1"
|
|
echo ""
|
|
|
|
echo "=== Run 2 - Initial Q-Values ==="
|
|
grep -E "Step 0.*Q-values:" /tmp/init_test_2.log | head -1 || echo "No Q-values found in Run 2"
|
|
echo ""
|
|
|
|
echo "=== Run 3 - Initial Q-Values ==="
|
|
grep -E "Step 0.*Q-values:" /tmp/init_test_3.log | head -1 || echo "No Q-values found in Run 3"
|
|
echo ""
|
|
|
|
# Extract entropy seeds
|
|
echo "=== Entropy Seeds Used ==="
|
|
echo "Run 1:"
|
|
grep "Device RNG seeded with entropy:" /tmp/init_test_1.log | head -1 || echo "No seed found"
|
|
echo "Run 2:"
|
|
grep "Device RNG seeded with entropy:" /tmp/init_test_2.log | head -1 || echo "No seed found"
|
|
echo "Run 3:"
|
|
grep "Device RNG seeded with entropy:" /tmp/init_test_3.log | head -1 || echo "No seed found"
|
|
echo ""
|
|
|
|
echo "=== Validation ==="
|
|
echo "SUCCESS: If the Q-values and seeds are DIFFERENT across runs, the fix is working!"
|
|
echo "FAILURE: If the Q-values are IDENTICAL across runs, the issue persists."
|
|
echo ""
|
|
echo "Logs saved to: /tmp/init_test_{1,2,3}.log"
|