diff --git a/DQN_HYPEROPT_DRYRUN_INSTRUCTIONS.md b/DQN_HYPEROPT_DRYRUN_INSTRUCTIONS.md new file mode 100644 index 000000000..8d9b9e1b1 --- /dev/null +++ b/DQN_HYPEROPT_DRYRUN_INSTRUCTIONS.md @@ -0,0 +1,241 @@ +# DQN Hyperopt Dry-Run Instructions + +**Created**: 2025-11-06 (Wave 12-A29) +**Purpose**: Validate hyperopt setup before full 100-trial campaign +**Location**: `/home/jgrusewski/Work/foxhunt/scripts/hyperopt_dqn_dryrun.sh` + +--- + +## Quick Start + +```bash +# Run the dry-run (5-10 minutes) +./scripts/hyperopt_dqn_dryrun.sh +``` + +--- + +## What It Does + +### 1. Configuration +- **Trials**: 5 (fast validation) +- **Epochs per trial**: 10 (enough to see convergence) +- **Duration**: 5-10 minutes +- **Cost**: $0.02-$0.04 (RTX A4000) + +### 2. Pre-Flight Checks +- ✅ Verifies parquet file exists (`test_data/ES_FUT_180d.parquet`) +- ✅ Checks CUDA availability +- ✅ Displays GPU information + +### 3. Training Run +- Executes `cargo run --release -p ml --example hyperopt_dqn_demo --features cuda` +- Captures all logs to `/tmp/dqn_hyperopt_logs/dqn_hyperopt_dryrun_TIMESTAMP.log` +- Displays output to console in real-time + +### 4. Validation Checks + +#### Critical Checks (must pass) +1. **Training completed without errors**: No panics, crashes, or errors +2. **Gradient clipping working**: No gradient explosions or Q-value divergence +3. **Action distribution logged**: Monitoring is active +4. **Gradient clipping enabled**: Config shows `gradient_clip_norm: Some(10.0)` + +#### Optional Checks (informational) +5. **HOLD penalty weight**: Config shows `hold_penalty_weight: 0.01` +6. **Portfolio tracking**: References to portfolio features found + +### 5. Results Summary +- **Pass**: All 4 critical checks passed → Ready for full campaign +- **Fail**: One or more critical checks failed → Review and fix issues + +--- + +## Wave 11 Validations + +The dry-run verifies that all Wave 11 bug fixes are operational: + +| Bug # | Description | Validation Method | +|-------|-------------|-------------------| +| **#1** | Gradient clipping (max_norm=10.0) | No gradient explosions in logs | +| **#2** | Portfolio tracking via PortfolioTracker | Portfolio feature references found | +| **#3** | HOLD penalty (0.01 weight) | Config shows hold_penalty_weight | +| **#4** | Close price extraction accuracy | No reward calculation errors | + +--- + +## Expected Output + +### Success Case + +``` +====================================== +Overall Result: 4/4 critical checks passed +====================================== + +🎉 DRY-RUN PASSED + +✅ All critical validations passed +✅ Wave 11 bug fixes appear operational +✅ Ready for full hyperopt campaign + +Next Steps: + 1. Review hyperopt results in logs: /tmp/dqn_hyperopt_logs/... + 2. Verify action diversity is reasonable (not 100% HOLD) + 3. Proceed with full 100-trial campaign +``` + +### Failure Case + +``` +====================================== +Overall Result: 2/4 critical checks passed +====================================== + +⚠️ DRY-RUN COMPLETED WITH WARNINGS + +❌ 2 critical checks failed +⚠️ Review issues above before proceeding + +Troubleshooting: + 1. Check logs: /tmp/dqn_hyperopt_logs/... + 2. Look for specific error messages + 3. Verify Wave 11 fixes are compiled correctly + 4. Re-run dry-run after fixes +``` + +--- + +## Logs Location + +All logs are saved to `/tmp/dqn_hyperopt_logs/` with timestamp: + +``` +/tmp/dqn_hyperopt_logs/dqn_hyperopt_dryrun_20251106_085230.log +``` + +Logs include: +- Full training output +- Hyperopt trial results +- Action distribution statistics +- Q-value monitoring +- Gradient norms +- Loss values + +--- + +## Next Steps After Success + +### Full Hyperopt Campaign + +Once dry-run passes, proceed with full campaign: + +```bash +cargo run --release -p ml --example hyperopt_dqn_demo --features cuda -- \ + --parquet-file test_data/ES_FUT_180d.parquet \ + --trials 100 \ + --epochs 50 +``` + +**Estimates**: +- Duration: 60-90 minutes +- Cost: $0.25-$0.38 (RTX A4000) +- Expected: Best hyperparameters for production DQN + +### Deploy to Runpod (Optional) + +For faster execution with more powerful GPU: + +```bash +python3 scripts/python/runpod/runpod_deploy.py \ + --gpu-type "RTX A4000" \ + --command "hyperopt_dqn_demo \ + --parquet-file /runpod-volume/test_data/ES_FUT_180d.parquet \ + --trials 100 --epochs 50" +``` + +--- + +## Troubleshooting + +### Issue: 100% HOLD Actions + +**Symptom**: Action distribution shows `HOLD=100.0%` +**Cause**: HOLD penalty not active or too weak +**Fix**: Check `hold_penalty_weight` and `movement_threshold` in config + +### Issue: Gradient Explosions + +**Symptom**: "Q-VALUE DIVERGENCE" or "gradient explosion" in logs +**Cause**: Gradient clipping disabled or too high +**Fix**: Verify `gradient_clip_norm: Some(10.0)` in DQNHyperparameters + +### Issue: Portfolio Tracking Errors + +**Symptom**: Errors about missing portfolio features +**Cause**: PortfolioTracker not initialized correctly +**Fix**: Check DQNTrainer initialization with `feature_vector_to_state` parameter + +### Issue: Training Takes Too Long + +**Symptom**: Each trial takes >5 minutes +**Cause**: CPU fallback or large dataset +**Fix**: Verify CUDA is available with `nvidia-smi` + +--- + +## Script Details + +### File Structure + +```bash +scripts/ +└── hyperopt_dqn_dryrun.sh # Main dry-run script (7.9KB) +``` + +### Script Sections + +1. **Configuration**: Trial/epoch counts, file paths +2. **Banner**: Display run parameters +3. **Pre-flight checks**: Validate environment +4. **Run hyperopt**: Execute training with logging +5. **Validation checks**: Parse logs for success/failure +6. **Results summary**: Display pass/fail status + +### Exit Codes + +- `0`: All critical checks passed (success) +- `1`: One or more critical checks failed (review needed) +- `>1`: Script error (file not found, compilation failure, etc.) + +--- + +## References + +- **CLAUDE.md**: Wave 11 DQN bug fix campaign +- **hyperopt_dqn_demo.rs**: Hyperopt implementation (`ml/examples/hyperopt_dqn_demo.rs`) +- **DQNHyperparameters**: Config struct (`ml/src/trainers/dqn.rs:48-79`) +- **TrainingMonitor**: Action distribution tracking (`ml/src/trainers/dqn.rs:118-287`) + +--- + +## Success Criteria + +✅ **PASS**: All 4 critical checks green +✅ **PASS**: Action diversity >5% per action (not 100% HOLD) +✅ **PASS**: No gradient explosions or Q-value divergence +✅ **PASS**: Loss values reasonable (<10) + +**Result**: Ready for full 100-trial hyperopt campaign + +--- + +## Wave 12-A29 Completion Summary + +- ✅ Script created: `scripts/hyperopt_dqn_dryrun.sh` +- ✅ Script is executable: `chmod +x` +- ✅ Script syntax valid: `bash -n` passed +- ✅ Hyperopt example compiled: No errors +- ✅ Documentation created: This file + +**Status**: Ready to run dry-run validation diff --git a/WAVE12_A29_DRYRUN_QUICK_REF.txt b/WAVE12_A29_DRYRUN_QUICK_REF.txt new file mode 100644 index 000000000..8252a95a1 --- /dev/null +++ b/WAVE12_A29_DRYRUN_QUICK_REF.txt @@ -0,0 +1,103 @@ +WAVE 12-A29: DQN HYPEROPT DRY-RUN SCRIPT - QUICK REFERENCE +============================================================ + +STATUS: ✅ COMPLETE (2025-11-06) + +SCRIPT LOCATION +--------------- +/home/jgrusewski/Work/foxhunt/scripts/hyperopt_dqn_dryrun.sh + +USAGE +----- +./scripts/hyperopt_dqn_dryrun.sh + +CONFIGURATION +------------- +Trials: 5 +Epochs: 10 per trial +Duration: 5-10 minutes +Cost: $0.02-$0.04 (RTX A4000) + +VALIDATION CHECKS (4 CRITICAL) +------------------------------- +1. Training completed without errors +2. Gradient clipping working (no explosions) +3. Action distribution logged +4. Gradient clipping enabled in config + +WAVE 11 BUG FIX VALIDATIONS +---------------------------- +Bug #1: Gradient clipping (max_norm=10.0) → No explosions +Bug #2: Portfolio tracking operational → Features present +Bug #3: HOLD penalty enabled (0.01) → Config shows weight +Bug #4: Close price extraction → No errors + +SUCCESS CRITERIA +---------------- +✅ 4/4 critical checks pass +✅ Action diversity >5% per action (not 100% HOLD) +✅ No gradient explosions or Q-value divergence +✅ Loss values reasonable (<10) + +LOGS LOCATION +------------- +/tmp/dqn_hyperopt_logs/dqn_hyperopt_dryrun_TIMESTAMP.log + +NEXT STEPS (AFTER DRY-RUN PASSES) +---------------------------------- +1. Review hyperopt results in logs +2. Verify action diversity is reasonable +3. Proceed with full 100-trial campaign: + + cargo run --release -p ml --example hyperopt_dqn_demo --features cuda -- \ + --parquet-file test_data/ES_FUT_180d.parquet \ + --trials 100 --epochs 50 + + Duration: 60-90 min + Cost: $0.25-$0.38 + +EXIT CODES +---------- +0: All checks passed (success) +1: One or more checks failed (review needed) +>1: Script error + +TROUBLESHOOTING +--------------- +100% HOLD: + → Check hold_penalty_weight and movement_threshold + +Gradient explosions: + → Verify gradient_clip_norm: Some(10.0) + +Portfolio errors: + → Check PortfolioTracker initialization + +Training too slow: + → Verify CUDA with nvidia-smi + +SCRIPT DETAILS +-------------- +Size: 7.9KB +Sections: + 1. Configuration (trials, epochs, paths) + 2. Pre-flight checks (files, CUDA) + 3. Hyperopt execution (with logging) + 4. Validation checks (4 critical + 2 optional) + 5. Results summary (pass/fail report) + +DELIVERABLES +------------ +✅ scripts/hyperopt_dqn_dryrun.sh (7.9KB, executable) +✅ DQN_HYPEROPT_DRYRUN_INSTRUCTIONS.md (complete guide) +✅ WAVE12_A29_DRYRUN_QUICK_REF.txt (this file) + +COMPILATION STATUS +------------------ +✅ hyperopt_dqn_demo example compiled (1 warning, non-critical) +✅ Script syntax validated (bash -n) +✅ Script is executable (chmod +x) + +READY TO RUN +------------ +./scripts/hyperopt_dqn_dryrun.sh diff --git a/scripts/hyperopt_dqn_dryrun.sh b/scripts/hyperopt_dqn_dryrun.sh new file mode 100755 index 000000000..a5d7ea6a7 --- /dev/null +++ b/scripts/hyperopt_dqn_dryrun.sh @@ -0,0 +1,253 @@ +#!/bin/bash +# +# DQN Hyperparameter Optimization Dry-Run Script +# +# Purpose: Quick validation of hyperopt setup before committing to full 100-trial campaign +# Duration: 5-10 minutes +# Cost: $0.02-$0.04 (RTX A4000) +# +# This script: +# 1. Runs 5 hyperopt trials with 10 epochs each +# 2. Captures all training logs (loss, action distribution, gradient norms) +# 3. Validates that Wave 11 bug fixes are working correctly +# 4. Provides pass/fail report with actionable next steps +# +# Usage: +# ./scripts/hyperopt_dqn_dryrun.sh + +set -e # Exit on error +set -o pipefail # Exit if any command in pipeline fails + +# ==================== CONFIGURATION ==================== + +TRIALS=5 +EPOCHS=10 +PARQUET_FILE="test_data/ES_FUT_180d.parquet" +LOG_DIR="/tmp/dqn_hyperopt_logs" +TIMESTAMP=$(date +%Y%m%d_%H%M%S) +LOG_FILE="${LOG_DIR}/dqn_hyperopt_dryrun_${TIMESTAMP}.log" + +# Create log directory +mkdir -p "$LOG_DIR" + +# ==================== BANNER ==================== + +echo "========================================" +echo "DQN Hyperopt Dry-Run" +echo "========================================" +echo "Configuration:" +echo " Trials: $TRIALS" +echo " Epochs per trial: $EPOCHS" +echo " Parquet file: $PARQUET_FILE" +echo " Log file: $LOG_FILE" +echo "" +echo "Expected duration: 5-10 minutes" +echo "Expected cost: \$0.02-\$0.04 (RTX A4000)" +echo "" +echo "Wave 11 Validations:" +echo " ✓ Gradient clipping active (max_norm=10.0)" +echo " ✓ Portfolio tracking operational" +echo " ✓ HOLD penalty enabled (0.01 weight)" +echo " ✓ Close price extraction accurate" +echo "========================================" +echo "" + +# ==================== PRE-FLIGHT CHECKS ==================== + +echo "[1/4] Pre-flight checks..." + +# Check if parquet file exists +if [ ! -f "$PARQUET_FILE" ]; then + echo "❌ ERROR: Parquet file not found: $PARQUET_FILE" + exit 1 +fi +echo " ✓ Parquet file exists: $PARQUET_FILE" + +# Check if CUDA is available +if command -v nvidia-smi &> /dev/null; then + echo " ✓ CUDA available (nvidia-smi found)" + nvidia-smi --query-gpu=name,memory.total,memory.free --format=csv,noheader | head -1 +else + echo " ⚠️ CUDA not available (will use CPU - slower)" +fi + +echo "" + +# ==================== RUN HYPEROPT ==================== + +echo "[2/4] Running hyperopt (this will take 5-10 minutes)..." +echo "" + +# Run hyperopt and capture output to both file and stdout +cargo run --release -p ml --example hyperopt_dqn_demo --features cuda -- \ + --parquet-file "$PARQUET_FILE" \ + --trials $TRIALS \ + --epochs $EPOCHS \ + 2>&1 | tee "$LOG_FILE" + +EXIT_CODE=$? + +if [ $EXIT_CODE -ne 0 ]; then + echo "" + echo "❌ HYPEROPT FAILED (exit code: $EXIT_CODE)" + echo " Check logs: $LOG_FILE" + exit $EXIT_CODE +fi + +echo "" + +# ==================== VALIDATION CHECKS ==================== + +echo "[3/4] Running validation checks..." +echo "" + +# Check for errors/panics +ERROR_COUNT=$(grep -i -c "error\|panic\|FAILED\|thread.*panicked" "$LOG_FILE" || true) +echo " Errors/panics: $ERROR_COUNT" + +# Check for gradient warnings (should be 0 with gradient clipping) +GRAD_WARNINGS=$(grep -c "Large gradient\|gradient explosion\|Q-VALUE DIVERGENCE" "$LOG_FILE" || true) +echo " Gradient warnings: $GRAD_WARNINGS" + +# Check for Q-value explosions +QVALUE_EXPLOSIONS=$(grep -c "Q-value explosion" "$LOG_FILE" || true) +echo " Q-value explosions: $QVALUE_EXPLOSIONS" + +# Check for action diversity (at least one trial should have varied actions) +ACTION_DIST_COUNT=$(grep -c "Action Distribution" "$LOG_FILE" || true) +echo " Action distribution logs: $ACTION_DIST_COUNT" + +# Extract last trial's action distribution +echo "" +echo " Last trial action distribution:" +if grep -q "Action Distribution" "$LOG_FILE"; then + grep "Action Distribution" "$LOG_FILE" | tail -1 | sed 's/^/ /' +else + echo " ⚠️ No action distribution logs found" +fi + +# Check for HOLD penalty usage (we use hold_penalty_weight in config) +HOLD_PENALTY_CONFIG=$(grep -c "hold_penalty_weight: 0.01" "$LOG_FILE" || true) +echo "" +echo " HOLD penalty weight in config: $HOLD_PENALTY_CONFIG occurrences" + +# Check for portfolio tracking (feature vector should include portfolio state) +PORTFOLIO_FEATURES=$(grep -c "portfolio" "$LOG_FILE" || true) +echo " Portfolio tracking references: $PORTFOLIO_FEATURES" + +# Check for gradient clipping activation +GRAD_CLIP_COUNT=$(grep -c "gradient_clip_norm: Some(10.0)" "$LOG_FILE" || true) +echo " Gradient clipping enabled: $GRAD_CLIP_COUNT occurrences" + +echo "" + +# ==================== RESULTS SUMMARY ==================== + +echo "[4/4] Validation summary" +echo "========================================" + +# Count passes and failures +CHECKS_PASSED=0 +CHECKS_FAILED=0 + +echo "" +echo "Critical Checks:" +echo "----------------" + +# Check 1: Training completed without errors +if [ $ERROR_COUNT -eq 0 ]; then + echo "✅ Training completed without errors" + CHECKS_PASSED=$((CHECKS_PASSED + 1)) +else + echo "❌ Training had $ERROR_COUNT errors/panics" + CHECKS_FAILED=$((CHECKS_FAILED + 1)) +fi + +# Check 2: Gradient clipping working (no explosions) +if [ $GRAD_WARNINGS -eq 0 ] && [ $QVALUE_EXPLOSIONS -eq 0 ]; then + echo "✅ Gradient clipping working (no explosions/warnings)" + CHECKS_PASSED=$((CHECKS_PASSED + 1)) +else + echo "❌ Gradient issues detected ($GRAD_WARNINGS warnings, $QVALUE_EXPLOSIONS explosions)" + CHECKS_FAILED=$((CHECKS_FAILED + 1)) +fi + +# Check 3: Action distribution logged +if [ $ACTION_DIST_COUNT -gt 0 ]; then + echo "✅ Action distribution logged ($ACTION_DIST_COUNT times)" + CHECKS_PASSED=$((CHECKS_PASSED + 1)) +else + echo "❌ No action distribution logs found" + CHECKS_FAILED=$((CHECKS_FAILED + 1)) +fi + +# Check 4: Gradient clipping config present +if [ $GRAD_CLIP_COUNT -gt 0 ]; then + echo "✅ Gradient clipping enabled in config" + CHECKS_PASSED=$((CHECKS_PASSED + 1)) +else + echo "⚠️ Gradient clipping config not found in logs" + CHECKS_FAILED=$((CHECKS_FAILED + 1)) +fi + +echo "" +echo "Optional Checks:" +echo "----------------" + +# Check 5: HOLD penalty weight configured +if [ $HOLD_PENALTY_CONFIG -gt 0 ]; then + echo "✅ HOLD penalty weight configured (0.01)" +else + echo "⚠️ HOLD penalty weight not found in logs (may still be working)" +fi + +# Check 6: Portfolio tracking present +if [ $PORTFOLIO_FEATURES -gt 0 ]; then + echo "✅ Portfolio tracking references found" +else + echo "⚠️ Portfolio tracking references not found (may still be working)" +fi + +echo "" +echo "========================================" +echo "Overall Result: $CHECKS_PASSED/4 critical checks passed" +echo "========================================" +echo "" + +# Final verdict +if [ $CHECKS_FAILED -eq 0 ]; then + echo "🎉 DRY-RUN PASSED" + echo "" + echo "✅ All critical validations passed" + echo "✅ Wave 11 bug fixes appear operational" + echo "✅ Ready for full hyperopt campaign" + echo "" + echo "Next Steps:" + echo " 1. Review hyperopt results in logs: $LOG_FILE" + echo " 2. Verify action diversity is reasonable (not 100% HOLD)" + echo " 3. Proceed with full 100-trial campaign:" + echo " cargo run --release -p ml --example hyperopt_dqn_demo --features cuda -- \\" + echo " --parquet-file $PARQUET_FILE --trials 100 --epochs 50" + echo "" + echo "Estimated full campaign:" + echo " Duration: 60-90 minutes" + echo " Cost: \$0.25-\$0.38 (RTX A4000)" + exit 0 +else + echo "⚠️ DRY-RUN COMPLETED WITH WARNINGS" + echo "" + echo "❌ $CHECKS_FAILED critical checks failed" + echo "⚠️ Review issues above before proceeding" + echo "" + echo "Troubleshooting:" + echo " 1. Check logs: $LOG_FILE" + echo " 2. Look for specific error messages" + echo " 3. Verify Wave 11 fixes are compiled correctly" + echo " 4. Re-run dry-run after fixes" + echo "" + echo "Common Issues:" + echo " - Gradient explosions: Check gradient_clip_norm in DQNHyperparameters" + echo " - 100% HOLD: Check hold_penalty_weight and movement_threshold" + echo " - Portfolio errors: Check PortfolioTracker initialization" + exit 1 +fi