2df1ea92e1
BREAKING CHANGES: - Removed orphaned dqn.rs monolithic trainer (4,975 lines) - Removed orphaned dqn_ensemble.rs module (816 lines) - Removed orphaned tft.rs and tft_complete_int8_integration_test.rs - TFT trainer split into modular directory structure DQN Module Refactoring: - Split trainers/dqn.rs into modular structure (config.rs, statistics.rs, trainer.rs) - Fixed hyperopt 39D search space (continuous params only) - Boolean flags (use_dueling, use_double_dqn, use_per, use_noisy_nets) are now FIXED architectural decisions - use_distributional defaults to false (Candle BUG #36 - scatter_add gradient issues) Clean Module Structure: - ml/src/trainers/dqn/ directory with proper mod.rs exports - ml/src/trainers/tft/ directory with config.rs, types.rs, model.rs, trainer.rs, tests.rs - All P0 features validated: TD-error clamping, batch diversity, LR scheduler, priority staleness Documentation: - Added comprehensive docs in docs/codebase-cleanup/ - ADR-001 for DQN refactoring decisions - Rainbow DQN component matrix and quick reference guides Build Status: Compiles with zero errors 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
62 lines
2.2 KiB
Bash
Executable File
62 lines
2.2 KiB
Bash
Executable File
#!/bin/bash
|
|
set -e
|
|
|
|
echo "========================================="
|
|
echo "DQN Hyperopt Deployment (CORRECTED OBJECTIVE)"
|
|
echo "========================================="
|
|
echo ""
|
|
|
|
# Configuration
|
|
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
|
|
OUTPUT_DIR="dqn_hyperopt_corrected_${TIMESTAMP}"
|
|
|
|
echo "Configuration:"
|
|
echo " Objective: Episode rewards (CORRECTED from validation loss)"
|
|
echo " Trials: 50"
|
|
echo " Epochs per trial: 100"
|
|
echo " GPU: RTX A4000 ($0.25/hr)"
|
|
echo " Expected duration: 12-25 min"
|
|
echo " Expected cost: $0.05-$0.10"
|
|
echo " Output: /runpod-volume/ml_training/${OUTPUT_DIR}"
|
|
echo ""
|
|
|
|
# Verify Docker image is up-to-date
|
|
echo "Verifying Docker image..."
|
|
IMAGE_DATE=$(docker images jgrusewski/foxhunt-hyperopt:latest --format "{{.CreatedAt}}" | head -1)
|
|
echo " Image timestamp: ${IMAGE_DATE}"
|
|
echo " Expected: Nov 1, 2025 23:18+ (after fix)"
|
|
echo ""
|
|
|
|
# Deploy DQN hyperopt with CORRECTED objective
|
|
echo "Deploying DQN hyperopt pod..."
|
|
python3 scripts/python/runpod/runpod_deploy.py \
|
|
--gpu-type "RTX A4000" \
|
|
--image "jgrusewski/foxhunt-hyperopt:latest" \
|
|
--command "hyperopt_dqn_demo --parquet-file /runpod-volume/test_data/ES_FUT_180d.parquet --trials 50 --epochs 100 --base-dir /runpod-volume/ml_training/${OUTPUT_DIR}"
|
|
|
|
echo ""
|
|
echo "✅ DQN hyperopt deployment initiated (CORRECTED OBJECTIVE)"
|
|
echo "Monitor logs: python3 scripts/python/runpod/monitor_logs.py <pod_id>"
|
|
echo ""
|
|
echo "CRITICAL FIX APPLIED:"
|
|
echo " Previous objective: val_loss (WRONG - rewarded tiny batches)"
|
|
echo " New objective: -avg_episode_reward (CORRECT)"
|
|
echo ""
|
|
echo "Expected Results:"
|
|
echo " Batch size: Should vary widely (not stuck at 32-43)"
|
|
echo " Learning rate: Should optimize for actual learning"
|
|
echo " Episode rewards: Should maximize trading returns"
|
|
echo " Q-values: Should show proper value estimation"
|
|
echo ""
|
|
echo "Previous Issue (FIXED):"
|
|
echo " Tiny batch sizes (32-43) prevented learning"
|
|
echo " Q-values stayed near zero (noisy gradients)"
|
|
echo " Validation loss was artificially low (misleading)"
|
|
echo ""
|
|
echo "Next Steps:"
|
|
echo " 1. Monitor pod logs for trial progress"
|
|
echo " 2. Check best hyperparameters after completion"
|
|
echo " 3. Compare batch sizes to previous run (32-43)"
|
|
echo " 4. Verify Q-values and episode rewards improve"
|
|
echo ""
|