3853988af7
- Fixed PSO budget calculation bug in ml/src/hyperopt/optimizer.rs - Root cause: Division by n_particles in sequential execution - Now correctly calculates max_iters = remaining_trials (no division) - Result: 50 trials complete instead of 23 (100% vs 46%) - Added comprehensive DQN hyperopt results analysis - 39/50 trials analyzed across 2 RunPod deployments - Best hyperparameters identified: LR 4.89e-5 (ultra-low) - Created DQN_HYPEROPT_RESULTS_SUMMARY.md with expert validation - GitLab CI/CD pipeline operational (48 lines fixed) - Fixed YAML syntax errors (unquoted colons) - All 7 jobs validated and working - Warning cleanup complete (136 → 0 warnings) - Removed 143 lines dead code - Fixed visibility, unused imports, Debug traits - Archived Wave D reports to docs/archive/ - 8 early stopping reports moved - Root directory cleaned up 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
83 lines
2.9 KiB
Plaintext
83 lines
2.9 KiB
Plaintext
################################################################################
|
|
# PPO PRODUCTION TRAINING - OPTIMIZED HYPERPARAMETERS
|
|
################################################################################
|
|
# Source: PPO Hyperopt Results (Pod 08w5n1ewf1ln8w)
|
|
# Date: 2025-11-01
|
|
# Best Trial: #1 (Objective: 2.4023)
|
|
################################################################################
|
|
|
|
# LOCAL TRAINING (RTX 3050 Ti)
|
|
# Duration: ~30-60 seconds
|
|
# Cost: FREE
|
|
################################################################################
|
|
|
|
cargo run -p ml --example train_ppo --release --features cuda -- \
|
|
--policy-lr 1e-6 \
|
|
--value-lr 0.001 \
|
|
--clip-epsilon 0.1126 \
|
|
--value-loss-coeff 0.5 \
|
|
--entropy-coeff 0.006142 \
|
|
--episodes 10000 \
|
|
--parquet-file test_data/ES_FUT_180d.parquet
|
|
|
|
|
|
# RUNPOD TRAINING (RTX A4000 - Optional)
|
|
# Duration: ~45 seconds
|
|
# Cost: $0.01
|
|
################################################################################
|
|
|
|
# 1. Deploy pod
|
|
python3 scripts/python/runpod/runpod_deploy.py \
|
|
--gpu-type "RTX A4000" \
|
|
--image jgrusewski/foxhunt:latest \
|
|
--name ppo_production
|
|
|
|
# 2. SSH into pod and run:
|
|
cd /workspace
|
|
train_ppo \
|
|
--policy-lr 1e-6 \
|
|
--value-lr 0.001 \
|
|
--clip-epsilon 0.1126 \
|
|
--value-loss-coeff 0.5 \
|
|
--entropy-coeff 0.006142 \
|
|
--episodes 10000 \
|
|
--parquet-file /runpod-volume/test_data/ES_FUT_180d.parquet \
|
|
--output-dir /runpod-volume/ml_training/ppo_production
|
|
|
|
# 3. Download checkpoint
|
|
aws s3 sync s3://se3zdnb5o4/ml_training/ppo_production/checkpoints/ \
|
|
./models/ppo/ \
|
|
--profile runpod \
|
|
--endpoint-url https://s3api-eur-is-1.runpod.io
|
|
|
|
|
|
################################################################################
|
|
# EXPECTED OUTPUT
|
|
################################################################################
|
|
# - Model checkpoint: models/ppo/ppo_final.safetensors
|
|
# - Training metrics: models/ppo/training_metrics.json
|
|
# - Validation loss: ~2.40 (value loss)
|
|
# - Policy loss: ~0.37
|
|
# - Total training time: ~30-60 seconds
|
|
|
|
|
|
################################################################################
|
|
# VALIDATION COMMAND
|
|
################################################################################
|
|
|
|
cargo run -p ml --example evaluate_ppo --release --features cuda -- \
|
|
--model-path models/ppo/ppo_final.safetensors \
|
|
--test-data test_data/ES_FUT_unseen.parquet \
|
|
--episodes 1000
|
|
|
|
|
|
################################################################################
|
|
# NOTES
|
|
################################################################################
|
|
# - Ultra-low policy LR (1e-6) is intentional - PPO is sensitive to policy updates
|
|
# - High value LR (0.001) allows fast value function learning
|
|
# - Conservative clipping (0.1126) provides stable training
|
|
# - Low entropy (0.006142) focuses on exploitation over exploration
|
|
# - These parameters achieved 99.99% better objective than worst trial
|
|
################################################################################
|