2df1ea92e1
BREAKING CHANGES: - Removed orphaned dqn.rs monolithic trainer (4,975 lines) - Removed orphaned dqn_ensemble.rs module (816 lines) - Removed orphaned tft.rs and tft_complete_int8_integration_test.rs - TFT trainer split into modular directory structure DQN Module Refactoring: - Split trainers/dqn.rs into modular structure (config.rs, statistics.rs, trainer.rs) - Fixed hyperopt 39D search space (continuous params only) - Boolean flags (use_dueling, use_double_dqn, use_per, use_noisy_nets) are now FIXED architectural decisions - use_distributional defaults to false (Candle BUG #36 - scatter_add gradient issues) Clean Module Structure: - ml/src/trainers/dqn/ directory with proper mod.rs exports - ml/src/trainers/tft/ directory with config.rs, types.rs, model.rs, trainer.rs, tests.rs - All P0 features validated: TD-error clamping, batch diversity, LR scheduler, priority staleness Documentation: - Added comprehensive docs in docs/codebase-cleanup/ - ADR-001 for DQN refactoring decisions - Rainbow DQN component matrix and quick reference guides Build Status: Compiles with zero errors 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
60 lines
1.8 KiB
Bash
Executable File
60 lines
1.8 KiB
Bash
Executable File
#!/bin/bash
|
|
# MAMBA2 Hyperopt RunPod Deployment Script
|
|
# Deploys MAMBA2 hyperparameter optimization to RunPod GPU
|
|
|
|
set -e
|
|
|
|
# Activate virtual environment
|
|
source .venv/bin/activate
|
|
|
|
# Set PYTHONPATH to include custom runpod module
|
|
export PYTHONPATH=/home/jgrusewski/Work/foxhunt:$PYTHONPATH
|
|
|
|
# Configuration
|
|
GPU_TYPE="RTX A4000"
|
|
POD_NAME="mamba2-hyperopt"
|
|
IMAGE="jgrusewski/foxhunt:latest"
|
|
TRIALS=50
|
|
EPOCHS=50
|
|
TIMEOUT="120m"
|
|
|
|
# MAMBA2 hyperopt command for RunPod
|
|
# Note: Binary is wrapped by entrypoint-self-terminate.sh
|
|
COMMAND="hyperopt_mamba2_demo \
|
|
--parquet-file /runpod-volume/test_data/ES_FUT_180d.parquet \
|
|
--base-dir /runpod-volume/ml_training \
|
|
--trials ${TRIALS} \
|
|
--epochs ${EPOCHS} \
|
|
--batch-size-max 96 \
|
|
--early-stopping-patience 5"
|
|
|
|
echo "======================================================================"
|
|
echo "MAMBA2 Hyperopt RunPod Deployment"
|
|
echo "======================================================================"
|
|
echo "GPU Type: ${GPU_TYPE}"
|
|
echo "Docker Image: ${IMAGE}"
|
|
echo "Trials: ${TRIALS}"
|
|
echo "Epochs per Trial: ${EPOCHS}"
|
|
echo "Max Monitoring: ${TIMEOUT}"
|
|
echo "Command: ${COMMAND}"
|
|
echo "======================================================================"
|
|
echo ""
|
|
|
|
# Deploy pod with monitoring and auto-stop
|
|
python3 scripts/runpod_deploy.py \
|
|
--gpu-type "${GPU_TYPE}" \
|
|
--image "${IMAGE}" \
|
|
--command "${COMMAND}" \
|
|
--monitor \
|
|
--auto-stop \
|
|
--timeout "${TIMEOUT}" \
|
|
--monitor-interval 15
|
|
|
|
echo ""
|
|
echo "======================================================================"
|
|
echo "Deployment Complete!"
|
|
echo "======================================================================"
|
|
echo "Results will be saved to: /runpod-volume/ml_training/"
|
|
echo "Check S3 bucket for outputs: s3://se3zdnb5o4/ml_training/"
|
|
echo "======================================================================"
|