# RunPod Deployment Quick Reference

## Prerequisites

```bash
# 1. Activate virtual environment
source .venv/bin/activate

# 2. Install dependencies
pip install -r ml/python/requirements.txt

# 3. Configure credentials in .env.runpod
# Required:
#   RUNPOD_API_KEY=<your_api_key>
#   RUNPOD_VOLUME_ID=<your_volume_id>
# Optional (for monitoring):
#   RUNPOD_S3_BUCKET=<bucket_name>
#   RUNPOD_S3_ACCESS_KEY=<access_key>
#   RUNPOD_S3_SECRET_KEY=<secret_key>
```

## Common Commands

### Basic Deployment
```bash
# Auto-select cheapest GPU
python3 scripts/runpod_deploy.py

# Specific GPU
python3 scripts/runpod_deploy.py --gpu-type "RTX A4000"

# Dry run (no deployment)
python3 scripts/runpod_deploy.py --dry-run
```

### With Monitoring
```bash
# Monitor training logs
python3 scripts/runpod_deploy.py --monitor --timeout 2h

# Full automation (deploy + monitor + auto-stop)
python3 scripts/runpod_deploy.py \
  --gpu-type "RTX A4000" \
  --monitor \
  --auto-stop \
  --timeout 2h
```

### Custom Training
```bash
# Custom epochs
python3 scripts/runpod_deploy.py \
  --command "--parquet-file /runpod-volume/test_data/ES_FUT_180d.parquet --epochs 100"

# Different model
python3 scripts/runpod_deploy.py \
  --command "/runpod-volume/binaries/train_mamba2_parquet --epochs 50"
```

## Flag Reference

| Flag | Description | Default |
|------|-------------|---------|
| `--gpu-type` | Preferred GPU (e.g., "RTX 4090") | Auto-select cheapest |
| `--image` | Docker image | jgrusewski/foxhunt:latest |
| `--command` | Training command | TFT with ES_FUT_180d.parquet |
| `--container-disk` | Disk size in GB | 50 |
| `--dry-run` | Show plan without deploying | False |
| `--monitor` | Stream logs from S3 | False |
| `--auto-stop` | Auto-terminate on completion | False |
| `--timeout` | Max monitoring time (30m, 2h) | 120m |
| `--monitor-interval` | Log polling interval (seconds) | 10 |

## Cost Estimates

| GPU | VRAM | Price/hr | 100 epochs* |
|-----|------|----------|-------------|
| RTX A4000 | 16GB | $0.25 | ~$0.50 |
| RTX A5000 | 24GB | $0.35 | ~$0.70 |
| Tesla V100 | 16GB | $0.10 | ~$0.20 |
| A100 80GB | 80GB | $1.20 | ~$2.40 |

*Estimated for TFT training (~2h)

## Troubleshooting

### Module Import Error
```
WARNING: Could not import foxhunt_runpod module
```
**Fix**: Install dependencies
```bash
pip install -r ml/python/requirements.txt
```

### S3 Monitoring Disabled
```
WARNING: S3 credentials not configured
```
**Fix**: Add to .env.runpod
```
RUNPOD_S3_BUCKET=se3zdnb5o4
RUNPOD_S3_ACCESS_KEY=<key>
RUNPOD_S3_SECRET_KEY=<secret>
```

### GPU Not Available
```
GPU not available in EUR-IS datacenters
```
**Fix**: Try again (availability changes frequently) or select different GPU

## Manual Pod Management

```bash
# View pods
curl -H "Authorization: Bearer $RUNPOD_API_KEY" \
  https://rest.runpod.io/v1/pods

# Stop pod
curl -X POST -H "Authorization: Bearer $RUNPOD_API_KEY" \
  https://rest.runpod.io/v1/pods/<pod_id>/stop

# Terminate pod
curl -X POST -H "Authorization: Bearer $RUNPOD_API_KEY" \
  https://rest.runpod.io/v1/pods/<pod_id>/terminate
```

## Python API Usage

```python
from foxhunt_runpod import RunPodClient, S3LogMonitor

# Deploy
client = RunPodClient(api_key="...", volume_id="...")
gpus = client.get_available_gpus(min_vram=16)
pod = client.deploy_pod(gpu_id=gpus[0]['id'], image="jgrusewski/foxhunt:latest")

# Monitor
monitor = S3LogMonitor(bucket_name="...", aws_access_key="...", aws_secret_key="...")
monitor.stream_logs(pod_id=pod['id'], timeout=7200)

# Cleanup
client.terminate_pod(pod['id'])
```

## Best Practices

1. **Always use --dry-run first** to verify configuration
2. **Set --timeout** to prevent runaway costs
3. **Use --monitor --auto-stop** for unattended training
4. **Check GPU pricing** before deploying (shown in output)
5. **Terminate pods** when done to avoid charges

## Example Workflow

```bash
# 1. Dry run to check configuration
python3 scripts/runpod_deploy.py --dry-run

# 2. Deploy with monitoring
python3 scripts/runpod_deploy.py \
  --gpu-type "RTX A4000" \
  --monitor \
  --auto-stop \
  --timeout 2h \
  --command "--parquet-file /runpod-volume/test_data/ES_FUT_180d.parquet --epochs 100"

# 3. Script will:
#    - Find available RTX A4000 in EUR-IS-1
#    - Deploy pod with volume mounted
#    - Stream logs in real-time
#    - Auto-terminate when complete
#    - Show final cost estimate
```

## Support

- **Module docs**: `ml/python/README.md`
- **Integration guide**: `RUNPOD_MODULE_INTEGRATION.md`
- **Script help**: `python3 scripts/runpod_deploy.py --help`