## RunPod Deployment Script

**Script**: `runpod_deploy.py`

Automated deployment script for RunPod GPU pods in EUR-IS region (SECURE cloud).

### Features
- Scans available GPUs with ≥16GB VRAM
- Auto-selects best value GPU (RTX 4090 preferred, then cheapest)
- Supports custom GPU selection
- Dry-run mode for testing
- Automatic network volume attachment

### Requirements
```bash
pip install requests python-dotenv
```

### Configuration
Create `.env.runpod` with:
```
RUNPOD_API_KEY=your_api_key
RUNPOD_VOLUME_ID=your_volume_id
```

### Usage Examples

```bash
# Auto-select best value GPU (dry run)
./scripts/runpod_deploy.py --dry-run

# Deploy with default settings (RTX 4090 preferred)
./scripts/runpod_deploy.py

# Deploy with specific GPU
./scripts/runpod_deploy.py --gpu-type "RTX 3090"

# Custom image and larger disk
./scripts/runpod_deploy.py \
  --image runpod/pytorch:2.1.0-py3.10-cuda11.8.0-devel \
  --container-disk 100

# With custom command
./scripts/runpod_deploy.py --command "jupyter lab --allow-root"
```

### Default Configuration
- **Cloud Type**: SECURE (no spot interruptions)
- **Region**: EUR-IS (Iceland - low latency to Europe)
- **Image**: `runpod/pytorch:2.4.0-py3.11-cuda12.4.1-devel-ubuntu22.04`
- **Container Disk**: 50GB
- **Network Volume**: Attached from `.env.runpod`
- **Ports**: 8888/http (Jupyter)

### GPU Selection Logic
1. If `--gpu-type` specified and available → use it
2. Else if RTX 4090 available → use it (best value)
3. Else → use cheapest available GPU

### Output
```
✅ POD DEPLOYED SUCCESSFULLY
======================================================================
Pod ID:          abc123-xyz789
GPU:             RTX 4090 (24GB)
Cost:            $0.340/hr
Image:           runpod/pytorch:2.4.0
Status:          RUNNING
======================================================================

📝 NEXT STEPS:
1. Wait 2-3 minutes for pod to initialize
2. Access Jupyter at: https://abc123-8888.proxy.runpod.net
3. SSH access: ssh root@abc123.ssh.runpod.io
4. Monitor pod: https://www.runpod.io/console/pods
```

### Cost Warning
The script will display hourly costs. Remember to stop pods when done to avoid unnecessary charges.

---

## Local CI/CD Pipeline Simulator

**Script**: `local_ci_pipeline.sh`

Simulates GitLab CI/CD pipeline locally before deployment. Tests Docker image builds and deployments in a safe, local environment.

### Features
- 3-stage pipeline: Build → Test → Push
- GLIBC 2.35 validation (Ubuntu 22.04)
- CUDA 12.4.1 + cuDNN 9 library checks
- Entrypoint script validation
- Docker Hub push readiness
- Color-coded output with timing
- Dry-run mode for testing
- Exit on first failure (CI/CD behavior)

### Requirements
```bash
# Docker installed and running
docker info

# Docker Hub authentication (for push stage)
docker login
```

### Usage Examples

```bash
# Full pipeline (Build + Test + Push)
./scripts/local_ci_pipeline.sh

# Test build only (skip push)
./scripts/local_ci_pipeline.sh --skip-push

# Dry-run (show commands without executing)
./scripts/local_ci_pipeline.sh --dry-run

# Verbose output for debugging
./scripts/local_ci_pipeline.sh --verbose --skip-push
```

### Pipeline Stages

#### Stage 0: Pre-Flight Checks (🔍)
- Docker daemon running
- Docker BuildKit available
- Docker Hub authentication
- Dockerfile exists
- Git repository status

#### Stage 1: Build (🔨)
- Build Docker image with CUDA 12.4.1 + cuDNN 9
- Verify image size (~4.8 GB)
- Duration: ~2-3 minutes

#### Stage 2: Test (🧪)
- GLIBC 2.35 validation
- CUDA libraries (libcuda, libcurand, libcublas, libcudnn)
- nvidia-smi availability (optional)
- Binary GLIBC dependencies
- Entrypoint script validation
- Duration: ~10-20 seconds

#### Stage 3: Push (🚀)
- Push image to Docker Hub
- Verify authentication
- Warn about PRIVATE repository
- Duration: ~1-5 minutes

### Output Example

```
========================================
🚀 LOCAL CI/CD PIPELINE SIMULATOR
========================================

ℹ Simulating GitLab CI/CD pipeline locally
ℹ Image: jgrusewski/foxhunt:latest

========================================
🔍 STAGE 0: PRE-FLIGHT CHECKS
========================================
✓ All required commands available
✓ Docker daemon running
✓ Docker Hub authenticated
⏱ Pre-flight checks completed in 0m 3s

========================================
🔨 STAGE 1: BUILD
========================================
✓ Docker image built successfully: 4.80 GB
⏱ Build completed in 2m 34s

========================================
🧪 STAGE 2: TEST
========================================
✓ GLIBC 2.35 validated
✓ CUDA libraries validated
⏱ Test completed in 0m 18s

========================================
🚀 STAGE 3: PUSH
========================================
✓ Image pushed successfully
⏱ Push completed in 3m 12s

========================================
✅ PIPELINE COMPLETE
========================================
✓ Total pipeline time: 6m 7s
ℹ GitLab CI/CD readiness: ✅
```

### Troubleshooting

**Error: Docker daemon not running**
```bash
sudo systemctl start docker
docker info
```

**Error: Docker Hub authentication failed**
```bash
docker login
# Enter credentials for jgrusewski account
```

**Error: GLIBC version mismatch**
```bash
# Expected: GLIBC 2.35 (Ubuntu 22.04)
docker run --rm jgrusewski/foxhunt:latest ldd --version
```

### Documentation
- Full guide: `/LOCAL_CI_PIPELINE_GUIDE.md`
- Dockerfile: `/Dockerfile.runpod`
- Total time: 4-9 minutes (vs. 10-15 min on GitLab)
- Cost: $0 (vs. GitLab CI/CD minutes)