# Foxhunt RunPod Integration

Python module for managing RunPod GPU pods with enhanced features:
- Pod deployment with GPU selection
- Real-time S3 log streaming
- Auto-termination on completion
- Health checks and status monitoring

## Installation

```bash
cd ml/python
pip install -r requirements.txt
```

## Usage

### Command-line (via runpod_deploy.py)

```bash
# Basic deployment
python3 scripts/runpod_deploy.py --gpu-type "RTX A4000"

# With monitoring
python3 scripts/runpod_deploy.py --monitor --timeout 120m

# Full automation
python3 scripts/runpod_deploy.py --monitor --auto-stop --timeout 2h
```

### Python API

```python
from foxhunt_runpod import RunPodClient, S3LogMonitor

# Deploy a pod
client = RunPodClient(
    api_key="your_api_key",
    volume_id="your_volume_id"
)

gpus = client.get_available_gpus(min_vram=16)
pod_data = client.deploy_pod(
    gpu_id=gpus[0]['id'],
    image="jgrusewski/foxhunt:latest",
    command="--epochs 100"
)

# Monitor logs
monitor = S3LogMonitor(
    bucket_name="your_bucket",
    aws_access_key="your_key",
    aws_secret_key="your_secret"
)

monitor.stream_logs(
    pod_id=pod_data['id'],
    interval=10,
    timeout=7200
)

# Terminate when done
client.terminate_pod(pod_data['id'])
```

## Environment Variables

Required in `.env.runpod`:
- `RUNPOD_API_KEY` - RunPod API key
- `RUNPOD_VOLUME_ID` - Network volume ID
- `RUNPOD_CONTAINER_REGISTRY_AUTH_ID` - Docker registry auth (optional)

Optional for S3 monitoring:
- `RUNPOD_S3_BUCKET` - S3 bucket name
- `RUNPOD_S3_ACCESS_KEY` - AWS access key
- `RUNPOD_S3_SECRET_KEY` - AWS secret key

## Architecture

```
scripts/runpod_deploy.py (CLI)
    ↓
ml/python/foxhunt_runpod/
    ├── client.py       - RunPodClient (pod management)
    ├── s3_monitor.py   - S3LogMonitor (log streaming)
    └── __init__.py     - Module exports
```

## Features

### RunPodClient
- Query available GPUs with pricing
- Deploy pods with datacenter filtering
- Stop/terminate pods
- Get pod status

### S3LogMonitor
- Stream logs in real-time from S3
- Detect training completion
- Configurable polling interval
- Timeout support

## Backward Compatibility

The script automatically falls back to legacy implementation if the module cannot be imported. All existing command-line flags are preserved.