d746008e1f
- Created entrypoint-self-terminate.sh wrapper script - Updates entrypoint-generic.sh to be called by wrapper - Modified Dockerfile.runpod to use self-terminate entrypoint - Adds automatic pod termination via runpodctl after training completes - Prevents infinite restart loops and wasted GPU credits - Saves ~96% cost per training run ($4.59 per run) Implements pod self-termination using RUNPOD_POD_ID environment variable. Training exits with code 0 → runpodctl remove pod → immediate shutdown. Co-Authored-By: Claude <noreply@anthropic.com>
2.2 KiB
2.2 KiB
RunPod GPU Scanner
Purpose
Query RunPod API to show available SECURE cloud GPUs in the EUR-IS region with ≥16GB VRAM.
Requirements
- Python 3.7+
- python-dotenv (
pip3 install python-dotenv) - requests (
pip3 install requests) - Valid RunPod API key in
/home/jgrusewski/Work/foxhunt/.env.runpod
Usage
Run the script:
# From foxhunt root directory
./scripts/scan_gpus.py
# Or with python3
python3 scripts/scan_gpus.py
Output
The script displays:
- GPU name (e.g., RTX 4090, A100 SXM)
- VRAM capacity (GB)
- Price per hour (USD)
- Availability (number of pods)
Results are sorted by price (cheapest first).
Filtering Criteria
- Memory ≥16GB VRAM
- Secure Cloud availability > 0
- Valid pricing information
- EUR-IS region
Example Output
======================================================================
RUNPOD SECURE CLOUD GPUs (≥16GB VRAM) - EUR-IS REGION
======================================================================
GPU Name VRAM Price/hr Available
----------------------------------------------------------------------
RTX A5000 24GB $0.160 True pods
RTX A4000 16GB $0.170 True pods
RTX 4090 24GB $0.340 True pods
...
======================================================================
Total GPUs found: 24
Top Recommendations for Foxhunt ML Training
Based on the current scan results (October 2025):
Budget Option (FP32 Models)
- RTX 4090: $0.34/hr, 24GB VRAM
- Best value for FP32 training (TFT-FP32 fits in ~500MB)
- Ideal for initial deployment and baseline metrics
Professional Option (QAT Models)
- RTX A6000: $0.33/hr, 48GB VRAM
- Best for QAT training (requires gradient checkpointing)
- Can run multiple models concurrently
Enterprise Option (Multi-Model Inference)
- A100 PCIe/SXM: $1.19-1.39/hr, 80GB VRAM
- Production-grade for ensemble inference
- Supports 4+ models with headroom
Notes
- Prices and availability fluctuate based on demand
- Run this script regularly to find the best deals
- Consider spot instances for training (not shown in this script)