2025-10-29 10:57:04 - INFO - Restarting script with venv Python...
2025-10-29 10:57:06 - INFO - ✓ Loaded environment from /home/jgrusewski/Work/foxhunt/.env.runpod
2025-10-29 10:57:06 - INFO - ✓ Environment variables validated
2025-10-29 10:57:06 - INFO - 
ℹ Skipping binary upload check (--skip-upload)
2025-10-29 10:57:06 - INFO - 
2025-10-29 10:57:06 - INFO - ======================================================================
2025-10-29 10:57:06 - INFO - STEP 2: GPU AVAILABILITY SCAN
2025-10-29 10:57:06 - INFO - ======================================================================
2025-10-29 10:57:06 - INFO - 🔍 Querying available GPU types...
2025-10-29 10:57:06 - INFO -    ⏭ MI300X: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ A100 PCIe: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ A100 SXM: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ A40: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ B200: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ RTX 3070: Skipped (VRAM 8GB < 16GB)
2025-10-29 10:57:06 - INFO -    ⏭ RTX 3080: Skipped (VRAM 10GB < 16GB)
2025-10-29 10:57:06 - INFO -    ⏭ RTX 3080 Ti: Skipped (VRAM 12GB < 16GB)
2025-10-29 10:57:06 - INFO -    ⏭ RTX 3090: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ RTX 3090 Ti: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ RTX 4070 Ti: Skipped (VRAM 12GB < 16GB)
2025-10-29 10:57:06 - INFO -    ⏭ RTX 4080: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ RTX 4080 SUPER: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ RTX 4090: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ RTX 5080: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ RTX 5090: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ H100 SXM: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ H100 NVL: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ H100 PCIe: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ H200 SXM: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ NVIDIA H200 NVL: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ L4: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ L40: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ L40S: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ RTX 2000 Ada: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ RTX 4000 Ada: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ RTX 4000 Ada SFF: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ RTX 5000 Ada: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ RTX 6000 Ada: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ RTX A2000: Skipped (VRAM 6GB < 16GB)
2025-10-29 10:57:06 - INFO -    ⏭ RTX A4000: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ RTX A4500: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ RTX A5000: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ RTX A6000: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ RTX PRO 6000 MaxQ: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ RTX PRO 6000: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ RTX PRO 6000 WK: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ V100 FHHL: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ Tesla V100: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ V100 SXM2: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ V100 SXM2 32GB: Skipped (0 instances available)
2025-10-29 10:57:06 - INFO -    ⏭ unknown: Skipped (VRAM 0GB < 16GB)
2025-10-29 10:57:06 - WARNING - ⚠ No GPU types found with ≥16GB VRAM
2025-10-29 10:57:06 - ERROR - 
✗ No GPUs available with ≥16GB VRAM in SECURE or COMMUNITY cloud