6da9d262db
CRITICAL P0 FIXES (Validated - Loss 0.87 → 0.07): - Add sigmoid activation to inference and training (ml/src/mamba/mod.rs:798, 1538) - Fix config.total_decay_steps (was hardcoded 10000) (ml/src/mamba/mod.rs:2271) - Update d_state: 16→64, 32→64 (Mamba-2 spec) (ml/src/mamba/mod.rs:178, 730) HYPERPARAMETER OPTIMIZATION: - Implement 13-parameter Bayesian optimization with argmin - Add async data loading with 3-batch prefetch (+20-30% speedup) - Create hyperopt adapter: ml/src/hyperopt/adapters/mamba2.rs - Add example: ml/examples/hyperopt_mamba2_demo.rs VALIDATION: - Local test: Loss 0.07 vs 0.87 (12× improvement) - Val loss: 0.04-0.14 vs 1.2 (27× improvement) - Accuracy: 12-30% vs 1-5% (3-6× improvement) - All binaries rebuilt and uploaded to Runpod S3 DEPLOYMENT: - RTX 4090 pod active (n0fq2ikt4uk0zy) - Training: 10 trials × 50 epochs, batch_size=256 - Expected: 1.3 days, $10.41 cost Fixes #P0-sigmoid #P0-decay-steps #hyperopt-mamba2
RunPod Deployment Script
Script: runpod_deploy.py
Automated deployment script for RunPod GPU pods in EUR-IS region (SECURE cloud).
Features
- Scans available GPUs with ≥16GB VRAM
- Auto-selects best value GPU (RTX 4090 preferred, then cheapest)
- Supports custom GPU selection
- Dry-run mode for testing
- Automatic network volume attachment
Requirements
pip install requests python-dotenv
Configuration
Create .env.runpod with:
RUNPOD_API_KEY=your_api_key
RUNPOD_VOLUME_ID=your_volume_id
Usage Examples
# Auto-select best value GPU (dry run)
./scripts/runpod_deploy.py --dry-run
# Deploy with default settings (RTX 4090 preferred)
./scripts/runpod_deploy.py
# Deploy with specific GPU
./scripts/runpod_deploy.py --gpu-type "RTX 3090"
# Custom image and larger disk
./scripts/runpod_deploy.py \
--image runpod/pytorch:2.1.0-py3.10-cuda11.8.0-devel \
--container-disk 100
# With custom command
./scripts/runpod_deploy.py --command "jupyter lab --allow-root"
Default Configuration
- Cloud Type: SECURE (no spot interruptions)
- Region: EUR-IS (Iceland - low latency to Europe)
- Image:
runpod/pytorch:2.4.0-py3.11-cuda12.4.1-devel-ubuntu22.04 - Container Disk: 50GB
- Network Volume: Attached from
.env.runpod - Ports: 8888/http (Jupyter)
GPU Selection Logic
- If
--gpu-typespecified and available → use it - Else if RTX 4090 available → use it (best value)
- Else → use cheapest available GPU
Output
✅ POD DEPLOYED SUCCESSFULLY
======================================================================
Pod ID: abc123-xyz789
GPU: RTX 4090 (24GB)
Cost: $0.340/hr
Image: runpod/pytorch:2.4.0
Status: RUNNING
======================================================================
📝 NEXT STEPS:
1. Wait 2-3 minutes for pod to initialize
2. Access Jupyter at: https://abc123-8888.proxy.runpod.net
3. SSH access: ssh root@abc123.ssh.runpod.io
4. Monitor pod: https://www.runpod.io/console/pods
Cost Warning
The script will display hourly costs. Remember to stop pods when done to avoid unnecessary charges.