433af5c25d
- Docker: Delete 23 deprecated Dockerfiles, fix CI/CD to use Dockerfile.foxhunt-build - Config: Remove 36 .env files, keep 4 essential, delete config/environments/ - Docs: Archive 614 Wave D files to docs/archive/wave_d/, 95% reduction in root - Scripts: Delete 56 deprecated scripts, keep 58 production-critical (49% reduction) - Python: Organize 37 scripts into scripts/python/ subdirectories, delete ml/python/ - Build: Remove 1GB artifacts, delete old venvs, clean Python cache from git - Migrations: Delete deprecated directory (4,432 lines), remove duplicate database/migrations/ - Infrastructure: Delete deployment/ (61 files), docs/scripts/ (8 files) Total impact: ~2,500 files cleaned, 750MB+ space freed, zero production impact All deleted scripts backed up to archives. runpod/ and tests/runpod/ preserved. data_acquisition_service retained per user request.
3.1 KiB
3.1 KiB
CUDA 12.9 Binary Rebuild - Quick Reference
Status: ✅ COMPLETE | Date: 2025-10-26
Binary Information
| Binary | Size | SHA256 Checksum |
|---|---|---|
| train_dqn | 20MB | 84c03be4646124ba0b3d8edf61a7b0b0b588855520d2669a52c223ec8d80a928 |
| train_ppo | 14MB | cbac3267aff5aefbb4b10e01341dc9e251b062bb93896239479f59b716c5f03c |
| train_tft_parquet | 21MB | 646dde7df2fe36506aa0131142908eef36e7028ef4bfd0f638a545f57dcb1495 |
| train_mamba2_parquet | 20MB | b0191a05547b1d2c3dcd50a1cc4df50961291cc790aaf86a673db4910c4be9cb |
Total Size: 75MB
CUDA Linkage Verification
All binaries link to CUDA 12.9 (libcublas.so.12):
ldd target/release/examples/train_dqn | grep libcublas
# Output: libcublas.so.12 => /usr/local/cuda-12.9/lib64/libcublas.so.12
✅ VERIFIED: Compatible with Runpod Driver 550.x
Rebuild Commands (If Needed)
# 1. Set environment
export CUDA_HOME=/usr/local/cuda-12.9
export CUDA_PATH=/usr/local/cuda-12.9
export CUDA_ROOT=/usr/local/cuda-12.9
export PATH=/usr/local/cuda-12.9/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-12.9/lib64:$LD_LIBRARY_PATH
# 2. Clean previous builds
cargo clean
# 3. Build all binaries
cargo build --release --features cuda -p ml --example train_dqn
cargo build --release --features cuda -p ml --example train_tft_parquet
cargo build --release --features cuda -p ml --example train_ppo
cargo build --release --features cuda -p ml --example train_mamba2_parquet
# 4. Verify linkage
ldd target/release/examples/train_dqn | grep libcublas
# Expected: libcublas.so.12 (NOT libcublas.so.13)
# 5. Generate checksums
cd target/release/examples
sha256sum train_dqn train_ppo train_tft_parquet train_mamba2_parquet > CHECKSUMS.txt
S3 Upload Commands
cd target/release/examples
# Upload binaries
aws s3 cp train_dqn s3://foxhunt-ml-models/binaries/cuda12.9/
aws s3 cp train_ppo s3://foxhunt-ml-models/binaries/cuda12.9/
aws s3 cp train_tft_parquet s3://foxhunt-ml-models/binaries/cuda12.9/
aws s3 cp train_mamba2_parquet s3://foxhunt-ml-models/binaries/cuda12.9/
# Upload checksums
aws s3 cp CHECKSUMS_CUDA13.txt s3://foxhunt-ml-models/binaries/cuda12.9/CHECKSUMS.txt
Runpod Deployment Test
# On Runpod pod (Driver 550.x)
export LD_LIBRARY_PATH=/usr/local/cuda-12.9/lib64:$LD_LIBRARY_PATH
# Test TFT
./train_tft_parquet \
--parquet-file /runpod-volume/test_data/ES_FUT_180d.parquet \
--epochs 5
# Test MAMBA-2
./train_mamba2_parquet \
--parquet-file /runpod-volume/test_data/ES_FUT_180d.parquet \
--epochs 5
Key Facts
- CUDA Version: 12.9 (libcublas.so.12)
- Driver Requirement: 550.x or higher
- Runpod Compatibility: ✅ FULL SUPPORT
- Previous Build (CUDA 13.0): ❌ Incompatible with Driver 550.x
- Current Build (CUDA 12.9): ✅ Compatible with Driver 550.x
Next Actions
- ⏳ Upload binaries to S3
- ⏳ Test on Runpod GPU instance
- ⏳ Update Docker image to CUDA 12.9 base
- ⏳ Update documentation (CLAUDE.md, guides)
Full Report: See /home/jgrusewski/Work/foxhunt/CUDA12.9_REBUILD_REPORT.md