feat(ml): Complete hyperopt infrastructure + documentation

Changes:
- CLAUDE.md: Update OOM fix validation status
- Add comprehensive documentation (30+ markdown reports)
- LSTM encoder varmap bug fix (tft/lstm_encoder.rs:290)
- Quantized LSTM layer matching fix (tft/quantized_lstm.rs)
- Hyperopt paths module (ml/src/hyperopt/paths.rs)
- Training path tests for all adapters (DQN, MAMBA-2, PPO, TFT)
- Checkpoint integrity tests
- Script cleanup: Remove 29 obsolete deployment scripts
- Archive old scripts to scripts/archive/
- New deployment utilities: check_gpu_availability.py, monitor_hyperopt.sh

Validation:
- OOM fixes validated: 5/5 trials successful (pod b6kc3mc5lbjiro)
- Batch-size-max 256 tested successfully
- All hyperopt adapters working correctly

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2025-10-29 19:52:21 +01:00
parent 59cce96d9d
commit e61e8f54da
111 changed files with 15450 additions and 838 deletions

View File

@@ -0,0 +1,30 @@
#!/usr/bin/env python3
import os
import sys
from pathlib import Path
from dotenv import load_dotenv
import runpod
# Load environment
env_path = Path.cwd().parent / '.env.runpod'
load_dotenv(env_path)
runpod.api_key = os.getenv('RUNPOD_API_KEY')
# Get GPU types and check specific GPUs
gpu_types = runpod.get_gpus()
target_gpus = ['RTX 4090', 'RTX 4000 Ada', 'RTX 5090']
for target in target_gpus:
for gpu in gpu_types:
if target in gpu.get('displayName', ''):
print(f"\n{target} details:")
print(f" secureCloud: {gpu.get('secureCloud', 0)}")
print(f" communityCloud: {gpu.get('communityCloud', 0)}")
print(f" memoryInGb: {gpu.get('memoryInGb', 0)}")
price_info = gpu.get('lowestPrice', {})
if price_info:
print(f" uninterruptablePrice: {price_info.get('uninterruptablePrice', 'N/A')}")
else:
print(f" price: N/A")
break