jgrusewski 5380bd9186 infra: split GPU pool into training (H100) and inference (L4), add auto-taint ...

- Split single H100 GPU pool into two purpose-specific pools:
  - gpu-training: H100-1-80G (€2.73/hr) for 10-model ensemble training
  - gpu-inference: L4-1-24G (€0.75/hr) for cost-effective trading inference
- Add GPU taint controller DaemonSet that auto-taints new GPU nodes
  with nvidia.com/gpu=true:NoSchedule to prevent non-GPU workloads
- Fix service log directory permissions with emptyDir volumes
  (observability init fails creating /app/logs as non-root user)
- Increase postgres max_connections from 25 to 100
  (7 services each requesting connection pools exhausted the limit)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

2026-02-24 17:39:56 +01:00

2.5 KiB

Raw Blame History

View Raw