foxhunt

gitadmin/foxhunt

Fork 0

Commit Graph

Author	SHA1	Message	Date
jgrusewski	68b6aa8313	feat(infra): migrate container registry from SCW to internal GitLab Full migration off Scaleway Container Registry to internal GitLab registry backed by MinIO S3. All 4 images (ci-builder, ci-builder-cpu, foxhunt-runtime, foxhunt-training-runtime) rebuilt in internal registry. Registry & images: - All image refs → gitlab-registry.foxhunt.svc.cluster.local:5000/root/foxhunt/ - imagePullSecrets: scw-registry → gitlab-registry - Kaniko build template: two-step DAG (git-clone → kaniko-build) with shared PVC - Kaniko layer cache enabled at root/foxhunt/cache - AWS_ACCESS_KEY_ID: $SCW_ACCESS_KEY → $MINIO_ACCESS_KEY in .gitlab-ci.yml Network policies: - ci-pipeline: add HTTP/80, registry/5000, webservice/8181 egress rules DNS & Tailscale proxy cleanup: - Remove ci, prometheus, monitor DNS records (no longer exposed) - Rename s3 → minio DNS record - Remove Argo UI, Prometheus, monitor nginx server blocks - Remove argo-htpasswd volume mount - Tailscale proxy nodeSelector: infra → platform Terraform cleanup: - Delete infra/modules/registry/ (SCW CR namespace) - Delete infra/modules/object-storage/ (SCW S3 buckets) - Delete infra/modules/secrets/ (SCW secrets) - Delete corresponding live configs - TF state backend: S3 → GitLab HTTP Argo workflows: - Add events/ (GitLab push eventsource + ci-pipeline sensor) - ci-pipeline + training templates: SCW → internal registry - Delete obsolete compile-training-template.yaml Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-05 13:52:24 +01:00
jgrusewski	1ad77b6427	fix(ci): request nvidia.com/gpu in GitLab runner pod specs The GPU runners used runtime_class_name=nvidia for CUDA access but didn't request nvidia.com/gpu as a K8s resource, allowing multiple training pods to share a GPU without K8s awareness. Add pod_spec patches to request GPU resources properly so K8s enforces mutual exclusion between GitLab CI and Argo training jobs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-04 22:58:40 +01:00
jgrusewski	1931d93d66	feat(ci): add 2×H100 GPU pool for multi-GPU training Pool ci-training-h100x2 (H100-2-80G, 160GB VRAM total). MultiGpuConfig::detect() in ml crate auto-enables data parallelism. Tags: kapsule,h100x2. Autoscale 0→1. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-04 11:32:26 +01:00

Author

SHA1

Message

Date

jgrusewski

68b6aa8313

feat(infra): migrate container registry from SCW to internal GitLab

Full migration off Scaleway Container Registry to internal GitLab
registry backed by MinIO S3. All 4 images (ci-builder, ci-builder-cpu,
foxhunt-runtime, foxhunt-training-runtime) rebuilt in internal registry.

Registry & images:
- All image refs → gitlab-registry.foxhunt.svc.cluster.local:5000/root/foxhunt/
- imagePullSecrets: scw-registry → gitlab-registry
- Kaniko build template: two-step DAG (git-clone → kaniko-build) with shared PVC
- Kaniko layer cache enabled at root/foxhunt/cache
- AWS_ACCESS_KEY_ID: $SCW_ACCESS_KEY → $MINIO_ACCESS_KEY in .gitlab-ci.yml

Network policies:
- ci-pipeline: add HTTP/80, registry/5000, webservice/8181 egress rules

DNS & Tailscale proxy cleanup:
- Remove ci, prometheus, monitor DNS records (no longer exposed)
- Rename s3 → minio DNS record
- Remove Argo UI, Prometheus, monitor nginx server blocks
- Remove argo-htpasswd volume mount
- Tailscale proxy nodeSelector: infra → platform

Terraform cleanup:
- Delete infra/modules/registry/ (SCW CR namespace)
- Delete infra/modules/object-storage/ (SCW S3 buckets)
- Delete infra/modules/secrets/ (SCW secrets)
- Delete corresponding live configs
- TF state backend: S3 → GitLab HTTP

Argo workflows:
- Add events/ (GitLab push eventsource + ci-pipeline sensor)
- ci-pipeline + training templates: SCW → internal registry
- Delete obsolete compile-training-template.yaml

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

2026-03-05 13:52:24 +01:00

jgrusewski

1ad77b6427

fix(ci): request nvidia.com/gpu in GitLab runner pod specs

The GPU runners used runtime_class_name=nvidia for CUDA access but
didn't request nvidia.com/gpu as a K8s resource, allowing multiple
training pods to share a GPU without K8s awareness. Add pod_spec
patches to request GPU resources properly so K8s enforces mutual
exclusion between GitLab CI and Argo training jobs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

2026-03-04 22:58:40 +01:00

jgrusewski

1931d93d66

feat(ci): add 2×H100 GPU pool for multi-GPU training

Pool ci-training-h100x2 (H100-2-80G, 160GB VRAM total).
MultiGpuConfig::detect() in ml crate auto-enables data parallelism.
Tags: kapsule,h100x2. Autoscale 0→1.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

2026-03-04 11:32:26 +01:00

3 Commits