68b6aa8313
Full migration off Scaleway Container Registry to internal GitLab registry backed by MinIO S3. All 4 images (ci-builder, ci-builder-cpu, foxhunt-runtime, foxhunt-training-runtime) rebuilt in internal registry. Registry & images: - All image refs → gitlab-registry.foxhunt.svc.cluster.local:5000/root/foxhunt/ - imagePullSecrets: scw-registry → gitlab-registry - Kaniko build template: two-step DAG (git-clone → kaniko-build) with shared PVC - Kaniko layer cache enabled at root/foxhunt/cache - AWS_ACCESS_KEY_ID: $SCW_ACCESS_KEY → $MINIO_ACCESS_KEY in .gitlab-ci.yml Network policies: - ci-pipeline: add HTTP/80, registry/5000, webservice/8181 egress rules DNS & Tailscale proxy cleanup: - Remove ci, prometheus, monitor DNS records (no longer exposed) - Rename s3 → minio DNS record - Remove Argo UI, Prometheus, monitor nginx server blocks - Remove argo-htpasswd volume mount - Tailscale proxy nodeSelector: infra → platform Terraform cleanup: - Delete infra/modules/registry/ (SCW CR namespace) - Delete infra/modules/object-storage/ (SCW S3 buckets) - Delete infra/modules/secrets/ (SCW secrets) - Delete corresponding live configs - TF state backend: S3 → GitLab HTTP Argo workflows: - Add events/ (GitLab push eventsource + ci-pipeline sensor) - ci-pipeline + training templates: SCW → internal registry - Delete obsolete compile-training-template.yaml Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
61 lines
1.7 KiB
YAML
61 lines
1.7 KiB
YAML
# DaemonSet image pre-puller — keeps training images cached on GPU nodes
|
|
# Runs on ci-training (L40S) pool so training jobs skip the pull.
|
|
# Init containers pull :latest tags, then the main container sleeps forever.
|
|
---
|
|
apiVersion: apps/v1
|
|
kind: DaemonSet
|
|
metadata:
|
|
name: image-prepuller
|
|
namespace: foxhunt
|
|
labels:
|
|
app: image-prepuller
|
|
app.kubernetes.io/part-of: foxhunt
|
|
spec:
|
|
selector:
|
|
matchLabels:
|
|
app: image-prepuller
|
|
template:
|
|
metadata:
|
|
labels:
|
|
app: image-prepuller
|
|
spec:
|
|
affinity:
|
|
nodeAffinity:
|
|
requiredDuringSchedulingIgnoredDuringExecution:
|
|
nodeSelectorTerms:
|
|
- matchExpressions:
|
|
- key: k8s.scaleway.com/pool-name
|
|
operator: In
|
|
values:
|
|
- ci-training
|
|
tolerations:
|
|
- key: nvidia.com/gpu
|
|
operator: Exists
|
|
effect: NoSchedule
|
|
- key: node.cilium.io/agent-not-ready
|
|
operator: Exists
|
|
effect: NoSchedule
|
|
imagePullSecrets:
|
|
- name: gitlab-registry
|
|
initContainers:
|
|
- name: pull-training-runtime
|
|
image: gitlab-registry.foxhunt.svc.cluster.local:5000/root/foxhunt/foxhunt-training-runtime:latest
|
|
command: ["echo", "foxhunt-training-runtime image pulled"]
|
|
resources:
|
|
requests:
|
|
cpu: 10m
|
|
memory: 16Mi
|
|
limits:
|
|
cpu: 10m
|
|
memory: 16Mi
|
|
containers:
|
|
- name: pause
|
|
image: registry.k8s.io/pause:3.10
|
|
resources:
|
|
requests:
|
|
cpu: 10m
|
|
memory: 16Mi
|
|
limits:
|
|
cpu: 10m
|
|
memory: 16Mi
|