Files
foxhunt/infra/k8s/argo/build-ci-image-template.yaml
jgrusewski 68b6aa8313 feat(infra): migrate container registry from SCW to internal GitLab
Full migration off Scaleway Container Registry to internal GitLab
registry backed by MinIO S3. All 4 images (ci-builder, ci-builder-cpu,
foxhunt-runtime, foxhunt-training-runtime) rebuilt in internal registry.

Registry & images:
- All image refs → gitlab-registry.foxhunt.svc.cluster.local:5000/root/foxhunt/
- imagePullSecrets: scw-registry → gitlab-registry
- Kaniko build template: two-step DAG (git-clone → kaniko-build) with shared PVC
- Kaniko layer cache enabled at root/foxhunt/cache
- AWS_ACCESS_KEY_ID: $SCW_ACCESS_KEY → $MINIO_ACCESS_KEY in .gitlab-ci.yml

Network policies:
- ci-pipeline: add HTTP/80, registry/5000, webservice/8181 egress rules

DNS & Tailscale proxy cleanup:
- Remove ci, prometheus, monitor DNS records (no longer exposed)
- Rename s3 → minio DNS record
- Remove Argo UI, Prometheus, monitor nginx server blocks
- Remove argo-htpasswd volume mount
- Tailscale proxy nodeSelector: infra → platform

Terraform cleanup:
- Delete infra/modules/registry/ (SCW CR namespace)
- Delete infra/modules/object-storage/ (SCW S3 buckets)
- Delete infra/modules/secrets/ (SCW secrets)
- Delete corresponding live configs
- TF state backend: S3 → GitLab HTTP

Argo workflows:
- Add events/ (GitLab push eventsource + ci-pipeline sensor)
- ci-pipeline + training templates: SCW → internal registry
- Delete obsolete compile-training-template.yaml

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 13:52:24 +01:00

167 lines
5.1 KiB
YAML

apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
name: build-ci-image
namespace: foxhunt
labels:
app.kubernetes.io/name: build-ci-image
app.kubernetes.io/part-of: foxhunt
spec:
activeDeadlineSeconds: 3600
serviceAccountName: argo-workflow
entrypoint: build
podMetadata:
labels:
app.kubernetes.io/component: ci-pipeline
app.kubernetes.io/part-of: foxhunt
ttlStrategy:
secondsAfterCompletion: 3600
arguments:
parameters:
- name: commit-sha
value: HEAD
- name: dockerfile
value: Dockerfile.ci-builder
- name: image-name
value: ci-builder
volumes:
- name: git-ssh-key
secret:
secretName: argo-git-ssh-key
defaultMode: 256
- name: registry-auth
secret:
secretName: gitlab-registry
items:
- key: .dockerconfigjson
path: config.json
volumeClaimTemplates:
- metadata:
name: workspace
spec:
accessModes: ["ReadWriteOnce"]
storageClassName: scw-bssd
resources:
requests:
storage: 5Gi
templates:
- name: build
inputs:
parameters:
- name: commit-sha
value: "{{workflow.parameters.commit-sha}}"
- name: dockerfile
value: "{{workflow.parameters.dockerfile}}"
- name: image-name
value: "{{workflow.parameters.image-name}}"
dag:
tasks:
- name: git-clone
template: git-clone
arguments:
parameters:
- name: commit-sha
value: "{{inputs.parameters.commit-sha}}"
- name: kaniko-build
template: kaniko-build
dependencies: [git-clone]
arguments:
parameters:
- name: dockerfile
value: "{{inputs.parameters.dockerfile}}"
- name: image-name
value: "{{inputs.parameters.image-name}}"
# ── git-clone: fetch source code into shared PVC ──
- name: git-clone
nodeSelector:
k8s.scaleway.com/pool-name: ci-compile-cpu
tolerations:
- key: node.cilium.io/agent-not-ready
operator: Exists
effect: NoSchedule
inputs:
parameters:
- name: commit-sha
container:
image: alpine/git:latest
command: ["/bin/sh", "-c"]
resources:
requests:
cpu: 200m
memory: 256Mi
limits:
cpu: "1"
memory: 512Mi
volumeMounts:
- name: git-ssh-key
mountPath: /etc/git-ssh
readOnly: true
- name: workspace
mountPath: /workspace
args:
- |
set -e
mkdir -p ~/.ssh
cp /etc/git-ssh/ssh-privatekey ~/.ssh/id_ed25519
chmod 600 ~/.ssh/id_ed25519
printf 'Host *\n StrictHostKeyChecking no\n UserKnownHostsFile /dev/null\n' > ~/.ssh/config
chmod 600 ~/.ssh/config
SHA="{{inputs.parameters.commit-sha}}"
REPO="ssh://git@gitlab-gitlab-shell.foxhunt.svc.cluster.local:2222/root/foxhunt.git"
if [ "$SHA" = "HEAD" ]; then
git clone --depth=1 "$REPO" /workspace/src
else
git clone --depth=1 "$REPO" /workspace/src
cd /workspace/src
git fetch --depth=1 origin "$SHA"
git checkout "$SHA"
fi
echo "Checked out $(cd /workspace/src && git rev-parse --short HEAD)"
# ── kaniko-build: build and push image from cloned source ──
- name: kaniko-build
nodeSelector:
k8s.scaleway.com/pool-name: ci-compile-cpu
tolerations:
- key: node.cilium.io/agent-not-ready
operator: Exists
effect: NoSchedule
inputs:
parameters:
- name: dockerfile
- name: image-name
container:
image: gcr.io/kaniko-project/executor:debug
command: ["/busybox/sh", "-c"]
env:
- name: DOCKER_CONFIG
value: /kaniko/.docker
resources:
requests:
cpu: "4"
memory: 8Gi
limits:
cpu: "8"
memory: 16Gi
volumeMounts:
- name: registry-auth
mountPath: /kaniko/.docker
readOnly: true
- name: workspace
mountPath: /workspace
args:
- |
/kaniko/executor \
--context=/workspace/src \
--dockerfile=/workspace/src/infra/docker/{{inputs.parameters.dockerfile}} \
--destination=gitlab-registry.foxhunt.svc.cluster.local:5000/root/foxhunt/{{inputs.parameters.image-name}}:latest \
--insecure-registry=gitlab-registry.foxhunt.svc.cluster.local:5000 \
--skip-tls-verify-registry=gitlab-registry.foxhunt.svc.cluster.local:5000 \
--cache=true \
--cache-repo=gitlab-registry.foxhunt.svc.cluster.local:5000/root/foxhunt/cache \
--snapshot-mode=redo