gitadmin/fxhnt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'ci/fxhnt-pipeline': fxhnt CI/CD pipeline

Browse Source 
Argo Events sensor (gitadmin/fxhnt master push) -> fxhnt-cockpit
WorkflowTemplate (foxhunt): git-diff picks build-vs-deploy, Kaniko builds
only on dep/Dockerfile changes, deploy rolls dashboard (scale 0->1) +
dagster. Both modes validated end-to-end live. Sensor lives in the bizworx
GitOps repo (bizworx-ops/infrastructure); RBAC + workflow + build/deploy
pods are foxhunt-owned (project boundary). Review-clean: injection-hardened
(env+hex-validate), least-privilege RBAC (no rolebinding-create), dagster
on :latest so dep-bumps propagate.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Jeroen Grusewski
2026-07-21 11:40:34 +00:00
parent c98b8d4b49 05f6d624c0
commit 7da51863a1
8 changed files with 465 additions and 223 deletions
Download Patch File Download Diff File Expand all files Collapse all files

135
infra/argo/cockpit-build-deploy.yaml
View File

@@ -1,135 +0,0 @@
# Argo WorkflowTemplate — build + deploy the fxhnt cockpit, fully in-cluster.
#
# Reuses cluster primitives: the `argo-workflow` ServiceAccount, the `argo-git-ssh-key` deploy key (clone
# the fxhnt repo over in-cluster Gitea SSH, gitea-sshd:22), and the `scw-registry` dockerconfig (push to
# the external Scaleway Container Registry rg.fr-par.scw.cloud/bizworx). Build runs in-cluster.
# (Phase 2B: migrated off the in-cluster GitLab shell + registry.)
#
# Submit: argo submit -n foxhunt --from=wftmpl/fxhnt-cockpit -p commit-sha=<sha> --watch
# (or use scripts/argo-deploy-cockpit.sh)
apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
name: fxhnt-cockpit
namespace: foxhunt
labels: { app.kubernetes.io/name: fxhnt-cockpit, app.kubernetes.io/part-of: foxhunt }
spec:
serviceAccountName: argo-workflow
entrypoint: pipeline
archiveLogs: true # archive build/deploy logs to MinIO (argo-logs bucket). Reachable now that the
# pods carry app.kubernetes.io/part-of: argo-workflows (MinIO ingress netpol allow).
activeDeadlineSeconds: 1800
ttlStrategy: { secondsAfterCompletion: 3600 }
podGC: { strategy: OnPodCompletion }
arguments:
parameters:
- { name: commit-sha, value: HEAD }
- { name: mode, value: build } # "build" = kaniko rebuild + deploy; "deploy" = skip build, just apply + rollout restart (git-sync re-pulls code). The sensor routes src-only pushes to "deploy".
templates:
- name: pipeline
dag:
tasks:
# Skip the ~5min kaniko build for code-only pushes (mode=deploy) — git-sync re-pulls src/ on the
# rollout restart, so only dep/Dockerfile changes need a new image (mode=build).
- { name: build, template: kaniko-build, when: "{{workflow.parameters.mode}} == build" }
- { name: deploy, template: kubectl-deploy, depends: "build.Succeeded || build.Skipped" }
# --- build: Kaniko builds infra/docker/fxhnt.Dockerfile -> internal registry ---
- name: kaniko-build
metadata:
labels: { app.kubernetes.io/part-of: argo-workflows } # MinIO ingress allows part-of in {foxhunt,foxhunt-ci,argo-workflows} → log archival works. NOT 'foxhunt' (that adds argo-base-egress which blocks registry:5000/git:2222).
nodeSelector: { k8s.scaleway.com/pool-name: ci-compile-cpu, topology.kubernetes.io/zone: fr-par-2 }
tolerations: [{ effect: NoSchedule, key: node.cilium.io/agent-not-ready, operator: Exists }]
initContainers:
- name: git-clone
image: alpine/git:latest
command: [/bin/sh, -c]
args:
- |
set -ex
mkdir -p /root/.ssh
cp /etc/git-ssh/ssh-privatekey /root/.ssh/id_ed25519
chmod 600 /root/.ssh/id_ed25519
printf 'Host *\n StrictHostKeyChecking no\n UserKnownHostsFile /dev/null\n' > /root/.ssh/config
git clone --no-checkout --filter=blob:none \
"ssh://git@gitea-sshd.foxhunt.svc.cluster.local:22/gitadmin/fxhnt.git" /workspace/src
cd /workspace/src && git checkout "{{workflow.parameters.commit-sha}}"
echo "checked out $(git rev-parse --short HEAD)"
volumeMounts:
- { name: git-ssh-key, mountPath: /etc/git-ssh, readOnly: true }
- { name: workspace, mountPath: /workspace }
container:
image: gcr.io/kaniko-project/executor:debug
command: [/busybox/sh, -c]
args:
- |
/kaniko/executor \
--context=/workspace/src \
--dockerfile=/workspace/src/infra/docker/fxhnt.Dockerfile \
--destination=rg.fr-par.scw.cloud/bizworx/fxhnt-cockpit:latest \
--cache=true \
--cache-repo=rg.fr-par.scw.cloud/bizworx/cache \
--snapshot-mode=redo
env:
- { name: DOCKER_CONFIG, value: /kaniko/.docker }
resources: { requests: { cpu: "2", memory: 4Gi }, limits: { cpu: "6", memory: 12Gi } }
volumeMounts:
- { name: registry-auth, mountPath: /kaniko/.docker, readOnly: true }
- { name: workspace, mountPath: /workspace }
volumes:
- { name: workspace, emptyDir: {} }
- { name: git-ssh-key, secret: { secretName: argo-git-ssh-key, defaultMode: 256 } }
- name: registry-auth
secret:
secretName: scw-registry
items: [{ key: .dockerconfigjson, path: config.json }]
# --- deploy: apply manifests from the repo + roll the dashboard to the fresh image ---
- name: kubectl-deploy
metadata:
labels: { app.kubernetes.io/part-of: argo-workflows } # MinIO ingress allow → log archival works (see kaniko-build note)
nodeSelector: { k8s.scaleway.com/pool-name: platform, topology.kubernetes.io/zone: fr-par-2 }
serviceAccountName: argo-workflow
initContainers:
- name: git-clone
image: alpine/git:latest
command: [/bin/sh, -c]
args:
- |
set -ex
mkdir -p /root/.ssh
cp /etc/git-ssh/ssh-privatekey /root/.ssh/id_ed25519
chmod 600 /root/.ssh/id_ed25519
printf 'Host *\n StrictHostKeyChecking no\n UserKnownHostsFile /dev/null\n' > /root/.ssh/config
git clone --no-checkout --filter=blob:none \
"ssh://git@gitea-sshd.foxhunt.svc.cluster.local:22/gitadmin/fxhnt.git" /workspace/src
cd /workspace/src && git checkout "{{workflow.parameters.commit-sha}}"
volumeMounts:
- { name: git-ssh-key, mountPath: /etc/git-ssh, readOnly: true }
- { name: workspace, mountPath: /workspace }
container:
image: alpine/k8s:1.31.0 # Alpine-based (has /bin/sh + kubectl). NOT registry.k8s.io/kubectl — that's distroless (no shell) and the deploy runs a shell script. bitnami/kubectl was removed from Docker Hub (2025).
command: [/bin/sh, -c]
args:
- |
set -e
cd /workspace/src
kubectl apply -f infra/k8s/services/fxhnt-dashboard.yaml
kubectl apply -f infra/k8s/orchestration/dagster.yaml
kubectl apply -f infra/k8s/network-policies/fxhnt-cockpit.yaml
# Dashboard: scale 0 -> 1, NOT `rollout restart`. Its tailscale sidecar serves dashboard.fxhnt.ai;
# a rolling restart briefly overlaps two tailscale sessions on the same identity -> session race ->
# the external proxy link goes stale and the cockpit reads "down" post-deploy. Scaling to 0 cleanly
# terminates the session before a fresh, non-overlapping one comes up; the git-sync initContainer
# re-pulls src the same way, so code still updates. See reference_fxhnt_dashboard_proxy.
kubectl -n foxhunt scale deploy/fxhnt-dashboard --replicas=0
kubectl -n foxhunt wait --for=delete pod -l app.kubernetes.io/name=fxhnt-dashboard --timeout=90s || true
kubectl -n foxhunt scale deploy/fxhnt-dashboard --replicas=1
kubectl -n foxhunt rollout restart deploy/dagster # dagster has no external-proxy race
kubectl -n foxhunt rollout status deploy/fxhnt-dashboard --timeout=180s
kubectl -n foxhunt rollout status deploy/dagster --timeout=300s
volumeMounts:
- { name: workspace, mountPath: /workspace }
volumes:
- { name: workspace, emptyDir: {} }
- { name: git-ssh-key, secret: { secretName: argo-git-ssh-key, defaultMode: 256 } }

57
infra/argo/fxhnt-build-sensor.yaml Normal file
View File

@@ -0,0 +1,57 @@
# Argo Events Sensor — hears the shared Gitea webhook for pushes to gitadmin/fxhnt master, and submits the
# fxhnt-cockpit WorkflowTemplate (ns foxhunt). Structurally identical to build-msp-portal (single dependency,
# single trigger) — the build-vs-deploy decision is made INSIDE the workflow via `git diff` (decide-mode step in
# fxhnt-cockpit-workflowtemplate.yaml), not via a sensor-side path filter. See
# docs/superpowers/plans/2026-07-21-fxhnt-cicd-pipeline.md Task 3 design note: the Sensor CRD here has an open
# schema, so trigger-level path-filter negation can't be reliably validated — keeping the sensor simple is the
# robust choice.
apiVersion: argoproj.io/v1alpha1
kind: Sensor
metadata:
name: build-fxhnt
namespace: argo-events
labels: { app.kubernetes.io/instance: build-pipeline }
spec:
dependencies:
- name: push-to-master
eventSourceName: gitea-webhook
eventName: push
filters:
dataLogicalOperator: and
data:
- path: body.ref
type: string
comparator: "="
value: ["refs/heads/master"]
- path: body.pusher.username
type: string
comparator: "!="
value: ["argocd-image-updater"]
template:
serviceAccountName: argo-events-sa
triggers:
- template:
name: submit-build
argoWorkflow:
operation: submit
source:
resource:
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: build-fxhnt-
namespace: foxhunt
spec:
workflowTemplateRef:
name: fxhnt-cockpit
arguments:
parameters:
- name: commit-sha
value: ""
- name: mode
value: auto
parameters:
- src:
dependencyName: push-to-master
dataKey: body.after
dest: spec.arguments.parameters.0.value

124
infra/argo/fxhnt-ci-rbac.yaml Normal file
View File

@@ -0,0 +1,124 @@
# RBAC for the fxhnt CI pipeline — all fxhnt-owned, in the foxhunt namespace (project boundary).
#
# Fills the two gaps the first real workflow run surfaced (2026-07-21):
# 1. The fxhnt-cockpit WorkflowTemplate references SA `argo-workflow`, which existed in `argo` (where msp
# builds) but NOT in `foxhunt` (where fxhnt's project boundary places its build/deploy pods).
# 2. The build-fxhnt Sensor (argo-events-sa) submits Workflows into `foxhunt`; the bizworx `workflow-submitter`
# Role only granted that in `argo`, so a cross-namespace submit into foxhunt was forbidden.
#
# Least privilege: the workflow SA gets exactly what Argo needs to run pods + what the kubectl-deploy step does
# (apply/scale/annotate/rollout-status the two daemon Deployments and their Services/NetworkPolicies), nothing more.
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: argo-workflow
namespace: foxhunt
labels: { app.kubernetes.io/part-of: fxhnt-ci }
---
# Argo needs the workflow SA to write task results (the minimum the workflow-controller requires per step).
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: fxhnt-argo-workflow
namespace: foxhunt
labels: { app.kubernetes.io/part-of: fxhnt-ci }
rules:
- apiGroups: ["argoproj.io"]
resources: ["workflowtaskresults"]
verbs: ["create", "patch"]
---
# The kubectl-deploy step's exact actions on the foxhunt daemons — scoped to what it runs, no wildcards.
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: fxhnt-ci-deployer
namespace: foxhunt
labels: { app.kubernetes.io/part-of: fxhnt-ci }
rules:
# apply + scale + annotate + rollout-status the daemon Deployments (dashboard, dagster).
- apiGroups: ["apps"]
resources: ["deployments"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
- apiGroups: ["apps"]
resources: ["deployments/scale"]
verbs: ["get", "update", "patch"]
- apiGroups: ["apps"]
resources: ["replicasets"]
verbs: ["get", "list", "watch"] # rollout status reads the RS
# apply the Services the manifests contain, and `wait --for=delete pod` + rollout status read pods.
- apiGroups: [""]
resources: ["services"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "list", "watch"]
# the fxhnt-dashboard + dagster manifests also bundle ConfigMaps (the tailscale sidecar serve-config) —
# kubectl apply of the whole file must be able to read/apply them. NOTE (CI review M1): the tailscale
# ServiceAccount + Role + RoleBinding objects were MOVED out of these app manifests into
# infra/k8s/rbac/tailscale-sidecars.yaml (applied once, out-of-band, by a human) specifically so this
# deployer SA does not need roles/rolebindings/serviceaccounts create-update-patch — that combination on a
# webhook-triggered SA is a privilege-escalation primitive. Neither manifest applied by CI creates a
# ServiceAccount anymore, so `serviceaccounts` is dropped from this grant too.
- apiGroups: [""]
resources: ["configmaps"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
# the manifests apply NetworkPolicies (fxhnt-cockpit.yaml + dagster's).
- apiGroups: ["networking.k8s.io"]
resources: ["networkpolicies"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: fxhnt-argo-workflow
namespace: foxhunt
labels: { app.kubernetes.io/part-of: fxhnt-ci }
subjects:
- { kind: ServiceAccount, name: argo-workflow, namespace: foxhunt }
roleRef:
kind: Role
name: fxhnt-argo-workflow
apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: fxhnt-ci-deployer
namespace: foxhunt
labels: { app.kubernetes.io/part-of: fxhnt-ci }
subjects:
- { kind: ServiceAccount, name: argo-workflow, namespace: foxhunt }
roleRef:
kind: Role
name: fxhnt-ci-deployer
apiGroup: rbac.authorization.k8s.io
---
# Cross-namespace: let the shared argo-events-sa submit Workflows into foxhunt (the sensor lives in argo-events
# but its Workflow runs here). Mirrors the bizworx `workflow-submitter` Role, scoped to foxhunt.
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: fxhnt-workflow-submitter
namespace: foxhunt
labels: { app.kubernetes.io/part-of: fxhnt-ci }
rules:
- apiGroups: ["argoproj.io"]
resources: ["workflows"]
verbs: ["create", "get", "list", "watch"]
- apiGroups: ["argoproj.io"]
resources: ["workflowtemplates"]
verbs: ["get", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: argo-events-sa-fxhnt-submitter
namespace: foxhunt
labels: { app.kubernetes.io/part-of: fxhnt-ci }
subjects:
- { kind: ServiceAccount, name: argo-events-sa, namespace: argo-events }
roleRef:
kind: Role
name: fxhnt-workflow-submitter
apiGroup: rbac.authorization.k8s.io

208
infra/argo/fxhnt-cockpit-workflowtemplate.yaml Normal file
View File

@@ -0,0 +1,208 @@
# Argo WorkflowTemplate — build + deploy the fxhnt cockpit, fully in-cluster.
#
# Reuses cluster primitives: the `argo-workflow` ServiceAccount (foxhunt, see fxhnt-ci-rbac.yaml), the
# `argo-git-ssh-key` deploy key (clone the fxhnt repo over in-cluster Gitea SSH, gitea-ssh.gitea.svc:22), and
# the `scw-registry` dockerconfig (push to the external Scaleway Container Registry rg.fr-par.scw.cloud/bizworx).
#
# Submit: argo submit -n foxhunt --from=wftmpl/fxhnt-cockpit -p commit-sha=<sha> -p mode=<auto|build|deploy> --watch
# Normally auto-triggered: a push to gitadmin/fxhnt master -> the build-fxhnt Sensor (argo-events) submits this.
apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
name: fxhnt-cockpit
namespace: foxhunt
labels: { app.kubernetes.io/name: fxhnt-cockpit, app.kubernetes.io/part-of: foxhunt }
spec:
serviceAccountName: argo-workflow
entrypoint: pipeline
archiveLogs: false # archive build/deploy logs to MinIO (argo-logs bucket). Reachable now that the
# pods carry app.kubernetes.io/part-of: argo-workflows (MinIO ingress netpol allow).
activeDeadlineSeconds: 1800
ttlStrategy: { secondsAfterCompletion: 3600 }
podGC: { strategy: OnPodCompletion }
arguments:
parameters:
- { name: commit-sha, value: HEAD }
- { name: mode, value: auto } # "auto" = decide-mode git-diffs commit-sha~1..commit-sha to pick build/deploy; "build" = force kaniko rebuild + deploy; "deploy" = force skip build, just apply + rollout restart (git-sync re-pulls code). A human can still force build/deploy via `argo submit -p mode=...`.
templates:
- name: pipeline
dag:
tasks:
# decide-mode: when mode=auto, git-diffs the pushed commit against its parent to pick build vs deploy
# (dep/Dockerfile changes -> build; anything else, or a missing parent, -> build/deploy per script logic
# below). When mode is forced (build|deploy), decide-mode just echoes it back unchanged.
- { name: decide-mode, template: decide-mode }
# Skip the ~5min kaniko build for code-only pushes (mode=deploy) — git-sync re-pulls src/ on the
# rollout restart, so only dep/Dockerfile changes need a new image (mode=build).
- { name: build, template: kaniko-build, depends: decide-mode, when: "{{tasks.decide-mode.outputs.parameters.mode}} == build" }
- { name: deploy, template: kubectl-deploy, depends: "decide-mode && (build.Succeeded || build.Skipped)" }
# --- decide-mode: resolve "auto" to build|deploy via git diff of the pushed commit vs its parent ---
- name: decide-mode
metadata:
labels: { app.kubernetes.io/part-of: argo-workflows }
nodeSelector: { k8s.scaleway.com/pool-name: large }
tolerations: [{ effect: NoSchedule, key: node.cilium.io/agent-not-ready, operator: Exists }]
outputs:
parameters:
- name: mode
valueFrom: { path: /workspace/mode }
container:
image: alpine/git:latest
command: [/bin/sh, -c]
# SECURITY: commit-sha/mode arrive via ENV (not inline {{...}} substitution) so an untrusted webhook
# payload cannot inject shell. commit-sha is validated hex before use; mode is checked against an allowlist.
env:
- { name: COMMIT_SHA, value: "{{workflow.parameters.commit-sha}}" }
- { name: MODE, value: "{{workflow.parameters.mode}}" }
# Absolute key path via GIT_SSH_COMMAND — Argo's emissary executor does not set HOME=/root, so the
# old `cp key -> ~/.ssh` idiom left git unable to find the key (clone failed with exit 128). This is
# HOME-independent and verified to clone under HOME=/nonexistent.
- { name: GIT_SSH_COMMAND, value: "ssh -i /etc/git-ssh/ssh-privatekey -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" }
args:
- |
set -e
case "$COMMIT_SHA" in *[!0-9a-fA-F]*|"") echo "refusing non-hex commit-sha" >&2; exit 1 ;; esac
case "$MODE" in auto|build|deploy) : ;; *) echo "refusing unknown mode: $MODE" >&2; exit 1 ;; esac
git clone --filter=blob:none \
"ssh://git@gitea-ssh.gitea.svc.cluster.local:22/gitadmin/fxhnt.git" /workspace/src
cd /workspace/src && git checkout "$COMMIT_SHA"
if [ "$MODE" != "auto" ]; then
printf '%s' "$MODE" > /workspace/mode
else
RESOLVED="$(git rev-parse "$COMMIT_SHA")"
if CHANGED=$(git diff --name-only "${RESOLVED}~1" "${RESOLVED}" 2>/dev/null); then
# diff succeeded (parent exists): build only if a dep/Dockerfile file changed, else deploy.
if echo "$CHANGED" | grep -qE '^pyproject\.toml$|^infra/docker/fxhnt\.Dockerfile$'; then
printf 'build' > /workspace/mode
else
printf 'deploy' > /workspace/mode
fi
else
# diff failed (no parent -- first commit / shallow history): default to build (safe, never wrong).
printf 'build' > /workspace/mode
fi
fi
cat /workspace/mode
volumeMounts:
- { name: git-ssh-key, mountPath: /etc/git-ssh, readOnly: true }
- { name: workspace, mountPath: /workspace }
volumes:
- { name: workspace, emptyDir: {} }
- { name: git-ssh-key, secret: { secretName: argo-git-ssh-key, defaultMode: 256 } }
# --- build: Kaniko builds infra/docker/fxhnt.Dockerfile -> internal registry ---
- name: kaniko-build
metadata:
labels: { app.kubernetes.io/part-of: argo-workflows } # MinIO ingress allows part-of in {foxhunt,foxhunt-ci,argo-workflows} → log archival works. NOT 'foxhunt' (that adds argo-base-egress which blocks registry:5000/git:2222).
nodeSelector: { k8s.scaleway.com/pool-name: large }
tolerations: [{ effect: NoSchedule, key: node.cilium.io/agent-not-ready, operator: Exists }]
initContainers:
- name: git-clone
image: alpine/git:latest
command: [/bin/sh, -c]
# SECURITY: commit-sha via ENV + hex-validated (see decide-mode note) — no inline shell substitution.
env:
- { name: COMMIT_SHA, value: "{{workflow.parameters.commit-sha}}" }
# Absolute key path via GIT_SSH_COMMAND — HOME-independent (Argo's emissary doesn't set HOME=/root).
- { name: GIT_SSH_COMMAND, value: "ssh -i /etc/git-ssh/ssh-privatekey -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" }
args:
- |
set -ex
case "$COMMIT_SHA" in *[!0-9a-fA-F]*|"") echo "refusing non-hex commit-sha" >&2; exit 1 ;; esac
git clone --no-checkout --filter=blob:none \
"ssh://git@gitea-ssh.gitea.svc.cluster.local:22/gitadmin/fxhnt.git" /workspace/src
cd /workspace/src && git checkout "$COMMIT_SHA"
echo "checked out $(git rev-parse --short HEAD)"
volumeMounts:
- { name: git-ssh-key, mountPath: /etc/git-ssh, readOnly: true }
- { name: workspace, mountPath: /workspace }
container:
image: gcr.io/kaniko-project/executor:v1.23.2
# Kaniko's own entrypoint + a YAML args LIST (the msp build-image pattern) — NOT `sh -c` with one big
# string. A shell wrapper made busybox try to parse the `{{=sprig.trunc(...)}}` expansion and choke on
# the `(` (syntax error). As a plain args list, Argo evaluates the expression and hands it straight to
# Kaniko, no shell in between.
args:
- --context=/workspace/src
- --dockerfile=/workspace/src/infra/docker/fxhnt.Dockerfile
- --destination=rg.fr-par.scw.cloud/bizworx/fxhnt-cockpit:latest
- --destination=rg.fr-par.scw.cloud/bizworx/fxhnt-cockpit:{{workflow.parameters.commit-sha}}
- --cache=true
- --cache-repo=rg.fr-par.scw.cloud/bizworx/cache
- --snapshot-mode=redo
env:
- { name: DOCKER_CONFIG, value: /kaniko/.docker }
resources: { requests: { cpu: "1", memory: 3Gi }, limits: { cpu: "3", memory: 10Gi } }
volumeMounts:
- { name: registry-auth, mountPath: /kaniko/.docker, readOnly: true }
- { name: workspace, mountPath: /workspace }
volumes:
- { name: workspace, emptyDir: {} }
- { name: git-ssh-key, secret: { secretName: argo-git-ssh-key, defaultMode: 256 } }
- name: registry-auth
secret:
secretName: scw-registry
items: [{ key: .dockerconfigjson, path: config.json }]
# --- deploy: apply manifests from the repo + roll the dashboard to the fresh image ---
- name: kubectl-deploy
metadata:
labels: { app.kubernetes.io/part-of: argo-workflows } # MinIO ingress allow → log archival works (see kaniko-build note)
nodeSelector: { k8s.scaleway.com/pool-name: large }
serviceAccountName: argo-workflow
initContainers:
- name: git-clone
image: alpine/git:latest
command: [/bin/sh, -c]
# SECURITY: commit-sha via ENV + hex-validated — no inline shell substitution.
env:
- { name: COMMIT_SHA, value: "{{workflow.parameters.commit-sha}}" }
# Absolute key path via GIT_SSH_COMMAND — HOME-independent (Argo's emissary doesn't set HOME=/root).
- { name: GIT_SSH_COMMAND, value: "ssh -i /etc/git-ssh/ssh-privatekey -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" }
args:
- |
set -ex
case "$COMMIT_SHA" in *[!0-9a-fA-F]*|"") echo "refusing non-hex commit-sha" >&2; exit 1 ;; esac
git clone --no-checkout --filter=blob:none \
"ssh://git@gitea-ssh.gitea.svc.cluster.local:22/gitadmin/fxhnt.git" /workspace/src
cd /workspace/src && git checkout "$COMMIT_SHA"
volumeMounts:
- { name: git-ssh-key, mountPath: /etc/git-ssh, readOnly: true }
- { name: workspace, mountPath: /workspace }
container:
image: alpine/k8s:1.31.0 # Alpine-based (has /bin/sh + kubectl). NOT registry.k8s.io/kubectl — that's distroless (no shell) and the deploy runs a shell script. bitnami/kubectl was removed from Docker Hub (2025).
command: [/bin/sh, -c]
args:
- |
set -e
cd /workspace/src
kubectl apply -f infra/k8s/services/fxhnt-dashboard.yaml
kubectl apply -f infra/k8s/orchestration/dagster.yaml
kubectl apply -f infra/k8s/network-policies/fxhnt-cockpit.yaml
kubectl -n foxhunt annotate deploy/fxhnt-dashboard deploy/dagster \
fxhnt.io/deployed-sha="$(cd /workspace/src && git rev-parse --short HEAD)" --overwrite
# Dashboard: scale 0 -> 1, NOT `rollout restart`. Its tailscale sidecar serves dashboard.fxhnt.ai;
# a rolling restart briefly overlaps two tailscale sessions on the same identity -> session race ->
# the external proxy link goes stale and the cockpit reads "down" post-deploy. Scaling to 0 cleanly
# terminates the session before a fresh, non-overlapping one comes up; the git-sync initContainer
# re-pulls src the same way, so code still updates. See reference_fxhnt_dashboard_proxy.
#
# SAFETY (CI review I1): once we scale to 0, a trap guarantees the deployment is scaled BACK to 1 on
# ANY exit path — so a failure between scale-0 and scale-1 (transient apiserver error, throttle) can
# never strand the production dashboard at 0 replicas. The trap is best-effort (its own failure is
# swallowed) and cleared on success.
trap 'kubectl -n foxhunt scale deploy/fxhnt-dashboard --replicas=1 || true' EXIT
kubectl -n foxhunt scale deploy/fxhnt-dashboard --replicas=0
kubectl -n foxhunt wait --for=delete pod -l app.kubernetes.io/name=fxhnt-dashboard --timeout=90s || true
kubectl -n foxhunt scale deploy/fxhnt-dashboard --replicas=1
trap - EXIT # scaled back up cleanly; no longer need the safety net
kubectl -n foxhunt rollout restart deploy/dagster # dagster has no external-proxy race
kubectl -n foxhunt rollout status deploy/fxhnt-dashboard --timeout=180s
kubectl -n foxhunt rollout status deploy/dagster --timeout=300s
volumeMounts:
- { name: workspace, mountPath: /workspace }
volumes:
- { name: workspace, emptyDir: {} }
- { name: git-ssh-key, secret: { secretName: argo-git-ssh-key, defaultMode: 256 } }

38
infra/k8s/orchestration/dagster.yaml
View File

@@ -40,8 +40,8 @@ data:
config:
service_account_name: tailscale-dagster
job_namespace: foxhunt
job_image: rg.fr-par.scw.cloud/bizworx/fxhnt-cockpit:dagster-k8s-25e3d7c
image_pull_policy: IfNotPresent
job_image: rg.fr-par.scw.cloud/bizworx/fxhnt-cockpit:latest
image_pull_policy: Always # :latest -> Always-pull so a dep-bump build reaches launched run Jobs (CI review C1)
image_pull_secrets:
- name: scw-registry
load_incluster_config: true
@@ -115,36 +115,6 @@ data:
}
}
---
apiVersion: v1
kind: ServiceAccount
metadata: { name: tailscale-dagster, namespace: foxhunt }
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata: { name: tailscale-dagster, namespace: foxhunt }
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["create", "get", "update", "patch", "list", "watch"]
# K8sRunLauncher: the daemon launches each Dagster run as its OWN K8s Job (own memory) so the heavy
# bybit_book_precompute asset (6-8Gi) never OOMs the 4Gi daemon. Needs to manage Jobs + read their pods/logs.
- apiGroups: ["batch"]
resources: ["jobs"]
verbs: ["create", "get", "list", "watch", "delete"]
- apiGroups: [""]
resources: ["pods", "pods/log"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata: { name: tailscale-dagster, namespace: foxhunt }
subjects:
- { kind: ServiceAccount, name: tailscale-dagster, namespace: foxhunt }
roleRef: { kind: Role, name: tailscale-dagster, apiGroup: rbac.authorization.k8s.io }
---
apiVersion: apps/v1
kind: Deployment
metadata:
@@ -201,7 +171,7 @@ spec:
- { name: git-ssh-key, mountPath: /etc/git-ssh, readOnly: true }
containers:
- name: webserver
image: rg.fr-par.scw.cloud/bizworx/fxhnt-cockpit:dagster-k8s-25e3d7c
image: rg.fr-par.scw.cloud/bizworx/fxhnt-cockpit:latest
command: ["dagster-webserver", "-h", "0.0.0.0", "-p", "3000",
"-w", "/dagster-home/workspace.yaml"]
env: &dagster_env
@@ -229,7 +199,7 @@ spec:
- { name: code, mountPath: /code, readOnly: true } # git-sync'd src (PYTHONPATH=/code/src)
resources: { requests: { cpu: 100m, memory: 512Mi }, limits: { cpu: "1", memory: 1Gi } }
- name: daemon
image: rg.fr-par.scw.cloud/bizworx/fxhnt-cockpit:dagster-k8s-25e3d7c
image: rg.fr-par.scw.cloud/bizworx/fxhnt-cockpit:latest
command: ["dagster-daemon", "run", "-w", "/dagster-home/workspace.yaml"]
env: *dagster_env
volumeMounts:

72
infra/k8s/rbac/tailscale-sidecars.yaml Normal file
View File

@@ -0,0 +1,72 @@
# Tailscale sidecar RBAC (ServiceAccount + Role + RoleBinding) for the fxhnt-dashboard and dagster
# Deployments. Moved out of infra/k8s/services/fxhnt-dashboard.yaml and infra/k8s/orchestration/dagster.yaml
# (CI review M1): those app manifests are applied every deploy by the fxhnt-ci-deployer ServiceAccount via
# the kubectl-deploy Argo template, and having RBAC objects bundled in there meant the deployer SA needed
# create/update/patch on roles/rolebindings in its own namespace — a privilege-escalation primitive on a
# webhook-triggered SA. This file is STATIC and applied ONCE, out-of-band, by a human:
#
# kubectl apply -f infra/k8s/rbac/tailscale-sidecars.yaml
#
# It is NOT applied by the CI pipeline. The Deployments still reference these ServiceAccounts via
# `serviceAccountName: tailscale-dashboard` / `tailscale-dagster` — that reference resolves because the
# SAs are managed here, separately, not because the app manifests create them anymore.
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: tailscale-dashboard
namespace: foxhunt
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: tailscale-dashboard
namespace: foxhunt
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["create", "get", "update", "patch", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: tailscale-dashboard
namespace: foxhunt
subjects:
- kind: ServiceAccount
name: tailscale-dashboard
namespace: foxhunt
roleRef:
kind: Role
name: tailscale-dashboard
apiGroup: rbac.authorization.k8s.io
---
apiVersion: v1
kind: ServiceAccount
metadata: { name: tailscale-dagster, namespace: foxhunt }
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata: { name: tailscale-dagster, namespace: foxhunt }
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["create", "get", "update", "patch", "list", "watch"]
# K8sRunLauncher: the daemon launches each Dagster run as its OWN K8s Job (own memory) so the heavy
# bybit_book_precompute asset (6-8Gi) never OOMs the 4Gi daemon. Needs to manage Jobs + read their pods/logs.
- apiGroups: ["batch"]
resources: ["jobs"]
verbs: ["create", "get", "list", "watch", "delete"]
- apiGroups: [""]
resources: ["pods", "pods/log"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata: { name: tailscale-dagster, namespace: foxhunt }
subjects:
- { kind: ServiceAccount, name: tailscale-dagster, namespace: foxhunt }
roleRef: { kind: Role, name: tailscale-dagster, apiGroup: rbac.authorization.k8s.io }

30
infra/k8s/services/fxhnt-dashboard.yaml
View File

@@ -112,33 +112,3 @@ metadata:
spec:
selector: { app.kubernetes.io/name: fxhnt-dashboard }
ports: [{ port: 80, targetPort: 8080, name: http }]
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: tailscale-dashboard
namespace: foxhunt
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: tailscale-dashboard
namespace: foxhunt
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["create", "get", "update", "patch", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: tailscale-dashboard
namespace: foxhunt
subjects:
- kind: ServiceAccount
name: tailscale-dashboard
namespace: foxhunt
roleRef:
kind: Role
name: tailscale-dashboard
apiGroup: rbac.authorization.k8s.io

24
scripts/argo-deploy-cockpit.sh
View File

@@ -1,24 +0,0 @@
#!/usr/bin/env bash
# Build + deploy the fxhnt cockpit via Argo Workflows (in-cluster Kaniko build, no laptop push).
#
# Prereqs (one-time): the `fxhnt-cockpit` WorkflowTemplate is registered
# (kubectl -n foxhunt apply -f infra/argo/cockpit-build-deploy.yaml)
# and this repo is pushed to the in-cluster GitLab as root/fxhnt.git.
#
# Usage:
# ./scripts/argo-deploy-cockpit.sh # build+deploy HEAD of root/fxhnt master
# ./scripts/argo-deploy-cockpit.sh <commit-sha> # a specific commit
set -euo pipefail
SHA="${1:-HEAD}"
NS="foxhunt"
# Resolve HEAD to the real master sha so the in-cluster clone checks out exactly what was pushed.
if [ "$SHA" = "HEAD" ]; then
SHA="$(git rev-parse HEAD)"
fi
echo "Submitting fxhnt-cockpit build+deploy for ${SHA} ..."
argo submit -n "$NS" --from=wftmpl/fxhnt-cockpit \
-p commit-sha="$SHA" \
--watch