infra(k8s): add NetworkPolicy for paper-backfill Job (DNS + 5432->postgres)

The Job failed with ConnectionTimeout — default-deny-all blocks egress and
part-of/argo-base-egress only opens DNS+443, not Postgres 5432. Add a dedicated
netpol (mirrors fxhnt-factory/forward) granting DNS + 5432->postgres. The exec
path worked only because it ran inside the dagster pod (which has its own netpol).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-06-22 18:55:08 +02:00
parent f6aa028b12
commit 8eeb4fc027

View File

@@ -14,9 +14,10 @@
# the dagster pod (topologyKey hostname) co-locates automatically and survives node recycling (no hardcoded
# nodeName). Reads the warehouse (closes) + writes Postgres (paper_*) only — no DuckDB write contention.
#
# EGRESS: the `app.kubernetes.io/part-of: foxhunt` POD label is REQUIRED — the namespace has default-deny-all
# egress; that label grants `argo-base-egress` (DNS + internal CIDRs), which is all this Job needs (Postgres
# is internal; no external fetches). Without it the backfill hangs on DB connect.
# EGRESS: the namespace has default-deny-all egress. `argo-base-egress` (via the part-of label) only opens
# DNS + 443 — NOT Postgres' 5432 — so this Job needs its OWN NetworkPolicy granting DNS + 5432→postgres
# (mirrors fxhnt-factory/fxhnt-forward). The `--days 3` exec worked only because it ran INSIDE the dagster
# pod, which already has the dagster netpol. Bundled below. No external egress needed (warehouse is a local PVC).
apiVersion: batch/v1
kind: Job
metadata:
@@ -64,3 +65,20 @@ spec:
volumes:
- name: surfer-data
persistentVolumeClaim: { claimName: fxhnt-surfer-data }
---
# Egress for the backfill Job: DNS + Postgres 5432 (internal). default-deny-all blocks all egress; the shared
# argo-base-egress only opens DNS + 443, so this Job needs its own 5432→postgres rule (same minimal shape as
# the fxhnt-factory / fxhnt-forward netpols). No external egress — the warehouse is a local PVC.
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: fxhnt-paper-backfill
namespace: foxhunt
labels: { app.kubernetes.io/part-of: foxhunt }
spec:
podSelector: { matchLabels: { app.kubernetes.io/name: fxhnt-paper-backfill } }
policyTypes: [Egress]
egress:
- ports: [{ port: 53, protocol: UDP }, { port: 53, protocol: TCP }]
- ports: [{ port: 5432, protocol: TCP }]
to: [{ podSelector: { matchLabels: { app.kubernetes.io/name: postgres } } }]