jgrusewski 99f54e3f94 fix(infra): resolve postgres disk-full cascade, add SPOF mitigations ...

Postgres PVC hit 100% → crash-looped → GitLab webservice couldn't
verify SSH keys → all git operations failed. Root cause: 10Gi PVC
outgrown by GitLab database.

Fixes applied:
- Expand postgres PVC 10Gi → 20Gi
- Expand minio PVC 100Gi → 150Gi (was 81.8% full)
- Expand prometheus PVC 2Gi → 10Gi, retentionSize 1500MB → 8GB
- Scale gitlab-webservice to 2 replicas for HA
- Add PVC autoscaler CronJob (every 15min, auto-expand at 85%)
- Add daily postgres backup CronJob (pg_dump → MinIO, 7d + 4w retention)
- Add PrometheusRule storage alerts (75% warning, 90% critical)
- Add network policies for maintenance job egress

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-03-28 18:55:25 +01:00

..

alertmanager.yaml

fix(netpol): fix 3 DOWN Prometheus targets — operator port, reloader ingress, argo label

2026-03-14 12:52:07 +01:00

allow-dns.yaml

fix(netpol): allow DNS + egress for CI executor pods

2026-03-02 11:56:44 +01:00

allow-monitoring.yaml

feat(infra): QuestDB metrics sink + monitoring network policies

2026-03-08 02:42:53 +01:00

api.yaml

refactor: update K8s, CI, Docker, Prometheus, scripts, and FXT CLI for api rename

2026-03-04 23:46:36 +01:00

backtesting-service.yaml

refactor: update K8s, CI, Docker, Prometheus, scripts, and FXT CLI for api rename

2026-03-04 23:46:36 +01:00

broker-gateway.yaml

refactor: update K8s, CI, Docker, Prometheus, scripts, and FXT CLI for api rename

2026-03-04 23:46:36 +01:00

ci-egress.yaml

fix(netpol): allow DNS + egress for CI executor pods

2026-03-02 11:56:44 +01:00

data-acquisition-service.yaml

refactor: update K8s, CI, Docker, Prometheus, scripts, and FXT CLI for api rename

2026-03-04 23:46:36 +01:00

data-download-job.yaml

feat(infra): training data cache architecture — MinIO → PVC sync

2026-03-07 14:45:30 +01:00

default-deny.yaml

fix(k8s): fix NetworkPolicy label selectors and scope default-deny

2026-03-01 23:51:54 +01:00

gitlab-packages.yaml

fix(infra): add network policy for GitLab package registry access

2026-03-07 21:15:19 +01:00

ib-gateway.yaml

infra: add per-service NetworkPolicy rules

2026-03-01 23:41:13 +01:00

infrastructure.yaml

feat(infra): Argo log persistence, network policies, lightweight CI images

2026-03-15 12:00:46 +01:00

maintenance-jobs.yaml

fix(infra): resolve postgres disk-full cascade, add SPOF mitigations

2026-03-28 18:55:25 +01:00

mattermost.yaml

feat(infra): deploy Mattermost on platform nodes with CI webhook egress

2026-03-12 09:20:32 +01:00

ml-training-service.yaml

refactor: update K8s, CI, Docker, Prometheus, scripts, and FXT CLI for api rename

2026-03-04 23:46:36 +01:00

prometheus-cilium.yaml

fix(monitoring): wire cluster resources, fix Prometheus networking

2026-03-04 12:58:12 +01:00

prometheus.yaml

fix(netpol): fix 3 DOWN Prometheus targets — operator port, reloader ingress, argo label

2026-03-14 12:52:07 +01:00

stalwart.yaml

fix(stalwart): TLS cert %{file} macros, HTTPS egress for webadmin

2026-03-12 18:27:18 +01:00

trading-agent-service.yaml

refactor: update K8s, CI, Docker, Prometheus, scripts, and FXT CLI for api rename

2026-03-04 23:46:36 +01:00

trading-service.yaml

refactor: update K8s, CI, Docker, Prometheus, scripts, and FXT CLI for api rename

2026-03-04 23:46:36 +01:00

training-job.yaml

feat(infra): QuestDB metrics sink + monitoring network policies

2026-03-08 02:42:53 +01:00

web-dashboard.yaml

fix(web-dashboard): switch protobuf codegen to js+dts for TS 5.9 compat

2026-03-05 14:32:41 +01:00