Commit Graph

6 Commits

Author SHA1 Message Date
jgrusewski
7382ffd1e2 feat(infra): QuestDB metrics sink + monitoring network policies
Add QuestDB ILP sink for training metrics, update Prometheus scrape
configs, and fix network policies for monitoring stack connectivity.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 02:42:53 +01:00
jgrusewski
68b6aa8313 feat(infra): migrate container registry from SCW to internal GitLab
Full migration off Scaleway Container Registry to internal GitLab
registry backed by MinIO S3. All 4 images (ci-builder, ci-builder-cpu,
foxhunt-runtime, foxhunt-training-runtime) rebuilt in internal registry.

Registry & images:
- All image refs → gitlab-registry.foxhunt.svc.cluster.local:5000/root/foxhunt/
- imagePullSecrets: scw-registry → gitlab-registry
- Kaniko build template: two-step DAG (git-clone → kaniko-build) with shared PVC
- Kaniko layer cache enabled at root/foxhunt/cache
- AWS_ACCESS_KEY_ID: $SCW_ACCESS_KEY → $MINIO_ACCESS_KEY in .gitlab-ci.yml

Network policies:
- ci-pipeline: add HTTP/80, registry/5000, webservice/8181 egress rules

DNS & Tailscale proxy cleanup:
- Remove ci, prometheus, monitor DNS records (no longer exposed)
- Rename s3 → minio DNS record
- Remove Argo UI, Prometheus, monitor nginx server blocks
- Remove argo-htpasswd volume mount
- Tailscale proxy nodeSelector: infra → platform

Terraform cleanup:
- Delete infra/modules/registry/ (SCW CR namespace)
- Delete infra/modules/object-storage/ (SCW S3 buckets)
- Delete infra/modules/secrets/ (SCW secrets)
- Delete corresponding live configs
- TF state backend: S3 → GitLab HTTP

Argo workflows:
- Add events/ (GitLab push eventsource + ci-pipeline sensor)
- ci-pipeline + training templates: SCW → internal registry
- Delete obsolete compile-training-template.yaml

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 13:52:24 +01:00
jgrusewski
b3cc52c13a infra: migrate QuestDB to platform pool (fresh start)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 10:22:57 +01:00
jgrusewski
e25bc8d835 fix(k8s): fix NetworkPolicy label selectors and scope default-deny
Caught during pre-apply review:
- Scope default-deny/allow-dns/allow-monitoring to pods with
  app.kubernetes.io/part-of=foxhunt (was podSelector:{} which would
  block gitlab, grafana, loki sharing the namespace)
- Fix infrastructure.yaml: use app.kubernetes.io/name instead of
  bare app label for postgres/redis/minio/questdb podSelectors
- Add app.kubernetes.io/part-of=foxhunt to all pod templates so
  default-deny and infrastructure ingress rules match correctly

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 23:51:54 +01:00
jgrusewski
1a41f9475a refactor(infra): update all manifests for pool rename (always-on→services, ci-build→ci-compile/ci-training)
All K8s nodeSelectors, CI config, runner config, monitoring docs, and
smoke tests now reference the new pool names provisioned by terragrunt.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-27 10:33:34 +01:00
jgrusewski
2a3c107b31 infra: add K8s manifests for full paper trading stack
- Tailscale subnet router (zero public access)
- PostgreSQL, Redis, QuestDB (PVCs on always-on node)
- 7 service deployments (trading, api-gw, broker-gw, ml, backtest, agent, web-gw)
- GPU training Job template + idle reaper CronJob
- Block Storage PV/PVC for shared training data

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 14:27:32 +01:00