gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
2,782 Commits 13 Branches 0 Tags
ae704a7b7346da17039e4aeef8265f65c6df40ee
Commit Graph

9 Commits

Author SHA1 Message Date
jgrusewski
b31329931f fix(infra): remove MinIO TLS, fix sccache 0% cache hits, update pool selectors 
- Remove all HTTPS/TLS from MinIO (plain HTTP for internal cluster traffic)
- Fix sccache 0% cache hit rate (rustls rejected self-signed MinIO cert)
- Remove hardcoded URLs from k8s_dispatcher.rs (S3_ENDPOINT, TRAINING_RUNTIME_IMAGE,
  CALLBACK_ENDPOINT now required env vars)
- Update GitLab registry S3 credentials to HTTP endpoint
- Fix PVC manifest (20Gi → 100Gi to match cluster)
- Fix nodeSelector: infra/foxhunt → platform (match actual node pool)
- Fix rclone trailing backslash causing chmod to be parsed as rclone args
- Remove minio-ca-cert ConfigMap references from all manifests
- Update trading-service GPU overlay to l40s pool

20 files changed, -118 lines net

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-06 23:53:05 +01:00
jgrusewski
68b6aa8313 feat(infra): migrate container registry from SCW to internal GitLab 
Full migration off Scaleway Container Registry to internal GitLab
registry backed by MinIO S3. All 4 images (ci-builder, ci-builder-cpu,
foxhunt-runtime, foxhunt-training-runtime) rebuilt in internal registry.

Registry & images:
- All image refs → gitlab-registry.foxhunt.svc.cluster.local:5000/root/foxhunt/
- imagePullSecrets: scw-registry → gitlab-registry
- Kaniko build template: two-step DAG (git-clone → kaniko-build) with shared PVC
- Kaniko layer cache enabled at root/foxhunt/cache
- AWS_ACCESS_KEY_ID: $SCW_ACCESS_KEY → $MINIO_ACCESS_KEY in .gitlab-ci.yml

Network policies:
- ci-pipeline: add HTTP/80, registry/5000, webservice/8181 egress rules

DNS & Tailscale proxy cleanup:
- Remove ci, prometheus, monitor DNS records (no longer exposed)
- Rename s3 → minio DNS record
- Remove Argo UI, Prometheus, monitor nginx server blocks
- Remove argo-htpasswd volume mount
- Tailscale proxy nodeSelector: infra → platform

Terraform cleanup:
- Delete infra/modules/registry/ (SCW CR namespace)
- Delete infra/modules/object-storage/ (SCW S3 buckets)
- Delete infra/modules/secrets/ (SCW secrets)
- Delete corresponding live configs
- TF state backend: S3 → GitLab HTTP

Argo workflows:
- Add events/ (GitLab push eventsource + ci-pipeline sensor)
- ci-pipeline + training templates: SCW → internal registry
- Delete obsolete compile-training-template.yaml

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-05 13:52:24 +01:00
jgrusewski
d0b896b01c fix(infra): probe TWS API port 4002, use Recreate strategy 
Readiness/liveness probes were checking port 4004 (socat), which
always listens even when the gateway is stuck at login — masking
failures. Now probes check port 4002 (TWS API), which only opens
after successful IBKR authentication.

Deploy strategy changed to Recreate because IBKR allows only one
session per account. RollingUpdate starts the new pod before killing
the old one, causing both to fight over the session in a crash loop.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-02 14:37:51 +01:00
jgrusewski
13be624220 fix(infra): add EXISTING_SESSION_DETECTED_ACTION to IB Gateway 
The gnzsnz/ib-gateway image renders its IBC config.ini from a template
using envsubst. The ExistingSessionDetectedAction field was rendering
as empty, causing IBC to show a GUI dialog when IBKR detected a
competing session — which nobody could dismiss in a headless pod.

Setting primaryoverride tells IBC to automatically take over any
existing session, which is the correct behavior for a K8s deployment.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-02 14:26:10 +01:00
jgrusewski
c86743f7eb infra: rename services → foxhunt pool in all K8s manifests 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-02 10:50:44 +01:00
jgrusewski
e25bc8d835 fix(k8s): fix NetworkPolicy label selectors and scope default-deny 
Caught during pre-apply review:
- Scope default-deny/allow-dns/allow-monitoring to pods with
  app.kubernetes.io/part-of=foxhunt (was podSelector:{} which would
  block gitlab, grafana, loki sharing the namespace)
- Fix infrastructure.yaml: use app.kubernetes.io/name instead of
  bare app label for postgres/redis/minio/questdb podSelectors
- Add app.kubernetes.io/part-of=foxhunt to all pod templates so
  default-deny and infrastructure ingress rules match correctly

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-01 23:51:54 +01:00
jgrusewski
8991e031f4 infra: add SecurityContext to all service deployments 
Pod-level: runAsNonRoot, runAsUser/Group 1000, seccomp RuntimeDefault.
Container-level: no privilege escalation, read-only rootfs, drop ALL caps.
Exception: ib-gateway skips runAsNonRoot and readOnlyRootFilesystem.
InitContainers keep runAsUser: 0 for binary fetch.
/tmp emptyDir (50Mi) mounted for writable scratch space.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-01 23:41:13 +01:00
jgrusewski
37cc6959c4 fix(infra): add explicit services nodeSelector to all service manifests 
Previously nodeSelector was only applied via kubectl patch at deploy time.
Now it's in the YAML files to keep git and cluster in sync.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-27 10:40:40 +01:00
jgrusewski
86f7f1fa76 fix: comprehensive audit — real brokers, deployment fixes, production safety 
Codebase audit identified 23 findings across 4 dimensions (production safety,
code health, deployment readiness, test quality). This commit fixes all of them.

Broker execution layer (was entirely stubbed):
- Real IBKR TWS client via ibapi crate (950+ lines, feature-gated)
- ICMarkets ctrader-openapi now always-on (removed feature flag)
- Real broker routing with health monitoring and exponential backoff reconnect
- Validated against live IB Gateway Docker (6/6 connectivity tests pass)

Deployment blockers:
- Fixed 6 broken Dockerfiles (removed COPY foxhunt-deploy)
- Created foxhunt K8s namespace, secret templates, migration job
- Added liveness probes to all 7 K8s services
- IB Gateway manifest (ghcr.io/gnzsnz/ib-gateway:stable)
- IBKR credentials in Scaleway Secret Manager via Terragrunt
- Fixed port collisions and mismatches across services

Production safety (9 critical + 6 high/medium fixes):
- Asset-class-specific VaR volatility (not flat 2%)
- Real parametric VaR with z-score 95th percentile
- Kyle's lambda regression (100-bar rolling window)
- Per-feature running statistics from historical data
- VWAP-based slippage reference, regime duration tracking
- Real Databento JSON parsing for OHLCV/Trade/Quote

Code health:
- Removed #![allow(dead_code)] from ml, data, config
- Fixed log:: → tracing:: in 4 production files
- Removed dead workspace deps (ratatui, crossterm)

Verified: cargo check --workspace (0 errors), trading_engine 330 tests pass.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-25 00:32:10 +01:00