129 Commits

Author SHA1 Message Date
0d8ce554a3 fix(dashboard): allow tailscale sidecar egress to kube-apiserver
The dashboard NetworkPolicy blocked egress to the kube-apiserver (kube-proxy
DNATs kubernetes.default.svc 10.32.0.1:443 -> 172.16.0.11:6443, and both
10.32/16 and 100.64/15 were excepted). On a FRESH sidecar start (e.g. after
the CI deploy's scale-0->1) tailscale must read its ts-state Secret via the
apiserver -> without the allow it crashlooped 'context deadline exceeded' ->
bad gateway on dashboard.fxhnt.ai. Adds the 6443->172.16.0.11/32 egress rule,
mirroring the dagster NP fix (06d12fd). Pre-existing gap, surfaced by the
first pipeline deploy that recreated the dashboard pod.
2026-07-21 12:27:00 +00:00
05f6d624c0 fix(ci): M1 split tailscale RBAC out of app manifests, drop deployer rolebinding grant
The dashboard/dagster manifests bundled the tailscale-sidecar SA+Role+RoleBinding,
which forced the webhook-triggered fxhnt-ci-deployer SA to hold create/update/patch
on roles/rolebindings in its own namespace — a privilege-escalation primitive. Move
those 6 objects to infra/k8s/rbac/tailscale-sidecars.yaml, applied once out-of-band
(not by CI); trim the deployer Role to drop rbac.authorization.k8s.io entirely, and
drop serviceaccounts too since neither CI-applied manifest creates one anymore.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-21 11:32:51 +00:00
40c0b187a7 fix(ci): C1 dagster to :latest+Always-pull; I1 trap restores dashboard replicas
C1 (Critical, CI review): dagster webserver/daemon + K8sRunLauncher job_image
were pinned to a hardcoded tag (dagster-k8s-25e3d7c), so a mode=build dep bump
updated the dashboard (:latest) but left dagster + its run Jobs on stale deps
-> version skew in the nightly ETL. Now :latest with image_pull_policy Always.

I1 (Important): a trap scales the dashboard back to replicas=1 on ANY exit
path, so a failure between scale-0 and scale-1 can't strand the cockpit at 0.
2026-07-21 11:30:40 +00:00
ee077a2cc1 docs(ci): fix stale header comments (gitea-ssh host, submit usage) 2026-07-21 11:23:35 +00:00
e00bac067f ci(fxhnt): retire old cockpit-build-deploy + argo-deploy-cockpit.sh
Superseded by fxhnt-cockpit-workflowtemplate.yaml + fxhnt-build-sensor.yaml
+ fxhnt-ci-rbac.yaml (all validated end-to-end, both modes). The old files
targeted decommissioned infra (ci-compile-cpu pool, gitea-sshd.foxhunt,
MinIO artifacts) and no longer worked.
2026-07-21 11:23:01 +00:00
9d35df6bfc fix(ci): use plain param substitution for image tag (this Argo doesn't eval {{=...}})
The {{=sprig.trunc(...)}} expression wasn't evaluated by this Argo (v4.0.6) —
Kaniko received it literally and rejected it as an invalid tag. Use the plain
{{workflow.parameters.commit-sha}} substitution (which works everywhere else
in this template); the full 40-char sha is a valid Docker tag, no trunc needed.
2026-07-21 11:14:34 +00:00
da28c049f7 fix(ci): kaniko as args-list not sh -c (busybox choked on sprig expansion)
/busybox/sh -c parsed the {{=sprig.trunc(...)}} destination and errored on
the '(' -> syntax error, exit 2. Switched to Kaniko's entrypoint + a plain
YAML args list (the proven msp build-image pattern): Argo evaluates the
expression and passes it straight to Kaniko, no shell. Also pin executor
v1.23.2 (matches msp; no busybox needed).
2026-07-21 11:11:19 +00:00
6677f73a64 fix(ci): lower Kaniko resource requests to fit the large pool
Requested cpu:2/mem:4Gi was sized for the removed dedicated ci-compile-cpu
pool; the shared large nodes (3800m, ~1.3-1.6 CPU free) couldn't schedule it
(Insufficient cpu, autoscaler didn't add nodes). cpu:1/mem:3Gi request
(limit 3/10Gi) fits and still builds fine (Kaniko is I/O-bound here).
2026-07-21 11:07:11 +00:00
330bb56b52 fix(ci): deployer RBAC covers configmaps/serviceaccounts/roles/rolebindings
The dashboard+dagster manifests bundle a ConfigMap, ServiceAccount, and the
tailscale-sidecar Role/RoleBinding alongside the Deployment, so kubectl apply
of the whole file needs those types. RBAC-management is scoped to foxhunt
only (documented as a deliberate, namespace-local trade-off).
2026-07-21 10:51:03 +00:00
9f2b47a789 fix(ci): disable archiveLogs (no artifact repository in this Argo)
archiveLogs=true was inherited from the old MinIO-backed pipeline; this
Argo has no artifact repository configured, so log archival failed the
step. We don't need archived logs -> false.
2026-07-21 10:48:23 +00:00
80bf243023 fix(ci): use GIT_SSH_COMMAND absolute key path (Argo emissary has no HOME=/root)
The cp-key-to-~/.ssh idiom failed under Argo's executor because emissary
doesn't set HOME=/root, so git couldn't find the key -> clone exit 128.
GIT_SSH_COMMAND with the absolute key path is HOME-independent (verified
cloning under HOME=/nonexistent). Applied to all three git-clone steps.
2026-07-21 10:46:56 +00:00
6cf19ec379 fix(ci): correct gitea SSH host (gitea-sshd.foxhunt -> gitea-ssh.gitea)
The reanimated template used the OLD foxhunt-cluster gitea SSH service
(gitea-sshd.foxhunt), which doesn't exist on bizworx -> git clone failed
with exit 128. The real service is gitea-ssh.gitea.svc:22 (verified: the
argo-git-ssh-key deploy key resolves gitadmin/fxhnt HEAD there).
2026-07-21 10:41:12 +00:00
7d708b03b1 fix(ci): write decide-mode output to /workspace emptyDir not /tmp
Argo's executor reads a valueFrom.path output parameter from a mounted
volume, not the container rootfs -> /tmp/mode gave 'no such file or
directory'. /workspace is the shared emptyDir the step already mounts.
2026-07-21 10:39:15 +00:00
7318f77054 fix(ci): drop stale fr-par-2 zone pin from nodeSelectors
The reanimated template inherited topology.kubernetes.io/zone=fr-par-2 from
the old ci-compile-cpu pool, but all  nodes (and every fxhnt workload)
are in fr-par-1 -> pods were Unschedulable. pool-name=large alone is correct.
2026-07-21 10:38:05 +00:00
6d8bcc3ae7 ci(fxhnt): RBAC for the pipeline in foxhunt (SA + deploy role + cross-ns submit)
First real workflow run surfaced two gaps: (1) the WorkflowTemplate's SA
argo-workflow didn't exist in foxhunt (only in argo where msp builds);
(2) argo-events-sa couldn't submit workflows cross-ns into foxhunt. Adds
the SA, a least-privilege deployer Role (apply/scale/annotate/rollout the
dashboard+dagster deployments, exactly what kubectl-deploy runs), the Argo
workflowtaskresults role, and a foxhunt-scoped workflow-submitter binding.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-21 10:31:24 +00:00
0069b75d0e fix(ci): harden WorkflowTemplate against template injection
commit-sha (from the untrusted Gitea webhook body.after) was substituted
INLINE into shell scripts via {{workflow.parameters.commit-sha}} in all
three git-clone/decide-mode steps -> a malicious payload could inject shell
on a cluster-privileged pod. Now passed via ENV + hex-validated before use
(the build-image template's pattern), and mode is allowlist-checked. Closes
the background security review's Orchestrator Template Injection finding.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-21 10:20:26 +00:00
603aaaabff ci(fxhnt): single-trigger sensor + in-workflow git-diff mode decision 2026-07-21 10:14:48 +00:00
9b2b25433e ci(fxhnt): deploy step — keep scale 0->1 dance + deployed-sha annotation
Verified live: fxhnt-dashboard has its own dedicated tailscale sidecar
(TS_HOSTNAME=fxhnt-dashboard) and there is no Ingress/Gateway/VirtualService
in ns foxhunt routing dashboard.fxhnt.ai anywhere. fxhnt-ingress is an
unrelated nginx+tailscale pod (its own Deployment/Service, own tailscale
identity) for git/ssh routes, not the dashboard. The tailscale-session race
on `rollout restart` is real, so keep scale 0->1. Added the
fxhnt.io/deployed-sha annotation on both fxhnt-dashboard and dagster before
the rollouts, per the CI/CD plan Task 2.
2026-07-21 10:12:33 +00:00
6e9dfec067 ci(fxhnt): reanimate cockpit WorkflowTemplate (node pool large, sha7 tag) 2026-07-21 10:09:15 +00:00
92b0871050 chore(vrp): remove fxhnt-opra-backfill CronJob (backfill-xsp-opra command deleted)
The CronJob invoked `fxhnt backfill-xsp-opra` (removed in Task 3); with the
command gone it would crash-loop. Its only purpose was freezing OPRA marks
for VRP recompute. OPRA data already frozen in xsp_option_bars (KEPT).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-21 00:01:30 +00:00
a16b341d90 fix(ib-gateway): zero-pad AUTO_RESTART_TIME (8:00->08:00 AM) — stops ~16 restarts/day
IBC's parser rejected '8:00 AM' ('Auto restart time setting must be hh:mm AM/PM'),
leaving the daily auto-restart UNCONFIGURED, so the gateway churned ~35 restarts over
2 days instead of one clean daily cycle. Zero-padded to '08:00 AM' — IBC now logs
'Auto restart time set to 08:00 AM' (verified live 2026-07-20, fresh pod RESTARTS=0).
08:00 UTC is before the 09:00 UTC UCITS rebalance window so the daily restart won't
collide with execution. Also bumped liveness failureThreshold 5->6 (180s) for the
reauth window.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 21:30:01 +00:00
1ecf3983d9 chore(scheduling): delete compare-measured + vrp crons; arm factory + ucits-volume
compare-measured-precompute is now the Dagster bybit_book_precompute asset
(K8sRunLauncher, own memory) — cron + NP deleted. vrp-rebalancer deleted
(shelved/falsified; vrp_nav de-wired from the nightly). Armed factory +
ucits-volume-ibkr in the manifests (source of truth) to match the live state;
bybit-rebalancer (no testnet creds) + ibkr-rebalancer (PRIIPs) stay suspended.
Final: Dagster owns all nightly ETL+precompute; K8s CronJobs own execution only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 10:12:45 +00:00
3461f0e331 feat(orchestration): K8sRunLauncher instance config (per-run Jobs, own memory)
Configure K8sRunLauncher in the dagster-instance ConfigMap: each run launches
as its own K8s Job so the heavy bybit_book_precompute asset (6-8Gi op_tags)
runs isolated, never OOMing the 4Gi daemon. Launched Jobs git-sync the same
code (initContainer + emptyDir /code, same gitea+key), get the fund env +
secrets (DSN/PGPASSWORD/databento/tiingo), and need NO PVC (compute is
Postgres-only; unlock calendar is DB-first, so the RWO surfer PVC stays with
the daemon — no deadlock). Validated: DagsterInstance.from_config loads it as
K8sRunLauncher. No extra NP needed (no postgres ingress NP + no default-deny,
so launched Jobs reach postgres/gitea with open egress).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 09:48:47 +00:00
5b016b9c4d feat(orchestration): dagster K8sRunLauncher RBAC + point at dagster-k8s image
Add Jobs/pods/events RBAC to the tailscale-dagster SA (K8sRunLauncher launches
each run as its own Job) and point both dagster containers at the new
fxhnt-cockpit:dagster-k8s-25e3d7c image (dagster_k8s 0.28.22 baked in). The
K8sRunLauncher instance config comes next once the daemon is verified on the
new image.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 09:45:12 +00:00
18cdf3f704 docs(plan): rewrite for all-Dagster K8sRunLauncher + proper podman image build
Operator: go all-Dagster properly + build the image right. Diagnosis
corrected: Dagster ETL was healthy all along (23:30 run SUCCESS, 14 assets,
forward_nav fresh); the stale data is the SUSPENDED compare-measured cron that
writes bybit_sleeve_ret. That cron exists because the compute needs 6-8Gi and
OOM-killed the 4Gi daemon — so the correct all-Dagster fix is K8sRunLauncher
(per-run Jobs with own memory), not daemon-run assets. Plan: add dagster-k8s,
build+push via podman (Argo/Kaniko pipeline is dead), add Job-launch RBAC,
configure K8sRunLauncher + per-asset memory, fold sleeve-ret/backtest-refs into
the DAG, materialize+verify, then delete compare-measured+VRP crons and arm
execution. Also commits the harmless working_directory=/code/src fix.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 09:13:33 +00:00
e67c6e1612 feat(migrate): P2 complete — fxhnt-ingress proxy verified for all 3 fund vhosts (no DNS change)
fxhnt-ingress proxy (nginx wildcard-TLS + socat SSH + kernel-mode tailscale)
now 3/3 and serving dashboard/cockpit/dagster/git.fxhnt.ai. EVAL P2 PASS:
from the devbox over the tailnet -> proxy tailnet IP 100.93.141.11, all four
hosts return HTTP 200 with a valid *.fxhnt.ai LE cert chain — foxhunt still
fully live, zero DNS change.

Three fixes to get there:
- Added tailscale-fxhnt-ingress SA+Role+RoleBinding (sidecar needs ts-state
  Secret RBAC; mirrors tailscale-dagster).
- Re-applied dagster.yaml (its NP ingress fxhnt-ingress edit was committed
  but never applied on-cluster -> dagster vhost hung).
- Added non-headless gitea-clusterip Service (ns gitea, 10.32.x): the Helm
  gitea-http/gitea-ssh are headless -> resolve to 100.64.x pod IPs the
  kernel-mode TS sidecar can't reach (CGNAT overlap). nginx+socat retarget it.

C4 DNS target recorded as 100.93.141.11 in the runbook. Phase-1 prep complete.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 20:01:11 +00:00
ce1c978d4b feat(migrate): P1 wildcard cert + cross-project DNS-01 IAM fix; P2 fxhnt-ingress proxy draft
P1: fxhnt-wildcard Certificate (*.fxhnt.ai) on bizworx. DNS-01 initially
403'd because fxhnt.ai zone is in the foxhunt project (c293eb98) and the
bizworx cert-manager key's DomainsDNSFullAccess was scoped to the bizworx
project only. Domain-move is blocked by Scaleway (external apex 'Root zone
can't be updated'), so fixed via IAM: added a rule granting
DomainsDNSFullAccess scoped to c293eb98 on policy bizworx-prod-runtime-scoped
(scw iam rule create; original rule untouched). Challenge then progressed
past 403 to normal DNS propagation wait.

P2 (draft, not applied): fxhnt-ingress = nginx (wildcard TLS termination) +
socat (git SSH) + tailscale sidecar, fronting the 3 fund vhosts by ClusterIP
(git/dashboard+cockpit/dagster). Uses ClusterIP not pod IPs (bizworx TS
sidecars reach 10.32.x ClusterIPs but not 100.64 pod IPs — CGNAT overlap,
both verified). dagster NP ingress already retargeted to this proxy's label.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 19:44:32 +00:00
9396f53669 fix(migrate): dagster NP apiserver egress (real endpoint) + ingress from fxhnt-ingress
Two corrections found while prepping P2:
1. The earlier 10.32.0.1/32 apiserver-egress fix was WRONG: kube-proxy
   DNATs kubernetes.default.svc (10.32.0.1:443) to the REAL apiserver
   endpoint 172.16.0.11:6443 BEFORE NetworkPolicy eval, and 172.16.0.0/16
   is in the public-HTTPS except list -> still blocked -> tailscale sidecar
   crashlooped again -> pod NotReady -> dagster Service had NO endpoints.
   Correct fix: allow egress to 172.16.0.11:6443 (post-DNAT dest). Now
   3/3 stable, endpoints populate. (apiserver endpoint from
   NAME         ENDPOINTS          AGE
kubernetes   172.16.0.11:6443   85d.)
2. dagster NP ingress referenced foxhunt-only label tailscale-gitlab-proxy;
   retargeted to app.kubernetes.io/name=fxhnt-ingress (the bizworx proxy
   built in P2) so dagster.fxhnt.ai can be fronted.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 19:22:08 +00:00
06d12fd4e3 fix(migrate): allow dagster tailscale sidecar egress to kube-apiserver
The dagster NP's public-HTTPS rule excepts the bizworx service CIDR
(10.32.0.0/16), which contains the apiserver ClusterIP 10.32.0.1. The
tailscale sidecar reads/writes its ts-state Secret via
kubernetes.default.svc, so it timed out (context deadline exceeded) and
crashlooped -> pod stuck 2/3. Add a surgical 10.32.0.1/32:443 egress
allow. daemon+webserver were always healthy; sidecar-only fix -> now 3/3.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 18:15:24 +00:00
30e62675fc fix(migrate): cross-ns gitea NP on all cronjob manifests; apply 7 CronJobs to bizworx (all suspended) 2026-07-18 18:07:01 +00:00
17c36d3afe fix(migrate): dagster cross-ns gitea NP + dagster DB created; T7 core verified (pg+cockpit+dagster up on bizworx) 2026-07-18 18:00:19 +00:00
de6f738719 fix(migrate): pin bizworx postgres to timescaledb 2.25.1-pg16 (match source; latest=2.28.3 broke catalog restore) 2026-07-18 15:09:48 +00:00
bd8ef0fed8 chore(migrate): re-target fund manifests to bizworx (git-sync->bizworx gitea, nodeSelector->large, ib-gateway public image no pull-secret)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 14:56:31 +00:00
jgrusewski
5d584c5dbb infra(cockpit): merge weekly backtest-refs into the daily bybit precompute CronJob
bybit_sleeve_ret (the cockpit sim's precompute + the recon gate's reconciliation
reference) was refreshed only WEEKLY, via the separate fxhnt-backtest-refs CronJob
(0 2 * * 1). The live forward book updates DAILY, so it ran up to a week ahead of
its reconciliation reference — the recent forward days couldn't be reconciled and
the recon gate silently skips its divergence band without a fresh ref (false-PASS
risk on the real-capital book).

Fold `backtest-refs --all` into the existing DAILY fxhnt-compare-measured-precompute
CronJob (already own-memory 6Gi/8Gi, pinned to the 16GB ci-compile-cpu autoscale
pool, identical egress). Now one daily job runs, in order: bybit-spread-refresh,
compare-measured-precompute (compare_measured_cache), backtest-refs --all
(bybit_sleeve_ret + backtest_summary refs). Both writers run independently (a
compare failure must not starve the recon ref); the Job still fails if either does.

NOT compute-once: the per-coin (compare) and per-sleeve (refs) paths stay separate
computations — co-scheduled only, so the numbers are unchanged. Retires the weekly
fxhnt-backtest-refs.yaml (its generic report-ref dispatch is covered by the same
`backtest-refs --all`; no live tooling references its name).

Apply is manual (deploy skips CronJob manifests):
  kubectl apply -f infra/k8s/jobs/fxhnt-compare-measured-precompute-cronjob.yaml
  kubectl delete cronjob fxhnt-backtest-refs -n foxhunt
  kubectl delete networkpolicy fxhnt-backtest-refs -n foxhunt

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 00:37:43 +02:00
jgrusewski
07e6091647 feat(infra): daemon uses ibkr UCITS volume source (real ADV; yahoo lacks CBU0.L)
Validated 2026-07-16: under yahoo the EU equity book drops (Yahoo has no CBU0.L);
under ibkr it sizes 441d / $1M ceiling / Sharpe 1.30 on authoritative on-exchange
volume of the exact traded lines. CronJob keeps ibkr_pit_volume fresh; empty ->
coverage guard -> nav fallback (graceful).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 13:07:00 +02:00
jgrusewski
d96b1ea666 feat(infra): fxhnt-ucits-volume-ibkr CronJob (real UCITS ADV from IBKR -> ibkr_pit_volume)
Runs in the multistrat-rebalancer network context (git-sync + pip ib_async +
ib-gateway:4004 ingress-allowed), writes real on-exchange volume to the DB; the
daemon reads it when FXHNT_ALLOCATION_UCITS_VOLUME_SOURCE=ibkr. 08:30 weekdays.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 12:54:00 +02:00
jgrusewski
206cde3ee5 fix(rbac): grant deployments/scale + pods to ci-deploy for the dashboard scale-0->1
The scale-0->1 deploy step (516499a) failed: the argo-workflow SA had
deployments[patch] (for rollout restart) but not deployments/scale (kubectl
scale) nor pods (kubectl wait --for=delete pod). Grant deployments/scale
[patch,update] + pods [get,list,watch]. No outage — it failed before touching
the running dashboard.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 17:36:04 +02:00
jgrusewski
516499aa17 fix(deploy): scale-0->1 the dashboard instead of rollout-restart
The cockpit deploy workflow rollout-restarted deploy/fxhnt-dashboard, which
briefly overlaps two tailscale sessions on the same identity -> session race ->
the external dashboard.fxhnt.ai proxy link goes stale and the cockpit reads
'down' after every deploy (observed 2026-07-15). The dashboard's tailscale
sidecar serves the public URL, so it must be scaled to 0 (clean session
teardown) before a fresh non-overlapping one comes up. git-sync re-pulls src the
same way, so code still updates. dagster keeps rollout restart (no external
proxy). See reference_fxhnt_dashboard_proxy.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 16:31:14 +02:00
jgrusewski
c5a5a0ba24 fix(ucits): resolve UCITS by ISIN on LSEETF (not SMART); $100k envelope
Live probe on DU9600528 (2026-07-15): SMART+ISIN returns 0 results for all
five UCITS lines (SMART grabs the wrong US grey-market listing or nothing);
LSEETF+ISIN+USD resolves each cleanly, and SMART stays in every valid-exchange
set so orders still route. Two fixes: (1) UcitsListing.exchange default
SMART->LSEETF; (2) _resolve_contract qualifies the ISIN WITHOUT a symbol (the
map ticker is cosmetic and differs from the USD line's real symbol: IEF->IDTM,
GLD->IGLN). Map tickers corrected to the USD lines. Rebalancer envelope
1,000,000 -> 100,000. 2004 tests green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 11:36:17 +02:00
jgrusewski
33318eca7c chore(infra): arm the UCITS rebalancer (EU hours), suspend the US leg
The EU-domiciled DU paper account can't trade US ETFs (PRIIPs/KID). Arm the
PRIIPs-legal UCITS leg (fxhnt-ucits-rebalancer, --venue ucits, ISIN-resolved
contracts + real UCITS pricing) on a weekday 09:00 UTC slot inside LSE/Euronext
hours so the UCITS ETFs actually price + fill; suspend the US leg
(fxhnt-ibkr-rebalancer) which only KID-rejects on this account. Paper only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 02:15:15 +02:00
jgrusewski
9c51b754ba fix(infra): ib-gateway fsGroup 1000 so IBC can write jts.ini on the PVC
A fresh tws_settings PVC mounts root-owned; the gnzsnz container runs as the
non-root ibgateway user (UID/GID 1000) -> "Permission denied" writing jts.ini
-> crashloop. fsGroup: 1000 makes the volume group-writable. Verified: new pod
ready, 0 restarts, "IBC: Login has completed".

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 20:36:37 +02:00
jgrusewski
d399982cd1 fix(infra): ib-gateway — fix replicas:0 landmine + in-place daily auto-restart + persist settings
Three fixes to the sole IBKR paper execution gateway:
- replicas 0->1: the manifest said "dev-mode paused" (replicas:0) while the live
  cluster ran 1 — a drift that would scale IBKR execution to ZERO on any re-apply.
- AUTO_RESTART_TIME "8:00 AM" UTC + TIME_ZONE Etc/UTC: IBC restarts the gateway
  IN-PLACE daily ("does not require daily 2FA validation") instead of the container
  exiting on IBKR's forced reset — the cause of ~13 container restarts + a full
  re-login each time. 08:00 UTC is clear of the 14:35 rebalancer + 23:30 nightly.
- TWS_SETTINGS_PATH + a 1Gi PVC: persist jts.ini/IBC config across pod restarts.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 20:26:22 +02:00
jgrusewski
7e7a657090 docs(infra): correct backtest-refs header — xsfunding/unlock are report-kind
Phase 0b (venue consolidation) made `xsfunding` + `unlock` report-kind,
routed through `_persist_bybit_book`/`_BYBIT_BOOK_SIDS` in this CronJob's
`backtest-refs --all` batch path — NOT inline. The stale header wrongly
listed them (and crypto_tstrend/stablecoin, which aren't in the registry)
among the inline refs. Reword to reflect the actual split: the report-kind
sids this Job covers are bybit_4edge / bybit_4edge_levered / positioning /
xsfunding / unlock; the truly-inline (recompute-replay) refs are only
multistrat / vrp. (This is why unlock had no ref until this Job ran.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 19:46:08 +02:00
jgrusewski
2677bcd02a chore(infra): remove the stale fxhnt-forward CronJob residue
The fxhnt-forward CronJob was superseded by the deterministic-forward-state
Dagster assets (dagster.yaml cutover) and had been suspended/dead since before
0a — its args referenced the long-removed `fetch --crypto` + `forward-track`
CLI. Deleted the orphaned live CronJob + its dead NetworkPolicy (whose egress
still whitelisted Binance for a job that no longer runs), removed the netpol
stanza from fxhnt-cockpit.yaml, and refreshed the now-stale RBAC comment.
The batch/cronjobs RBAC rule stays (still used by the bybit/ibkr rebalancers +
compare-measured-precompute). Live objects already deleted via kubectl.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 17:16:09 +02:00
jgrusewski
49e30b17ea chore(0b): scrub residual Alpaca labels (task 5 review minors — stale rebalancer comment + test fixtures)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 22:24:07 +02:00
jgrusewski
4cf6de17ee feat(0b): remove dead Alpaca execution; IBKR is the sole equity executor (Broker port kept)
Alpaca's rebalancer CronJob has been suspended with 401'd keys since Task 4's
venue consolidation; IBKR paper is already the live forward-track venue for
the multi-strat book. This removes the dead code/config/manifests instead of
leaving them to rot, and relabels the multistrat/multistrat_exec registry
venues from the vestigial "alpaca-paper" to "ibkr-equity" (they execute on
IBKR paper, not Alpaca — exec_venue itself is still computed dynamically at
runtime).

- Delete src/fxhnt/adapters/broker/alpaca.py, AlpacaSettings + the `alpaca`
  config field; move the one real setting it carried (min_order_notional,
  a generic "skip sub-min-notional dust" floor for any future
  fractional-capable broker) onto ExecutionSettings.
- Remove the `--broker` option and its alpaca branches from `execute` and
  `execute-multistrat` (cli.py) — IBKR is now implicit; IbkrBroker was
  already the module-level import, so the branch was pure dead code once
  Alpaca is gone.
- Delete infra/k8s/jobs/fxhnt-alpaca-rebalancer.yaml and
  infra/k8s/secrets/alpaca-credentials.yaml; delete the superseded
  infra/k8s/services/multistrat-rebalancer.yaml (PVC + NetworkPolicy +
  CronJob), replaced by fxhnt-ibkr-rebalancer. Drop the now-invalid
  `--broker ibkr` argument from fxhnt-ibkr-rebalancer.yaml and
  fxhnt-ucits-rebalancer.yaml (the flag no longer exists).
- Delete tests/integration/test_alpaca_broker.py (tested the deleted
  adapter directly); add tests/unit/test_no_alpaca.py as a standing guard
  (no "alpaca" substring anywhere in src/fxhnt + IbkrBroker still imports).
- Update test_registry_multistrat_exec.py + test_cockpit_ssot.py (venue
  label) and test_cli_execute_helpers.py (drop the broker_name kwarg and
  AlpacaBroker fake, use IbkrBroker instead) to match.

MANUAL cluster ops (not run by CI, record only):
  kubectl delete cronjob/fxhnt-alpaca-rebalancer secret/alpaca-credentials -n foxhunt
  kubectl delete cronjob/multistrat-rebalancer networkpolicy/multistrat-rebalancer pvc/multistrat-state -n foxhunt

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 22:17:04 +02:00
jgrusewski
b0901a8b1d chore(backtest): park B3 equity + carry file-stores; guard live feature store = Timescale
Step 1 investigation (verified, no code migration needed): the live paper
path (paper_book.py, paper_sleeves.py, paper_backfill.py) has ZERO
DuckDbFeatureStore/.duckdb/read_parquet usage — it only consumes a store via
get_feature_store(). The only non-test DuckDbFeatureStore constructor is
get_feature_store() itself (adapters/warehouse/__init__.py), gated behind the
explicit feature_store == "duckdb" opt-in; the default feature_store="postgres"
(config.py) returns TimescaleFeatureStore. DuckDB is dev/test/opt-in-only on
the live path — no migration required.

- Add tests/unit/test_no_live_duckdb_files.py: asserts Settings().feature_store
  defaults to "postgres" and get_feature_store() returns TimescaleFeatureStore,
  not DuckDbFeatureStore, locking the safety invariant noted in config.py (a
  Job that forgets FXHNT_FEATURE_STORE can't silently write a throwaway DuckDB
  file). Uses a sqlite DSN (matching every other TimescaleFeatureStore test in
  this repo) so the unit test needs no live Postgres server.
- Retire the orphaned B3 platform Jobs/PVC (no kustomization/script referenced
  them): infra/k8s/jobs/fxhnt-backtest-ingest-job.yaml,
  fxhnt-backtest-verdict-job.yaml, infra/k8s/storage/fxhnt-backtest-pvc.yaml.
- Mark equity_backtest_runner.py PARKED in its module docstring (no live
  consumer; revivable — plugs into backtest_refs when an equity strat is
  added to the registry). Logic unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 12:50:32 +02:00
jgrusewski
dbc37900cf docs(backtest): correct liquid-universe-bound attribution in backtest-refs CronJob comment
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 12:41:24 +02:00
jgrusewski
83a0e29894 fix(backtest): unify bybit-book scheduled writer — backtest-refs --all writes bybit_sleeve_ret + refs via shared compute-once helper (no cockpit-data regression) 2026-07-13 12:37:49 +02:00
jgrusewski
c38f3372ab feat(backtest): route nightly refs through backtest_ref + scheduled resourced backtest-refs CronJob (bug B)
Refactor _persist_track_backtest_ref (assets.py) to delegate to the unified backtest_ref dispatcher
instead of re-implementing the recompute-replay path inline, and wire xsfunding's missing sleeve-kind
ref through it (bug A's asset half) using the Binance-scoped feature store. Replace the manual
fxhnt-bybit-sleeve-ret Job with a weekly-scheduled fxhnt-backtest-refs CronJob running
`fxhnt backtest-refs --all` (bug B) so the report-kind refs (bybit_4edge/bybit_4edge_levered/positioning)
no longer depend on a human remembering to apply a one-off Job.
2026-07-13 12:12:47 +02:00