feat(migrate): T8 cutover C1-C5 EXECUTED — fxhnt.ai now serves from bizworx

Cutover complete, foxhunt frozen-but-intact (rollback available):
- C1 froze foxhunt fund writers (cronjobs suspended, dagster/cockpit/
  ib-gateway scaled to 0); infra cronjobs left running.
- C2 final delta pg_dump/restore fxhnt -> bizworx. --clean over the existing
  rehearsal copy broke the tsdb catalog; recovered via drop+recreate empty DB
  then clean restore (exit 0, 0 errors). All row counts match frozen source
  (identical to rehearsal = zero real delta). Lesson: never --clean over a
  live timescaledb DB.
- C3 brought bizworx fund live: cockpit serves real migrated data, ib-gateway
  1/1 stable (foxhunt released the IBKR session), dagster 3/3.
- C4 flipped Scaleway fxhnt.ai dashboard/cockpit/dagster/git A-records ->
  100.93.141.11 (bizworx fxhnt-ingress). Verified over real public DNS:
  HTTP 200 + valid cert; git=bizworx Gitea, dashboard=cockpit w/ data, SSH
  ls-remote returns real refs.
- C5 armed only fxhnt-ucits-rebalancer (next run 09:00 UTC); rest suspended.

Rollback until T11 = flip DNS back to 100.95.225.27 + un-suspend/scale-up
foxhunt. Next: T9 verify, T10 observe, then triage/teardown. DELETE NOTHING.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-18 20:16:36 +00:00
parent e67c6e1612
commit 05c209b64c

View File

@@ -1,5 +1,38 @@
# Foxhunt→Bizworx Consolidation — Live Progress (resume anchor)
## 🚦 T8 CUTOVER EXECUTED (2026-07-18 ~20:15 UTC) — fxhnt.ai NOW SERVES FROM BIZWORX ✅
**The fund has cut over. foxhunt is FROZEN but INTACT (rollback available). DELETE NOTHING until T10 observation passes.**
- **C1 (freeze foxhunt):** suspended fund cronjobs (ucits-rebalancer, ucits-volume-ibkr, compare-measured-precompute)
+ scaled foxhunt dagster/fxhnt-dashboard/ib-gateway → 0/0. Infra cronjobs (postgres-backup/pvc-autoscaler/
billing-scraper) LEFT RUNNING (they die at teardown, not now). PASS.
- **C2 (delta DB migrate):** pg_dump -Fc fxhnt (47.8M) from frozen foxhunt → restore into bizworx. GOTCHA:
`pg_restore --clean --if-exists` over the existing rehearsal copy FAILED (dropped _timescaledb_internal schema
mid-restore → 1136 errors, DB stuck restoring:on). FIX (rehearsal-proven path): DROP+CREATE empty fxhnt DB,
then pre_restore → pg_restore (NO --clean) → post_restore = exit 0, 0 errors. EVAL C2 PASS: all counts match
frozen source (forward_nav 106, backtest_summary 6, bybit_sleeve_ret 7023, xsp_option_bars 814373, exec_fills 9,
ibkr_account_nav 4, forward_anchor 20; max_anchor_t0 2026-07-14, max_fwd_nav 2026-07-17; 37 tables/3 hypertables).
NOTE: counts were IDENTICAL to the rehearsal (fund already suspended during parallel work → zero real delta).
LESSON: NEVER pg_restore --clean over a live timescaledb DB; always restore into a fresh empty DB.
- **C3 (bizworx fund live):** scaled dagster/cockpit/ib-gateway→1 + rollout-restart cockpit&dagster (fresh DB
conns). EVAL C3 PASS: cockpit serves real migrated data over the proxy (/ + /strategy/multistrat + /xsfunding
= 200, shows PASS/WAIT/Sharpe/multistrat/xsfunding); ib-gateway 1/1 STABLE (~12min, foxhunt released IBKR
session); dagster 3/3, ingress 3/3, postgres 1/1.
- **C4 (DNS flip):** Scaleway fxhnt.ai records dashboard/cockpit/dagster/git A → **100.93.141.11** (bizworx
fxhnt-ingress), TTL 60 (was 100.95.225.27 foxhunt proxy). SYNTAX: `scw dns record set dns-zone=fxhnt.ai
name=<n> type=A ttl=60 values.0=100.93.141.11` (values.0=, NOT data=). EVAL C4 PASS over REAL public DNS:
dashboard/git/dagster.fxhnt.ai → 100.93.141.11, HTTP 200, tls_verify=0. Content confirmed: git=bizworx Gitea,
dashboard=cockpit "Overview · fxhnt" w/ real data, SSH `git ls-remote ssh://git@git.fxhnt.ai/gitadmin/fxhnt.git`
returns real refs (HEAD d66d1132) via socat path. (SSH hostkey-changed warning = correct, it's bizworx now.)
- **C5 (arm fund):** un-suspended ONLY fxhnt-ucits-rebalancer on bizworx (schedule 0 9 * * 1-5 = 09:00 UTC wkdays;
next run tomorrow AM). US(ibkr)/bybit/vrp/factory/ucits-volume all stay SUSPENDED. EVAL C5 (first armed run
green) is OBSERVABLE at next 09:00 UTC — deps verified (DB+ib-gateway+git-sync all live).
- **ROLLBACK (still trivial until T11):** `scw dns record set ... values.0=100.95.225.27` (back to foxhunt proxy)
+ `kubectl --context $FOX -n foxhunt scale deploy/dagster deploy/fxhnt-dashboard deploy/ib-gateway --replicas=1`
+ un-suspend foxhunt fund cronjobs. foxhunt Postgres still holds pre-cutover state.
- **NEXT:** T9 verify (cockpit LOCAL browser + prod, gate/forward-track intact) → T10 observation window (watch
the first ucits run + a day) → T11 Phase-2 triage → T12 teardown. **ROTATE the pasted tailscale key.** DELETE NOTHING.
Branch `chore/foxhunt-bizworx-consolidation`. Real date 2026-07-18. Plan:
`docs/superpowers/plans/2026-07-18-foxhunt-to-bizworx-consolidation.md`. Contexts:
`FOX=foxhunt-34a1e3c4-…`, `BIZ=bizworx-prod-kapsule-05e1dae3-…` (both in ~/.kube/config).