feat(migrate): T8 cutover C1-C5 EXECUTED — fxhnt.ai now serves from bizworx
Cutover complete, foxhunt frozen-but-intact (rollback available): - C1 froze foxhunt fund writers (cronjobs suspended, dagster/cockpit/ ib-gateway scaled to 0); infra cronjobs left running. - C2 final delta pg_dump/restore fxhnt -> bizworx. --clean over the existing rehearsal copy broke the tsdb catalog; recovered via drop+recreate empty DB then clean restore (exit 0, 0 errors). All row counts match frozen source (identical to rehearsal = zero real delta). Lesson: never --clean over a live timescaledb DB. - C3 brought bizworx fund live: cockpit serves real migrated data, ib-gateway 1/1 stable (foxhunt released the IBKR session), dagster 3/3. - C4 flipped Scaleway fxhnt.ai dashboard/cockpit/dagster/git A-records -> 100.93.141.11 (bizworx fxhnt-ingress). Verified over real public DNS: HTTP 200 + valid cert; git=bizworx Gitea, dashboard=cockpit w/ data, SSH ls-remote returns real refs. - C5 armed only fxhnt-ucits-rebalancer (next run 09:00 UTC); rest suspended. Rollback until T11 = flip DNS back to 100.95.225.27 + un-suspend/scale-up foxhunt. Next: T9 verify, T10 observe, then triage/teardown. DELETE NOTHING. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -1,5 +1,38 @@
|
||||
# Foxhunt→Bizworx Consolidation — Live Progress (resume anchor)
|
||||
|
||||
## 🚦 T8 CUTOVER EXECUTED (2026-07-18 ~20:15 UTC) — fxhnt.ai NOW SERVES FROM BIZWORX ✅
|
||||
**The fund has cut over. foxhunt is FROZEN but INTACT (rollback available). DELETE NOTHING until T10 observation passes.**
|
||||
- **C1 (freeze foxhunt):** suspended fund cronjobs (ucits-rebalancer, ucits-volume-ibkr, compare-measured-precompute)
|
||||
+ scaled foxhunt dagster/fxhnt-dashboard/ib-gateway → 0/0. Infra cronjobs (postgres-backup/pvc-autoscaler/
|
||||
billing-scraper) LEFT RUNNING (they die at teardown, not now). PASS.
|
||||
- **C2 (delta DB migrate):** pg_dump -Fc fxhnt (47.8M) from frozen foxhunt → restore into bizworx. GOTCHA:
|
||||
`pg_restore --clean --if-exists` over the existing rehearsal copy FAILED (dropped _timescaledb_internal schema
|
||||
mid-restore → 1136 errors, DB stuck restoring:on). FIX (rehearsal-proven path): DROP+CREATE empty fxhnt DB,
|
||||
then pre_restore → pg_restore (NO --clean) → post_restore = exit 0, 0 errors. EVAL C2 PASS: all counts match
|
||||
frozen source (forward_nav 106, backtest_summary 6, bybit_sleeve_ret 7023, xsp_option_bars 814373, exec_fills 9,
|
||||
ibkr_account_nav 4, forward_anchor 20; max_anchor_t0 2026-07-14, max_fwd_nav 2026-07-17; 37 tables/3 hypertables).
|
||||
NOTE: counts were IDENTICAL to the rehearsal (fund already suspended during parallel work → zero real delta).
|
||||
LESSON: NEVER pg_restore --clean over a live timescaledb DB; always restore into a fresh empty DB.
|
||||
- **C3 (bizworx fund live):** scaled dagster/cockpit/ib-gateway→1 + rollout-restart cockpit&dagster (fresh DB
|
||||
conns). EVAL C3 PASS: cockpit serves real migrated data over the proxy (/ + /strategy/multistrat + /xsfunding
|
||||
= 200, shows PASS/WAIT/Sharpe/multistrat/xsfunding); ib-gateway 1/1 STABLE (~12min, foxhunt released IBKR
|
||||
session); dagster 3/3, ingress 3/3, postgres 1/1.
|
||||
- **C4 (DNS flip):** Scaleway fxhnt.ai records dashboard/cockpit/dagster/git A → **100.93.141.11** (bizworx
|
||||
fxhnt-ingress), TTL 60 (was 100.95.225.27 foxhunt proxy). SYNTAX: `scw dns record set dns-zone=fxhnt.ai
|
||||
name=<n> type=A ttl=60 values.0=100.93.141.11` (values.0=, NOT data=). EVAL C4 PASS over REAL public DNS:
|
||||
dashboard/git/dagster.fxhnt.ai → 100.93.141.11, HTTP 200, tls_verify=0. Content confirmed: git=bizworx Gitea,
|
||||
dashboard=cockpit "Overview · fxhnt" w/ real data, SSH `git ls-remote ssh://git@git.fxhnt.ai/gitadmin/fxhnt.git`
|
||||
returns real refs (HEAD d66d1132) via socat path. (SSH hostkey-changed warning = correct, it's bizworx now.)
|
||||
- **C5 (arm fund):** un-suspended ONLY fxhnt-ucits-rebalancer on bizworx (schedule 0 9 * * 1-5 = 09:00 UTC wkdays;
|
||||
next run tomorrow AM). US(ibkr)/bybit/vrp/factory/ucits-volume all stay SUSPENDED. EVAL C5 (first armed run
|
||||
green) is OBSERVABLE at next 09:00 UTC — deps verified (DB+ib-gateway+git-sync all live).
|
||||
- **ROLLBACK (still trivial until T11):** `scw dns record set ... values.0=100.95.225.27` (back to foxhunt proxy)
|
||||
+ `kubectl --context $FOX -n foxhunt scale deploy/dagster deploy/fxhnt-dashboard deploy/ib-gateway --replicas=1`
|
||||
+ un-suspend foxhunt fund cronjobs. foxhunt Postgres still holds pre-cutover state.
|
||||
- **NEXT:** T9 verify (cockpit LOCAL browser + prod, gate/forward-track intact) → T10 observation window (watch
|
||||
the first ucits run + a day) → T11 Phase-2 triage → T12 teardown. **ROTATE the pasted tailscale key.** DELETE NOTHING.
|
||||
|
||||
|
||||
Branch `chore/foxhunt-bizworx-consolidation`. Real date 2026-07-18. Plan:
|
||||
`docs/superpowers/plans/2026-07-18-foxhunt-to-bizworx-consolidation.md`. Contexts:
|
||||
`FOX=foxhunt-34a1e3c4-…`, `BIZ=bizworx-prod-kapsule-05e1dae3-…` (both in ~/.kube/config).
|
||||
|
||||
Reference in New Issue
Block a user