Commit Graph

1164 Commits

Author SHA1 Message Date
dcf333d97e Merge: UCITS live-exec 478/10349 + false-EXECUTED fix + China/India Stage-1 kill
- fix/ucits-exec: Error 478 (strip ISIN/localSymbol, route by conId), Error
  10349 (explicit TIF=DAY), false-EXECUTED (executed requires all legs filled;
  degraded exits 1 + exec-record skips booking so the gate stays WAIT).
- research(em-asia): China/India CNYA.L/NDIA.L KILLED at the Stage-1 LOO
  marginal-Sharpe gate (co-crashing EM beta); reusable em_asia_marginal_eval.py.

523 exec/forward/gate tests green, mypy clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 09:46:27 +00:00
0ed2cfdff3 fix(exec): UCITS live-run 478/10349 + false-EXECUTED (Mon 2026-07-20)
The first live UCITS paper run on bizworx reported EXECUTED/exit-0 but was
degraded: the IEF->CBU0 leg got 0 fills and DBMF hung Submitted.

- Error 478 (contract conflict): the ISIN-qualified UCITS contract kept a
  localSymbol/secId conflicting with the pinned conId (requested CBU0 vs
  canonical CSBGU0). Strip secId/secIdType/localSymbol after resolve and route
  by conId + SMART.
- Error 10349 (TIF preset): the bare MarketOrder had no TIF so the account
  preset forced DAY + cancel/reconfirm. Set explicit tif="DAY".
- False-success (the dangerous one): plan.executed counted Cancelled/Submitted
  legs as placed, and exec-record booked on any non-blocked run. Now executed
  requires EVERY leg filled; a degraded run prints DEGRADED, exits 1, and
  skips exec-track booking so the reconciliation gate stays WAIT (never books a
  phantom rebalance) — real money errs false-WAIT.

Tests: new test_ucits_exec_fill_status.py + degraded cases in test_execution.py
and test_execute_multistrat_b2a.py (TDD failing-first). 523 exec/forward/gate
tests green, mypy clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 09:20:47 +00:00
a00001e826 research(em-asia): Stage-1 LOO marginal-Sharpe kill-switch — China/India KILLED
Backtest-only gate (no forward wiring) for the China/India diversifier question,
decided from ~7y of real Yahoo data. Reuses the SSOT marginal_sharpe primitive +
multistrat.book_series verbatim; cost model = max(vol-norm re-sizing turnover,
monthly-rebalance floor) so a quiet co-crasher can't survive at ~0 modeled cost.

Result (10bp UCITS, 2019-08..2026-07): CNYA.L marginal Sharpe -0.099, NDIA.L
-0.048 — both co-crash the book (+0.23 corr) and worsen with cost. KILL both:
the accessible UCITS wrapper carries EM beta, not the mainland inefficiency
(direct A-shares/NSE closed to the EU-retail entity). Same co-crash death as VRP
and unlock. Do NOT wire the em_asia forward sleeve.

Gate is generic — reusable for any single-instrument diversifier candidate.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 22:56:30 +00:00
4e7adf42f2 docs(migrate): scheduling-consistency upgrade complete + verified
All-Dagster K8sRunLauncher upgrade done: nightly ETL+precompute runs in
per-run 6-8Gi Jobs (daemon never OOMs), bybit_book_precompute asset folds in
the compare-measured/backtest-refs crons, execution stays in crons. Verified
end-to-end (run SUCCESS, daemon 0 restarts, all data fresh 2026-07-19). VRP +
compare-measured crons deleted; factory + ucits armed. Documented.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 10:14:28 +00:00
1ecf3983d9 chore(scheduling): delete compare-measured + vrp crons; arm factory + ucits-volume
compare-measured-precompute is now the Dagster bybit_book_precompute asset
(K8sRunLauncher, own memory) — cron + NP deleted. vrp-rebalancer deleted
(shelved/falsified; vrp_nav de-wired from the nightly). Armed factory +
ucits-volume-ibkr in the manifests (source of truth) to match the live state;
bybit-rebalancer (no testnet creds) + ibkr-rebalancer (PRIIPs) stay suspended.
Final: Dagster owns all nightly ETL+precompute; K8s CronJobs own execution only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 10:12:45 +00:00
06156bdb1a Merge: foxhunt→bizworx consolidation + all-Dagster K8sRunLauncher scheduling
Promotes the full consolidation branch to master (production ref the fund
git-syncs): DB/gitea/DNS migration to bizworx, the fxhnt-ingress proxy, the
.dbn archive work, and the scheduling-consistency upgrade — Dagster now owns
all nightly ETL+precompute via K8sRunLauncher (per-run 6-8Gi Jobs), with the
new bybit_book_precompute asset folding in the compare-measured/backtest-refs
crons. Execution stays in K8s CronJobs (blast-radius isolation).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 10:01:53 +00:00
3461f0e331 feat(orchestration): K8sRunLauncher instance config (per-run Jobs, own memory)
Configure K8sRunLauncher in the dagster-instance ConfigMap: each run launches
as its own K8s Job so the heavy bybit_book_precompute asset (6-8Gi op_tags)
runs isolated, never OOMing the 4Gi daemon. Launched Jobs git-sync the same
code (initContainer + emptyDir /code, same gitea+key), get the fund env +
secrets (DSN/PGPASSWORD/databento/tiingo), and need NO PVC (compute is
Postgres-only; unlock calendar is DB-first, so the RWO surfer PVC stays with
the daemon — no deadlock). Validated: DagsterInstance.from_config loads it as
K8sRunLauncher. No extra NP needed (no postgres ingress NP + no default-deny,
so launched Jobs reach postgres/gitea with open egress).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 09:48:47 +00:00
5b016b9c4d feat(orchestration): dagster K8sRunLauncher RBAC + point at dagster-k8s image
Add Jobs/pods/events RBAC to the tailscale-dagster SA (K8sRunLauncher launches
each run as its own Job) and point both dagster containers at the new
fxhnt-cockpit:dagster-k8s-25e3d7c image (dagster_k8s 0.28.22 baked in). The
K8sRunLauncher instance config comes next once the daemon is verified on the
new image.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 09:45:12 +00:00
04305a1c72 feat(orchestration): fold bybit precompute into a Dagster asset (K8sRunLauncher-ready)
Move _persist_bybit_book + _BYBIT_BOOK_SIDS to application/bybit_book_persist.py
(single source of truth; breaks the cli<->assets coupling). Add a new nightly
asset bybit_book_precompute (deps=[bybit_warehouse_refresh]) that wraps the
same helper backtest-refs/bybit-persist-sleeve-ret run — writing bybit_sleeve_ret
+ the report-kind gate refs in one compute-once pass. Tagged
dagster-k8s/config 6Gi req / 8Gi limit so K8sRunLauncher runs it in its own
right-sized Job (never the 4Gi daemon). Wired into combined_book_job + defs;
tests updated (13->14 assets + presence + wiring). CLI commands left intact
(deleted at the cron layer later). 60 tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 09:36:05 +00:00
25e3d7ca85 build(deps): add dagster-k8s to orchestration extra (K8sRunLauncher)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 09:19:50 +00:00
18cdf3f704 docs(plan): rewrite for all-Dagster K8sRunLauncher + proper podman image build
Operator: go all-Dagster properly + build the image right. Diagnosis
corrected: Dagster ETL was healthy all along (23:30 run SUCCESS, 14 assets,
forward_nav fresh); the stale data is the SUSPENDED compare-measured cron that
writes bybit_sleeve_ret. That cron exists because the compute needs 6-8Gi and
OOM-killed the 4Gi daemon — so the correct all-Dagster fix is K8sRunLauncher
(per-run Jobs with own memory), not daemon-run assets. Plan: add dagster-k8s,
build+push via podman (Argo/Kaniko pipeline is dead), add Job-launch RBAC,
configure K8sRunLauncher + per-asset memory, fold sleeve-ret/backtest-refs into
the DAG, materialize+verify, then delete compare-measured+VRP crons and arm
execution. Also commits the harmless working_directory=/code/src fix.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 09:13:33 +00:00
22a1a404ba docs(plan): production scheduling consistency — fix dagster stall + arm crons + drop VRP
Diagnosed why bybit went stale over the weekend: the Dagster code-server is
crash-looping (workspace.yaml working_directory=/app but git-sync code is at
/code/src -> no heartbeat -> 23:30 nightly never fires since Sat), AND all
fund crons are suspended from the cutover. Architecture is a deliberate
two-layer split (Dagster ETL @23:30 7-day + K8s cron execution) — operator
chose to keep it and make it consistent. Plan: fix the code-server dir,
catch-up materialize, arm compare-measured/factory/ucits-volume, delete the
dead VRP cron (shelved/falsified) + document. Real-money: only ucits trades.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 08:55:20 +00:00
fa6c0c11f3 chore(migrate): .dbn S3 archive complete + verified (80 obj/171GiB, 0 diffs); drain resumed
The .dbn.zst S3 upload finished and verified byte-for-byte vs source
(rclone check --size-only: training 0 diffs/59 files, test 0 diffs/21 files;
80 objects total = source count). Two independent verified archives now exist
(S3 object storage + block snapshots, both in bizworx project, both survive
teardown). Cleaned up uploader/speedtest pods + s3 secrets; resumed the
foxhunt platform pool drain to 0. Compute heading to ~$0.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 02:41:59 +00:00
dce4a72c45 docs(migrate): correct 'stray S3 junk' note — no junk, bucket IS the active archive
The '132MB / 23 objects' were the first partial .dbn transfers, not garbage;
the resumed upload continues them. Verified singletest/+speedtest/ test files
already deleted (0 objects), bucket holds only legit training/*.dbn*. Marked
DO NOT wipe fxhnt-dbn-archive so it isn't mistaken for cleanable later.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 22:12:44 +00:00
2986c0d940 chore(migrate): background S3 copy of .dbn.zst running; keep snapshots as safety net
Operator wants the compressed .dbn.zst copied to S3 too (browsable archive
alongside the block snapshots). Diagnosed the earlier stall: node egress
~10-17MB/s (not compression), and too many concurrent big-chunk streams
choked the dev1_l node. Retuned to --transfers 2 --s3-chunk-size 32M and it's
moving (137GiB, ETA ~6-7h). Uploading directly from a foxhunt pod mounting the
live PVCs (CSI import of restored volumes was a rabbit hole). Persistent
monitor watching completion. Snapshots kept as the validated safety net until
the S3 copy is verified. Pool held at 1 node for the uploader.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 21:54:51 +00:00
aacc7eff8f chore(migrate): archive 171GB .dbn via block snapshots (rclone upload was dead end)
The rclone S3 upload stalled on the starved foxhunt cluster (dev1_l egress +
slow bssd read -> 76KiB/s, ETA 4 days). Pivoted to Scaleway block-volume
snapshots: instant, storage-layer, restorable, no tunnel. Snapshotted both
.dbn PVCs (training 537GB, test 50GB) into the bizworx project so they survive
teardown; both available in ~20s. Cleaned up uploader/inspector pods + s3
secret; resumed the node drain to 0. Stray fxhnt-dbn-archive bucket (132MB
junk) noted for later cleanup. Surfer .dbn already migrated + verified.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 21:15:38 +00:00
2438294f48 chore(migrate): migrate surfer .dbn to bizworx + archive 171GB Rust-ML .dbn to private S3
Operator caught databento .dbn data not covered by the pg migration.
- Surfer 22 .dbn (fund tsmom/carry research universe) migrated into the
  bizworx fxhnt-surfer-data PVC; verified from the dagster pod (/data/surfer/
  *.dbn + state JSONs). Research assets would've failed on the empty PVC.
- 171GB archived Rust-ML .dbn (training+test PVCs, not fund-used) uploading
  to a new PRIVATE Scaleway bucket fxhnt-dbn-archive via an in-cluster rclone
  pod — intra-Scaleway (fr-par pod -> fr-par S3), no tunnel (can't move 175GB
  over the tailscale hop). S3 creds in a foxhunt secret, never printed.
Bumped platform pool min-size 0->1 so the inspector/uploader can schedule;
resume the drain to 0 after the upload completes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 21:12:19 +00:00
33c1f62918 chore(migrate): shut down foxhunt compute (pool->0), cluster+PVCs intact
Operator confirmed migration complete + accepted dropping the instant-rollback.
Verified bizworx has zero runtime dep on foxhunt, the pre-cutover dump is
backed up off-cluster (~/Work/fxhnt-cutover-backup, 805 objects), and the fund
is live on bizworx. Scaled all foxhunt workloads (incl postgres/gitea/proxy)
+ cert-manager/tailscale to 0, suspended all cronjobs, set platform pool
min-size=0 so the autoscaler drains the 3 dev1_l nodes to 0. Cluster + all
PVCs preserved (not destroyed) — reversible by scaling back up; terraform
destroy deferred to T12. Rollback is now dump-restore, not instant DNS flip.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 20:52:41 +00:00
6ff522cf7c verify(migrate): T9 visual browser pass — cockpit renders migrated DB in real Chrome
Installed the playwright chrome channel and drove the local cockpit (bizworx
migrated DB) in a real browser. Full render confirmed: overview forward-tracks
with correct gates (sixtyforty GO, unlock PASS, bybit/multistrat WAIT), 4-edge
book $832,534, and the /paper IBKR card showing the real DU account $1,029,015
(not the 'not connected' empty-state). Matches DB cross-checks. T9 complete.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 20:42:21 +00:00
1240113d33 verify(migrate): T9 cockpit + forward-track verification against migrated DB
Local cockpit (dev-cockpit-local.sh, bizworx context) serves the migrated DB
read-only; numbers cross-checked against the DB: /paper IBKR card $1,029,015
== ibkr_account_nav.nlv latest, bybit Sh 2.11 / unlock Sh 1.03 == backtest_summary.
The /paper IBKR card renders the real DU account (not the 'not connected'
empty-state that degraded in a prior session). Forward tracks intact: anchor
t0 dates are original historical inceptions, not re-incepted to today — DB-anchor
SSOT survived the migration; recon gate recomputes correctly.

Also fixed dev-cockpit-local.sh to pass --extra web --extra pg (bare uv run
failed ModuleNotFoundError: uvicorn/psycopg on a fresh env).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 20:28:56 +00:00
05c209b64c feat(migrate): T8 cutover C1-C5 EXECUTED — fxhnt.ai now serves from bizworx
Cutover complete, foxhunt frozen-but-intact (rollback available):
- C1 froze foxhunt fund writers (cronjobs suspended, dagster/cockpit/
  ib-gateway scaled to 0); infra cronjobs left running.
- C2 final delta pg_dump/restore fxhnt -> bizworx. --clean over the existing
  rehearsal copy broke the tsdb catalog; recovered via drop+recreate empty DB
  then clean restore (exit 0, 0 errors). All row counts match frozen source
  (identical to rehearsal = zero real delta). Lesson: never --clean over a
  live timescaledb DB.
- C3 brought bizworx fund live: cockpit serves real migrated data, ib-gateway
  1/1 stable (foxhunt released the IBKR session), dagster 3/3.
- C4 flipped Scaleway fxhnt.ai dashboard/cockpit/dagster/git A-records ->
  100.93.141.11 (bizworx fxhnt-ingress). Verified over real public DNS:
  HTTP 200 + valid cert; git=bizworx Gitea, dashboard=cockpit w/ data, SSH
  ls-remote returns real refs.
- C5 armed only fxhnt-ucits-rebalancer (next run 09:00 UTC); rest suspended.

Rollback until T11 = flip DNS back to 100.95.225.27 + un-suspend/scale-up
foxhunt. Next: T9 verify, T10 observe, then triage/teardown. DELETE NOTHING.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 20:16:36 +00:00
e67c6e1612 feat(migrate): P2 complete — fxhnt-ingress proxy verified for all 3 fund vhosts (no DNS change)
fxhnt-ingress proxy (nginx wildcard-TLS + socat SSH + kernel-mode tailscale)
now 3/3 and serving dashboard/cockpit/dagster/git.fxhnt.ai. EVAL P2 PASS:
from the devbox over the tailnet -> proxy tailnet IP 100.93.141.11, all four
hosts return HTTP 200 with a valid *.fxhnt.ai LE cert chain — foxhunt still
fully live, zero DNS change.

Three fixes to get there:
- Added tailscale-fxhnt-ingress SA+Role+RoleBinding (sidecar needs ts-state
  Secret RBAC; mirrors tailscale-dagster).
- Re-applied dagster.yaml (its NP ingress fxhnt-ingress edit was committed
  but never applied on-cluster -> dagster vhost hung).
- Added non-headless gitea-clusterip Service (ns gitea, 10.32.x): the Helm
  gitea-http/gitea-ssh are headless -> resolve to 100.64.x pod IPs the
  kernel-mode TS sidecar can't reach (CGNAT overlap). nginx+socat retarget it.

C4 DNS target recorded as 100.93.141.11 in the runbook. Phase-1 prep complete.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 20:01:11 +00:00
ce1c978d4b feat(migrate): P1 wildcard cert + cross-project DNS-01 IAM fix; P2 fxhnt-ingress proxy draft
P1: fxhnt-wildcard Certificate (*.fxhnt.ai) on bizworx. DNS-01 initially
403'd because fxhnt.ai zone is in the foxhunt project (c293eb98) and the
bizworx cert-manager key's DomainsDNSFullAccess was scoped to the bizworx
project only. Domain-move is blocked by Scaleway (external apex 'Root zone
can't be updated'), so fixed via IAM: added a rule granting
DomainsDNSFullAccess scoped to c293eb98 on policy bizworx-prod-runtime-scoped
(scw iam rule create; original rule untouched). Challenge then progressed
past 403 to normal DNS propagation wait.

P2 (draft, not applied): fxhnt-ingress = nginx (wildcard TLS termination) +
socat (git SSH) + tailscale sidecar, fronting the 3 fund vhosts by ClusterIP
(git/dashboard+cockpit/dagster). Uses ClusterIP not pod IPs (bizworx TS
sidecars reach 10.32.x ClusterIPs but not 100.64 pod IPs — CGNAT overlap,
both verified). dagster NP ingress already retargeted to this proxy's label.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 19:44:32 +00:00
9396f53669 fix(migrate): dagster NP apiserver egress (real endpoint) + ingress from fxhnt-ingress
Two corrections found while prepping P2:
1. The earlier 10.32.0.1/32 apiserver-egress fix was WRONG: kube-proxy
   DNATs kubernetes.default.svc (10.32.0.1:443) to the REAL apiserver
   endpoint 172.16.0.11:6443 BEFORE NetworkPolicy eval, and 172.16.0.0/16
   is in the public-HTTPS except list -> still blocked -> tailscale sidecar
   crashlooped again -> pod NotReady -> dagster Service had NO endpoints.
   Correct fix: allow egress to 172.16.0.11:6443 (post-DNAT dest). Now
   3/3 stable, endpoints populate. (apiserver endpoint from
   NAME         ENDPOINTS          AGE
kubernetes   172.16.0.11:6443   85d.)
2. dagster NP ingress referenced foxhunt-only label tailscale-gitlab-proxy;
   retargeted to app.kubernetes.io/name=fxhnt-ingress (the bizworx proxy
   built in P2) so dagster.fxhnt.ai can be fronted.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 19:22:08 +00:00
0d3fca6b0c docs(migrate): T8 cutover runbook + DNS/TLS ingress map
Mapped the full fxhnt.ai ingress: all 13 A-records -> 100.95.225.27
(tailscale-gitlab-proxy pod, ns foxhunt = nginx vhosts + socat + tailscale)
which DIES at teardown, so DNS MUST move. Only git + dashboard/cockpit +
dagster are fund-critical (operator: grafana can go too; api/mail/chat/minio
legacy). Fund pipeline never uses fxhnt.ai (in-cluster git-sync + ClusterIP)
-> DNS flip only affects browser access, low-risk.

TLS resolved: bizworx already has cert-manager + scaleway DNS-01 webhook +
letsencrypt-prod issuer -> *.fxhnt.ai wildcard is just a Certificate. Chosen
cutover = stripped fund-only nginx+tailscale proxy on bizworx fronting the 3
vhosts by ClusterIP; DNS points at its tailnet IP. Written full executable
runbook (P1-P2 pre-flight, C1-C5 window) with VERIFY+ROLLBACK per step.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 19:14:00 +00:00
06d12fd4e3 fix(migrate): allow dagster tailscale sidecar egress to kube-apiserver
The dagster NP's public-HTTPS rule excepts the bizworx service CIDR
(10.32.0.0/16), which contains the apiserver ClusterIP 10.32.0.1. The
tailscale sidecar reads/writes its ts-state Secret via
kubernetes.default.svc, so it timed out (context deadline exceeded) and
crashlooped -> pod stuck 2/3. Add a surgical 10.32.0.1/32:443 egress
allow. daemon+webserver were always healthy; sidecar-only fix -> now 3/3.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 18:15:24 +00:00
30e62675fc fix(migrate): cross-ns gitea NP on all cronjob manifests; apply 7 CronJobs to bizworx (all suspended) 2026-07-18 18:07:01 +00:00
17c36d3afe fix(migrate): dagster cross-ns gitea NP + dagster DB created; T7 core verified (pg+cockpit+dagster up on bizworx) 2026-07-18 18:00:19 +00:00
edeabc611b docs(migrate): T7 update — tailscale+cockpit 2/2, ib-gateway->DU9600528, dagster PVC fixed 2026-07-18 17:29:27 +00:00
3844a9774d docs(migrate): live progress anchor — T1-T6 done, T7 cockpit verified, tailscale decision pending 2026-07-18 17:21:21 +00:00
de6f738719 fix(migrate): pin bizworx postgres to timescaledb 2.25.1-pg16 (match source; latest=2.28.3 broke catalog restore) 2026-07-18 15:09:48 +00:00
bd8ef0fed8 chore(migrate): re-target fund manifests to bizworx (git-sync->bizworx gitea, nodeSelector->large, ib-gateway public image no pull-secret)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 14:56:31 +00:00
c182c0209a docs: foxhunt->bizworx consolidation implementation plan (verify-first, per-step eval, teardown last)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 14:22:43 +00:00
543129a248 docs: foxhunt->bizworx platform consolidation design (Phase 1 fund+gitea, Phase 2 triage, Phase 3 teardown)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 14:19:19 +00:00
jgrusewski
d66d11325b Merge: multistrat self-cleans superseded lines (IDTM->CBU0) 2026-07-18 11:30:00 +02:00
jgrusewski
38b8ff26d2 feat(exec): multistrat self-cleans its OWN superseded lines (fixes lingering IDTM)
The IEF->CBU0 UCITS remap (515117d, 2026-07-16 12:28) left the paper account
holding IDTM (the old thin Dist line, bought on the 07-15/16 09:00 runs under the
prior IEF->IDTM config). The rebalancer only ever traded its TARGET symbols
(_plan iterates targets.items()), so it never sold the orphaned IDTM — it lingered
in the account NLV and (correctly) tripped the stray-holdings warning.

Fix — scoped, safe self-cleaning:
- superseded_holdings(): the symbols multistrat ITSELF previously traded (per its
  own exec_fills) that are still held but no longer a target — priced at the last
  fill. Scoped to multistrat's own fills, so a position ANOTHER strategy holds on
  the SHARED IBKR account is never touched.
- plan_and_record: adds them at target weight 0 (+ LSEETF/USD spec so a UCITS line
  resolves by symbol to SELL) before the rebalance. Best-effort: a scan failure
  never blocks a real rebalance.
- _plan: a full EXIT (weight 0 on a held line) ALWAYS trades — hysteresis damps
  churn, it must not strand a position we're deliberately closing.

Next UCITS rebalance (09:00) closes IDTM and buys CBU0; the warning clears. Also
prevents future mapping-change legacy from silently accreting in the NLV.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 11:30:00 +02:00
jgrusewski
8d66f37981 Merge: multistrat_levered observe-only shadow + financing sensitivity band 2026-07-18 11:15:55 +02:00
jgrusewski
713424a89d feat(fund): multistrat_levered — observe-only levered shadow of the ETF book + financing sensitivity band
The multistrat ETF book is an UNDER-levered risk-parity book: it runs at ~5.7%
vol vs its own 10% target because leverage is capped at 1x (MAXLEV=1.0), leaving
return on the table. Add a return-defined observe-only shadow (cf.
bybit_4edge_levered) that levers the SAME series to the 10% vol target minus the
HONEST financing drag on the borrowed leg.

- LeveredShadowStrategy (paper_strategies.py): L = min(max_lev, target_vol/vol),
  floored at 1x; levered_ret = L·r − (L−1)·financing/252. Never executed.
- multistrat_levered_nav asset (deps=multistrat_nav) publishes its forward track
  + basis-matched backtest ref like the base; registered in the nightly job.
- multistrat_levered registry entry + SIM_BOOKS (Backtest switcher).
- financing_bps=530 GROUNDED in IBKR's real USD schedule (verified 2026-07-18:
  Fed funds 3.63% + Pro tiered spread → 5.83% <$100k / ~5.3% $100k-1M).
- leverage_financing_sensitivity() + a cockpit BAND on the levered book's
  Backtest view: levered return/vol/Sharpe across 1..6.83% + the break-even
  (= the book's 7.7% return), so the verdict is never hostage to one rate.

Honest finding it surfaces: at any realistic IBKR retail rate (~5-7%) levering
this modest-return book LOWERS risk-adjusted return (Sharpe 1.35 -> ~0.9); it
only pays at institutional financing (~1-3%). The financing drag is what makes
the shadow honest (without it, free leverage would look like 13.5%/Sharpe 1.35).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 11:15:55 +02:00
jgrusewski
8e0016a302 Merge: strip jargon/hardcoded values from cockpit pages 2026-07-18 10:46:51 +02:00
jgrusewski
f6f1c9082b chore(cockpit): strip jargon text blocks + hard-coded values from the pages
Following the /paper blurb removal, sweep the remaining pages:
- Backtest (sim.html): drop the paragraph-long verdict banner; keep a one-line
  plain subtitle ("a what-if on capital & period at real trading cost").
- Measured-cost caption (_sim_result.html): visible text now plain
  ("real trading cost, per coin — fee + spread · avg drag X%"); the technical
  detail (L1 quoted spread / current-snapshot caveat / "not a slider") moves to
  the hover tooltip. IBKR book caption likewise plain.
- Hard-coded "vol-targeted @ 20%" label (Overview, paper_crypto, strategy) ->
  "each shown on its own" (drops the hard-coded 20% + the jargon).

Tests updated to the new plain copy; technical strings that stay in tooltips
(real L1 quoted spread, the snapshot caveat) are still asserted there.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 10:46:51 +02:00
jgrusewski
2055d0366f Merge: drop /paper orientation banner 2026-07-18 10:36:06 +02:00
jgrusewski
ebd31d7c07 chore(cockpit): drop the orientation blurb on /paper — the two cards say it
The Paper-books landing led with a paragraph-long verdict banner ("Two paper
accounts, one job..."). The two book cards below already convey account value,
holdings and status, so the blurb was redundant text. Keep just the h1 + a
one-line subtitle. Removes the two tests that asserted the banner copy.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 10:36:06 +02:00
jgrusewski
c71a1d8a06 Merge: IBKR stray-holdings warning in the cockpit 2026-07-18 10:29:38 +02:00
jgrusewski
9e3ff64e1d feat(cockpit): warn when the IBKR account holds a symbol outside the book
Verifying the IBKR side surfaced an IBIT (bitcoin ETF) position in the DU paper
account on 2026-07-14 — a legacy leftover from the pre-UCITS US book (exec_fills
has zero IBIT fills; the current multistrat only ever traded the 5 UCITS lines).
It was liquidated in the 07-15 UCITS transition, but until then it sat silently
in the account NLV, unattributed to any book.

Add a stray-holdings guard: `_expected_book_symbols()` derives the allowed set
from the SAME two SSOTs the executor uses (multistrat.FUND_INSTRUMENTS + the
UCITS map) — never a hardcoded duplicate. Each holding gets an `in_book` flag;
StrategyDetail carries `stray_holdings` (held, nonzero-qty symbols outside the
set). The holdings view now renders a red "Position outside the book: <syms>"
banner + a ⚠ on the offending rows, so legacy/stray ballast can't inflate the
NLV unnoticed. [] = clean (no warning).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 10:29:38 +02:00
jgrusewski
ecc51cb2db Merge: bybit_sleeve_ret refresh weekly->daily (merge backtest-refs into daily precompute) 2026-07-18 00:39:23 +02:00
jgrusewski
5d584c5dbb infra(cockpit): merge weekly backtest-refs into the daily bybit precompute CronJob
bybit_sleeve_ret (the cockpit sim's precompute + the recon gate's reconciliation
reference) was refreshed only WEEKLY, via the separate fxhnt-backtest-refs CronJob
(0 2 * * 1). The live forward book updates DAILY, so it ran up to a week ahead of
its reconciliation reference — the recent forward days couldn't be reconciled and
the recon gate silently skips its divergence band without a fresh ref (false-PASS
risk on the real-capital book).

Fold `backtest-refs --all` into the existing DAILY fxhnt-compare-measured-precompute
CronJob (already own-memory 6Gi/8Gi, pinned to the 16GB ci-compile-cpu autoscale
pool, identical egress). Now one daily job runs, in order: bybit-spread-refresh,
compare-measured-precompute (compare_measured_cache), backtest-refs --all
(bybit_sleeve_ret + backtest_summary refs). Both writers run independently (a
compare failure must not starve the recon ref); the Job still fails if either does.

NOT compute-once: the per-coin (compare) and per-sleeve (refs) paths stay separate
computations — co-scheduled only, so the numbers are unchanged. Retires the weekly
fxhnt-backtest-refs.yaml (its generic report-ref dispatch is covered by the same
`backtest-refs --all`; no live tooling references its name).

Apply is manual (deploy skips CronJob manifests):
  kubectl apply -f infra/k8s/jobs/fxhnt-compare-measured-precompute-cronjob.yaml
  kubectl delete cronjob fxhnt-backtest-refs -n foxhunt
  kubectl delete networkpolicy fxhnt-backtest-refs -n foxhunt

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 00:37:43 +02:00
jgrusewski
fd2316b1e8 fix(cockpit): Backtest state panel said "active sleeves" for a coin count
Verifying the Backtest page: the scrub state panel labelled n_active "active
sleeves", but n_active counts the instruments HELD that day — the Bybit book's
~64-coin liquid universe (compare_measured_cost sets n_active=[n_coins]*len),
the ETF book's ~5 ETFs — NOT the book's sleeves (its 4 edges). So a 4-edge book
read as having "64 sleeves". Relabelled "active positions", accurate for both
the coin book and the ETF book.

Regression test pins "active positions" (and asserts "active sleeves" absent).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 16:56:06 +02:00
jgrusewski
b1ceee39a5 fix(cockpit): Overview real-trades return was a naked % under the "allocated" column
Verifying the Overview: the IBKR "↳ real trades" sub-row put the real account's
cumulative return (e.g. -0.01%) bare in the borrowed "allocated" column — on
desktop it read as an allocation, and on mobile the card reflow labelled it
"allocated: -0.01%" (actively wrong; it's a return, not capital allocated).

The data was correct and reconciled (real -0.01% vs plan +0.56% => "0.6% behind
plan"); only the labelling was off. Mirrored the exec-twin row's self-labelling:
the return now renders "real -0.01%" inline, and every sub-row cell carries an
accurate data-label (real return / trades / vs plan / cost) so the mobile card
reflow shows the true meaning, not the borrowed column name.

Test pins the labelled form ("real +3.10%"), so the label can't silently drop.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 16:46:03 +02:00
jgrusewski
8bd87e8ab1 fix(cockpit): honest FAIL verdict + plain UCITS holdings on the strategy detail
Two inconsistencies found verifying the IBKR /strategy/multistrat page:

1. A FAILED backtest with a high Sharpe (multistrat: Sharpe 1.7, gate FAIL)
   rendered a GREEN "Validated edge ... backtest passed" banner directly above
   the FAIL badge — the Sharpe>=1.0 fallback overrode the explicit FAIL. The
   verdict now respects the badge: an explicit FAIL is an amber "Backtest
   didn't clear the gate ... treat it as unproven", never a green "passed".
   The backtest Sharpe figure also goes neutral-grey (not alarming red) when
   unvalidated — the number itself (1.7) isn't bad, it just didn't validate.

2. The UCITS lines the EU DU account actually holds (CSPX/CBU0/IDTM/IGLN/ICOM)
   weren't in ASSET_DESCRIPTIONS, so the holdings "what it is" column read
   "other". Mapped each to its plain description (US stocks / US Treasuries /
   Gold / Commodities), same as its US cousin.

Regression tests: a passed=False + Sharpe 1.7 detail renders amber
"didn't clear the gate" and never "backtest passed"/"Validated edge"; UCITS
tickers resolve to plain descriptions, never "other". Verified desktop+mobile.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 16:31:27 +02:00
jgrusewski
ce464f82f4 fix(cockpit): IBKR paper card was falsely "not connected" — it is live via UCITS
The IBKR paper account (DU9600528) IS connected and trading: the armed
fxhnt-ucits-rebalancer (0 9 * * 1-5) books the multi-strat ETF leg through
UCITS lines (EU KID/PRIIPs blocks the US-ETF leg, whose rebalancer is
correctly suspended). Prod already rendered it ($1,029,621, 5 holdings).

Two self-inflicted "not connected" artifacts fixed:
- scripts/dev_cockpit_local.py did NOT wire ibkr_account_repo, so the LOCAL
  dev cockpit always showed the IBKR card empty regardless of real data —
  a misleading local-verification artifact. Now wires every repo prod wires.
- paper.html's new orientation banner hardcoded "not connected yet" (written
  from that stale local view). Made it data-aware (keyed on ibkr_view) so it
  states the live NLV + holdings when trading, "not connected yet" only when
  genuinely empty — it can't drift from the card beneath it.

Adds a regression assertion: connected account => banner says "live now,
trading through UCITS" and never "not connected yet".

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 16:11:41 +02:00