Commit Graph

205 Commits

Author SHA1 Message Date
jgrusewski
8d4d389096 feat(b2): TiingoFundamentalsClient (daily metrics + statements)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 17:35:51 +02:00
jgrusewski
98e264f14c refactor(b2): tilt uses statistics.median for consistency
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 17:33:11 +02:00
jgrusewski
9e4379a3e8 feat(b2): equity-factor domain (z-score, composite, 3 constructions, book return)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 17:30:01 +02:00
jgrusewski
159b9b1fc8 feat(b2): FundamentalsClient + UniverseSource ports
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 17:27:10 +02:00
jgrusewski
c033607317 docs(b2): implementation plan — equity-factor sleeve (7 tasks)
Ports → domain (z-score/composite/3 constructions, full code) → Tiingo fundamentals+universe adapters →
3 ForwardStrategy services → registry+assets → deploy. TDD, deterministic domain + fake-port tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 17:19:08 +02:00
jgrusewski
5f292deb45 docs(b2): spec — equity-factor sleeve (value+quality+momentum, 3 construction variants)
Cross-sectional multi-factor book on broad liquid US universe (Tiingo fundamentals + 30yr EOD); long-only /
long-short / long-tilt as 3 ForwardStrategy paper-tracks through the gate. Deterministic domain+fake-port
tests. Backtest/DSR layer = separate increment.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 10:32:00 +02:00
jgrusewski
98ce7b3c2b merge: B1 — 6 paper-track strategies as Dagster assets + Tiingo equity data
Faithful ports (sixtyforty/multistrat/gd/funding/crossvenue/poc incl. torch), generic ForwardTracker,
6 assets wired into cockpit_forward, equity sleeves on Tiingo native adjClose (33yr total-return).
327 tests green, mypy clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 08:59:39 +02:00
jgrusewski
c192010c36 deploy(b1): wire TIINGO_API_KEY into the Dagster pod (equity sleeves)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 08:59:34 +02:00
jgrusewski
10c0c4a59f feat(b1): TiingoDailyClient (native adjClose, 33yr); swap equity sleeves Massive->Tiingo
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 08:50:25 +02:00
jgrusewski
17eca26bb3 test(b1): lock multi-dividend back-adjustment compounding in MassiveDailyClient
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 08:38:29 +02:00
jgrusewski
6e547dad80 feat(b1): MassiveDailyClient total-return adapter; swap equity sleeves Yahoo->Massive
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 08:32:51 +02:00
jgrusewski
e100502c17 refactor(b1): paper-track assets swallow only transient errors, not real bugs
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 08:22:17 +02:00
jgrusewski
2e391489bf feat(b1): wire 6 paper-track assets into the Dagster graph
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 08:18:12 +02:00
jgrusewski
6468b756ba feat(b1): surfer poc paper-forward strategy — faithful port with torch + real signal_sweep/pit_sweep
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 08:03:10 +02:00
jgrusewski
559795b3fd refactor(b1): crossvenue single venue snapshot per run (funding+volume one fetch)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 07:22:25 +02:00
jgrusewski
3a0e6cf06e feat(b1): cross-venue funding paper-forward strategy (faithful port)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 07:16:13 +02:00
jgrusewski
e2168ee0ef fix(b1): funding port faithful — qualify on tf30 (30d mean), crypto-native+hedgeable universe
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 07:08:39 +02:00
jgrusewski
84bfb2110e feat(b1): crypto funding-harvest paper-forward strategy (Binance, faithful port)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 07:02:48 +02:00
jgrusewski
a2d7bfe512 docs(b1): note gd QQQ-alignment choice vs foxhunt original
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 06:58:10 +02:00
jgrusewski
6e004f1749 feat(b1): growth-discipline paper-forward blend (composes multistrat, faithful)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 06:54:27 +02:00
jgrusewski
f2aade706a feat(b1): multistrat adaptive paper-forward book (Yahoo, faithful port)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 06:48:53 +02:00
jgrusewski
4cf5733591 refactor(b1): tz-aware fromtimestamp + zip strict in Yahoo template
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 06:44:37 +02:00
jgrusewski
2e7ba4764c feat(b1): sixtyforty paper-forward strategy (Yahoo) — template
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 06:40:41 +02:00
jgrusewski
94dbf79133 feat(b1): market-data client ports
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 06:36:46 +02:00
jgrusewski
f6aecfece4 refactor(b1): tidy ForwardTracker imports + module docstring
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 06:35:31 +02:00
jgrusewski
4f9b6fcc0f feat(b1): generic ForwardTracker over a ForwardStrategy port
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 06:31:10 +02:00
jgrusewski
5da77b6959 docs(b1): poc ported faithfully WITH torch (real signal_sweep/pit_sweep) — no algo changes to dodge a dep
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 06:27:45 +02:00
jgrusewski
3e57f2db01 docs(b1): implementation plan — 6 paper-track strategies as Dagster assets
10 tasks: generic ForwardTracker + ForwardStrategy port; 5 market-data ports; sixtyforty (template),
multistrat, gd, funding, crossvenue, poc (numpy, no torch); orchestration wiring; deploy + cron retirement.
TDD per task, fakes (no live network), strict-mypy, days-list round-trip through state_reader.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 00:13:50 +02:00
jgrusewski
544e47b04d docs(b1): fold poc into scope — paper path is numpy-only (no torch/pit_sweep)
All 6 remaining strategies in one increment. poc's paper(cfg) path uses numpy xs_weights (residual momentum
top-30 market-neutral) + a Deribit-DVOL VRP sleeve, VRP sizing calibrated from the live lookback window instead
of pit_sweep — so no torch, no warehouse PIT dep, no new state_reader case (emits days-list). Added two data
ports (CryptoPerpBarClient, VolIndexClient) + a poc detail section.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 00:04:44 +02:00
jgrusewski
bff1fe2dee docs(b1): spec — 5 paper-track strategies as clean Dagster assets
Port funding/crossvenue/multistrat/sixtyforty/gd to hexagonal fxhnt services + assets feeding the existing
cockpit_forward ingest; generic ForwardTracker over a ForwardStrategy port (recomputable-series + live-booking
patterns); unify on days-list state shape; retire foxhunt surfer crons. poc deferred to B1b.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 23:59:31 +02:00
jgrusewski
860f7a6325 infra(rbac): drop duplicate ci-deploy Role — canonical home is foxhunt repo
The deploy SA's RBAC is cluster-bootstrap and already lives + is kustomize-applied in the foxhunt repo
(infra/k8s/argo/ci-deploy-rbac.yaml). Keeping a second copy here would drift. Single source of truth.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 22:36:10 +02:00
jgrusewski
e9acfed8d9 infra(rbac): version-control the ci-deploy Role+RoleBinding (was live-only)
The deploy SA's Role existed only in the cluster — a rebuild would silently lose it and break the cockpit
deploy at the RBAC step. Captures current live state (incl. the serviceaccounts/pvc/cronjobs/deployments-watch
grants added 2026-06-15) as bootstrap IaC. Apply with an admin kubeconfig, not from the deploy step.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 22:23:53 +02:00
jgrusewski
7f54941ca4 fix(ci): deploy uses alpine/k8s (has /bin/sh); registry.k8s.io/kubectl is distroless
registry.k8s.io/kubectl is distroless — no /bin/sh — so the multi-command deploy script failed with
exit 64 (no such file /bin/sh). alpine/k8s bundles kubectl on an Alpine base with a shell.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 22:14:07 +02:00
jgrusewski
aef1412f66 fix(ci): deploy step uses official registry.k8s.io/kubectl (bitnami image removed from Docker Hub)
The deploy node hit ImagePullBackOff: bitnami/kubectl:1.31 no longer resolves on docker.io (Bitnami
retired their free Docker Hub catalog in 2025). Switch to the maintained official k8s kubectl image.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 22:11:27 +02:00
jgrusewski
2ca1e9541b fix(ci): enable workflow-level archiveLogs for cockpit builds
spec.archiveLogs:false was the real kill-switch — it disabled archival for the whole workflow regardless
of the per-template MinIO-reachability fix. Flip to true so logs actually land in the argo-logs bucket.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 21:59:18 +02:00
jgrusewski
2379b465b7 fix(ci): label cockpit workflow pods part-of=argo-workflows so MinIO ingress admits them → logs ARE archived
Root cause of months-long missing CI logs: MinIO's ingress netpol allows part-of in {foxhunt,foxhunt-ci,
argo-workflows}; the cockpit build/deploy pods carried neither label → MinIO denied the wait-sidecar's
log upload (i/o timeout) → builds errored. Using 'argo-workflows' (not 'foxhunt') admits them to MinIO
WITHOUT triggering argo-base-egress (which would block registry:5000/git:2222). Reverts the earlier
archiveLogs-disable approach — logs are now SAVED, not skipped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 21:54:52 +02:00
jgrusewski
07cd72533e fix(orchestration): schedule default_status=RUNNING (auto-arm on fresh deploy) + Argo archiveLogs=false per-template (build no longer fails on unreachable MinIO log-sink)
Both latent bugs: a recreated Dagster instance would silently not arm the nightly schedule; and the
Argo build marked Error on the MinIO log-sink (masking real failures + skipping the auto-deploy step).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 21:23:22 +02:00
jgrusewski
10b6bfb6da merge: warehouse bulk ingest (one-transaction, ~50x faster)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 19:20:30 +02:00
jgrusewski
5000333aeb perf(warehouse): bulk ingest in one transaction (~50x faster crypto ingest on PVC)
write_features_bulk writes many symbols in a single transaction (one fsync/checkpoint vs one per
symbol); ingest_crypto_pit collects all 155 symbols then one bulk call. Equivalence test proves
identical result. Targets the ~52min nightly crypto ingest in the Dagster combined-book run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 19:20:30 +02:00
jgrusewski
c7b71616e3 merge: lifecycle B0 — Dagster orchestration of the combined-book forward track (deployed + cut over)
CombinedBookSource port (Warehouse/Raw + A->B equivalence), warehouse read_panel, 4 Dagster assets
(fetch+ingest crypto/futures -> warehouse SSOT -> combined-book NAV -> cockpit), Definitions + daily
schedule, cluster deploy (one-pod webserver+daemon+tailscale, pg storage). Cron suspended; reversible.
Follow-up: batch the ~52min warehouse ingest. 297 tests, mypy-strict clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 18:00:05 +02:00
jgrusewski
d939f9b33f fix(orchestration): assets fetch fresh data before ingest (crypto daily, futures Sunday) — track advances
The B0 assets only ingested existing files → forward track froze on stale data. Mirror the retired
cron: crypto_bars fetches crypto_pit (Binance, free, daily, best-effort); futures_bars fetches via
Databento on Sundays (cost-capped, key-gated) before the silver→warehouse ingest.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 16:52:25 +02:00
jgrusewski
f628d78cbb fix(orchestration): serialize futures_bars after crypto_bars (DuckDB single-writer lock)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 15:15:48 +02:00
jgrusewski
bf2029868b deploy(orchestration): Dagster manifest (single-pod webserver+daemon+tailscale, pg storage) + Argo deploy + Dockerfile extra
Task 10 repo prep: one-pod Dagster (RWO surfer PVC shared by webserver+daemon), tailnet-private UI,
storage in cockpit Postgres `dagster` DB, FXHNT_COMBINED_BOOK_DATA_SOURCE=warehouse. Argo build-deploy
extended to apply+roll dagster. Cron suspend = deliberate post-verify cutover step (RWO-forced).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 12:47:38 +02:00
jgrusewski
8efb78ab15 style(orchestration): drop dead type-ignore comments on @asset (strict-mypy clean)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 11:58:12 +02:00
jgrusewski
f4cfec0f73 feat(orchestration): Definitions + daily combined-book schedule
Wire the 4 B0 assets into combined_book_forward_job and a
ScheduleDefinition (cron 30 23 * * *, UTC) via a Dagster Definitions
object; dagster definitions validate passes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-15 11:51:55 +02:00
jgrusewski
c0a4932de9 feat(orchestration): B0 Dagster assets (crypto/futures bars -> nav -> cockpit) + forward_ingest extract
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-15 11:40:39 +02:00
jgrusewski
2d7d7470fd build(orchestration): add dagster optional-dependency extra
Pins adjusted from >=1.8,<1.9 to >=1.12,<1.13 (dagster-webserver matching)
and >=0.28,<0.29 (dagster-postgres) — the 1.8.x/1.9.x/1.10.x lines cap at
Python <3.13 and the 1.11.x line caps at <=3.13.3; 1.12.x is the nearest line
that supports Python 3.13.6 (the venv runtime).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-15 11:30:55 +02:00
jgrusewski
5a75effd8d feat(combined-book): FXHNT_COMBINED_BOOK_DATA_SOURCE flag + source factory; wire CLI
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 11:27:24 +02:00
jgrusewski
c3c90881e0 refactor(combined-book): read via injected CombinedBookSource (default RawFileSource)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-15 11:24:19 +02:00
jgrusewski
c66bffabef feat(combined-book): WarehouseSource reads SSOT; equivalence test proves A->B faithful
- Extend FeatureStore protocol with catalog() and read_panel() so WarehouseSource
  can be typed against the port (mypy strict clean)
- Add silver_ingest_helpers.ingest_crypto_pit — shared bronze→silver→warehouse
  write path for assets and tests
- Implement WarehouseSource in combined_book_source.py; delegates crypto panel
  to read_panel+_panel and futures to WarehousePriceProvider
- Add equivalence test: warehouse-fed panel == raw-file-fed panel (A→B gate)
- 291 tests pass, 0 regressions

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-15 11:16:25 +02:00