1222 Commits

Author SHA1 Message Date
jgrusewski
4a6b5b8f2f fix(D2.1): market-order UNSET_DOUBLE lmtPrice fabricated ~-10000bps slippage; guard capture migration
CRITICAL: ib_async Order.lmtPrice defaults to the IBAPI sentinel UNSET_DOUBLE
(1.79e308, truthy), which every MarketOrder carries — capture_fills treated it
as a real intended price and fabricated ~-10000 bps slippage on EVERY real market
fill. Now treat None-or-UNSET_DOUBLE as "no intended price" -> honest 0.0.
Minor 1: wrap the capture migration in its own try/except so a broken capture
migration can never block a real rebalance (falls back to no capture).
Minor 2: tests exercising capture_fills against a real MarketOrder sentinel and a
raising account_repo/broker driven all the way through plan_and_record.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 23:41:51 +02:00
jgrusewski
ff7a81ceeb feat(D2.1): capture real IBKR fills->exec_fills (strategy-tagged) + ibkr_account_nav snapshot (best-effort, paper-only)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 23:32:37 +02:00
jgrusewski
784cd27949 fix(D1): gate sim_curve_ret write with backtest_summary (same-basis invariant)
Move the recompute-replay sim_curve_ret persist to after the `if sm is None: return`
guard so it is gated on the SAME condition as the reconciliation-gate backtest_summary
write. A too-short replay (< 2 rows, no gate ref) now writes NEITHER, never a degenerate
sim curve with no matching gate ref. Adds test_insufficient_replay_writes_neither_ref_nor_sim_curve.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 23:20:48 +02:00
jgrusewski
3532671892 feat(D1): /paper/sim sources any book's curve by backtest.kind (adds IBKR multistrat/vrp)
Generalizes the Backtest page beyond the Bybit-only venue: bybit_4edge/bybit_4edge_levered
keep reading the precomputed bybit_sleeve_ret table; the IBKR paper strategies (multistrat,
vrp) now read their own precomputed recompute-replay curve from a new sim_curve_ret table,
written nightly by _persist_track_backtest_ref from the SAME rows the reconciliation-gate
ref is derived from. New sim_curves.sim_returns_for dispatches on registry_backtest_kind;
app.py wires an _ibkr_measured_context render path (reusing the existing pure
simulate_naive_eqwt engine) alongside the unchanged Bybit measured-cost path, with
plain-language pill labels for the two IBKR books.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 23:13:27 +02:00
jgrusewski
49b6a102d8 docs: update IBKR parity spec+plan to mockup-validated UX
Consolidate /paper into ONE "Paper books" page (crypto simulated + IBKR real,
both clickable -> holdings detail); the holdings detail shows what it's holding
(assets + shares + value + %), recent trades (cost-to-trade %), value-over-time
(real vs plan) + per-strategy breakdown. Overview gets a plain "real trades"
sub-line + "behind plan" pill. HARD RULE: plain language, no dev jargon — a
single display_names map is the source of every rendered name; REAL PAPER vs
PAPER badges. Backtests page includes the IBKR tracks (D1).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 22:54:44 +02:00
jgrusewski
2cbd2736e5 docs: plan — IBKR paper cockpit parity (D1 sim curve, D2 capture+detail, D3 overview, D4 account view)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 22:30:04 +02:00
jgrusewski
8a99aff76a docs: spec — expand IBKR paper parity to full UX (overview + account view + multi-strat)
Add D3 (overview EXEC badge + real-vs-sim divergence chip + account rollup),
D4 (/paper/ibkr account-level view: total NAV + all positions/fills +
per-strategy breakdown), a UX & information-architecture section (3-level
progressive disclosure, one divergence signal rendered consistently), and a
strategy-tagged data model so a second executor is a new ROW, not a redesign.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 22:27:15 +02:00
jgrusewski
5c0b9ddb27 docs: spec — IBKR paper cockpit parity (backtest sim curve + real DU-account book + reconciliation)
Bring the IBKR paper strategies to cockpit parity with the Bybit book: their
0a backtest curve in /paper/sim (D1), and capture the REAL DU paper-account
state (fills -> exec_fills, positions, NAV) rendered as a cockpit book +
reconciled vs the recomputed multistrat sim (D2) — the honest pre-live
verification. Paper-only (no live/arm), Bybit untouched (no EU testnet keys),
faithful extension of the existing exec path (IbkrBroker already reads NLV+
positions; exec_fills exists but is empty).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 22:19:53 +02:00
jgrusewski
9c51b754ba fix(infra): ib-gateway fsGroup 1000 so IBC can write jts.ini on the PVC
A fresh tws_settings PVC mounts root-owned; the gnzsnz container runs as the
non-root ibgateway user (UID/GID 1000) -> "Permission denied" writing jts.ini
-> crashloop. fsGroup: 1000 makes the volume group-writable. Verified: new pod
ready, 0 restarts, "IBC: Login has completed".

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 20:36:37 +02:00
jgrusewski
86db827740 Merge: ib-gateway replicas landmine + in-place auto-restart + persistent settings 2026-07-14 20:26:30 +02:00
jgrusewski
d399982cd1 fix(infra): ib-gateway — fix replicas:0 landmine + in-place daily auto-restart + persist settings
Three fixes to the sole IBKR paper execution gateway:
- replicas 0->1: the manifest said "dev-mode paused" (replicas:0) while the live
  cluster ran 1 — a drift that would scale IBKR execution to ZERO on any re-apply.
- AUTO_RESTART_TIME "8:00 AM" UTC + TIME_ZONE Etc/UTC: IBC restarts the gateway
  IN-PLACE daily ("does not require daily 2FA validation") instead of the container
  exiting on IBKR's forced reset — the cause of ~13 container restarts + a full
  re-login each time. 08:00 UTC is clear of the 14:35 rebalancer + 23:30 nightly.
- TWS_SETTINGS_PATH + a 1Gi PVC: persist jts.ini/IBC config across pod restarts.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 20:26:22 +02:00
jgrusewski
bb61323904 Merge: Phase 3 cleanup — gauntlet test + cockpit crypto_tstrend 404
- gauntlet test_keeps_real_premium was RED on master (marginal, order-dependent
  synthetic signal the gauntlet correctly rejected); now a local-seeded,
  seed-robust strong premium (200/200 seeds). Gauntlet code untouched.
- cockpit M5: the bybit_4edge per-edge breakdown rendered the crypto_tstrend
  SLEEVE with a raw id + a dead /strategy/crypto_tstrend 404 link. Now a
  _SLEEVE_DISPLAY_NAMES map (independent of STRATEGY_REGISTRY) + a `linkable`
  field so only sleeves that are real strategies get a /strategy link.

Full suite 1891 passed. Clean sonnet review.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 20:19:14 +02:00
jgrusewski
39fa034f49 fix(test): strengthen gauntlet keep-premium signal to seed-robust (200/200)
Reviewer noted drift 0.001/vol 0.008 (Sharpe ~2) still failed the OOS-decay
check on ~5% of seeds (seed 11 happened to pass). Bump to 0.0015/0.007
(Sharpe ~3.4), verified 200/200 seeds pass — robust, not seed-lucky.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 20:17:44 +02:00
jgrusewski
fd5d97bc94 fix(cockpit): give the bybit_4edge sleeve breakdown its own display names
crypto_tstrend's standalone STRATEGY_REGISTRY entry was retired in Phase 0b
Task 4, but the still-live crypto_tstrend book SLEEVE in the bybit_4edge
per-edge breakdown (Overview + /paper) shared the same string id, so its
display-name lookup blanked to the raw id and linked to a dead
/strategy/crypto_tstrend (404).

Add _SLEEVE_DISPLAY_NAMES next to _DEFAULT_BYBIT_SLEEVES in
bybit_book_eval.py — the sleeve breakdown's own display-name source,
independent of STRATEGY_REGISTRY, so a future forward-track retirement can
never blank a live sleeve again. _deploy_individual_edges() now sources
display_name from that map and adds a `linkable` field (true only when the
sleeve id is also a real registry entry); deploy_constituent_row renders the
/strategy link only when linkable, plain text otherwise.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 20:11:31 +02:00
jgrusewski
43afd24062 fix(test): gauntlet keep-real-premium used a marginal, order-dependent signal
test_keeps_real_premium was RED on master: at the shared module RNG's state it
drew an in-sample half with ~zero drift (is_sharpe 0.02), so the gauntlet
CORRECTLY rejected it (no in-sample evidence) — the test's premise, not the
gauntlet, was wrong. Now uses a local seeded RNG (order-independent) and a
genuinely strong premium (drift 0.001/vol 0.008, in-sample Sharpe ~2, t~4.8);
verified 30/30 seeds pass. The gauntlet keeps genuine premia (is_sharpe 0.82-1.23
-> dsr 0.978-0.999); it was never broken.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 20:03:19 +02:00
jgrusewski
129409a282 Merge: vrp freeze targets the last COMPLETE OPRA session (robust, timing-independent)
The vrp OPRA freeze froze calendar-today, which fails intraday with
422 data_schema_not_fully_available — timing-dependent/fragile, leaving vrp
perpetually STALE. Now it freezes the latest COMPLETE session via a bounded
step-back from the dataset available-end (steps back one trading day on the
catchable session-unavailable signal, skips weekends, bounded to 5). All
candidates failing raises DatabentoRangeUnavailable -> vrp_nav degrades to
recompute-off-frozen (no crash) and the health axis keeps vrp loudly STALE —
never a silent no-op. Also corrects the fxhnt-backtest-refs.yaml header
(xsfunding/unlock/positioning/bybit_4edge* are report-kind, not inline).

Full suite 1887 passed (the 1 failing test_gauntlet::test_keeps_real_premium
is pre-existing/RED on clean master, unrelated — tracked separately).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 19:50:06 +02:00
jgrusewski
7e7a657090 docs(infra): correct backtest-refs header — xsfunding/unlock are report-kind
Phase 0b (venue consolidation) made `xsfunding` + `unlock` report-kind,
routed through `_persist_bybit_book`/`_BYBIT_BOOK_SIDS` in this CronJob's
`backtest-refs --all` batch path — NOT inline. The stale header wrongly
listed them (and crypto_tstrend/stablecoin, which aren't in the registry)
among the inline refs. Reword to reflect the actual split: the report-kind
sids this Job covers are bybit_4edge / bybit_4edge_levered / positioning /
xsfunding / unlock; the truly-inline (recompute-replay) refs are only
multistrat / vrp. (This is why unlock had no ref until this Job ran.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 19:46:08 +02:00
jgrusewski
70708ca8a7 fix(vrp): freeze the LAST COMPLETE OPRA session via bounded step-back
The vrp health axis flagged vrp=STALE because the OPRA freeze
(xsp_option_bars) was stuck: vrp_nav froze as_of=today, but today's
session is only partially available intraday, so the freeze failed with
`422 data_schema_not_fully_available` (distinct from the C1a clamp's
`data_end_after_available_end`). That made the freeze timing-dependent —
it could only succeed in a narrow post-settlement window and otherwise
silently degraded to recompute-off-stale-frozen.

Make it robust and timing-independent:
- databento: normalize the `data_schema_not_fully_available` 422 into the
  single catchable DatabentoRangeUnavailable (surgical — only that 422;
  any other client error propagates raw). Expose
  last_available_option_date() as the step-back's starting candidate.
- assets: new _freeze_latest_session entry point walks back one TRADING
  day (skip Sat/Sun) from OPRA's available-end until a COMPLETE session
  freezes, bounded to 5 steps; returns/logs the frozen session date.
  Never freezes a partial session's marks.
- No silent degradation: if no complete session is found in the bound it
  raises DatabentoRangeUnavailable; the existing C1b wrapper in vrp_nav
  catches it → recompute off frozen, and the health axis keeps vrp STALE
  (loud), never a quiet no-op.

TDD: step-back-past-partial, weekend-skip, bound-exhausted-raises, 422
normalization (both chain + bars paths), unrelated-error-propagates,
last_available_option_date caching. Full suite green (one pre-existing,
unrelated seed-dependent gauntlet flake).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 19:45:48 +02:00
jgrusewski
de9beb89ac Merge: Phase 1 gate-unstick + robustness hardening
Unsticks the nightly gate/promotion/allocation pipeline (cockpit_forward),
which had been silently SKIPPED since 2026-07-12 because vrp_nav hard-failed
on a databento 422 and cockpit_forward deps on it.

C1  vrp_nav + databento: clamp OPRA query end to the dataset's available range
    (via cached get_dataset_range, robust multi-shape parse); wrap _freeze_day
    so a freeze failure degrades to recompute-off-frozen, never kills the asset.
C2  every *_nav asset is failure-isolating (data/freeze errors degrade to a
    no-op, genuine math bugs stay loud) so one broken track can't skip the gate.
H3  multistrat forward books the final close it just froze (drop_incomplete_today
    flag: nightly=False, intraday exec=True) — was permanently 1 trading day stale.

Robustness (no silent degradation — the rule that vrp's 2-week silent breakage
earned): a runtime HEALTH axis (HEALTHY/STALE/NO_REF/BROKEN, separate from the
gate) surfaces any live/gated track that stops updating as a LOUD red cockpit
badge + ERROR log on day 1 — freeze-staleness axis catches the exact vrp mode,
NO_REF is the M6 runtime invariant, and a wiped/never-produced-data track with
an old anchor reads STALE (not a quiet "no data yet").

Full suite 1881 passed. Two clean opus reviews.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 19:13:54 +02:00
jgrusewski
01e437816b fix(forward-health): close no-anchor silent-HEALTHY hole; fix BROKEN docstring scope
The "no forward data yet" carve-out in compute_health let a reconciliation-gated
recompute track that never produced data since inception (empty freeze table / no
forward_summary, old anchor t0 or no active anchor at all) read HEALTHY the same
way a genuinely brand-new track does. Thread the active forward_anchor.t0 through
evaluate_health so compute_health can tell "just inceptioned" (anchor within the
stale threshold -> HEALTHY) apart from "broken from inception" or "never anchored"
(-> STALE), closing the silent-degradation hole without false-alarming new tracks.

Also corrects the module docstring: BROKEN only covers per-track compute failures,
not the migrate/all_summaries/all_backtest_summaries/xsp_freeze_max_date reads
evaluate_health hoists outside the per-track try (those fail the whole
cockpit_forward Dagster asset instead).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 19:13:13 +02:00
jgrusewski
d070ec9127 feat(B): runtime staleness/health axis — kill silent rot (red badge + ERROR log)
vrp rotted for 2 weeks because a data failure degraded QUIETLY (recompute off a
frozen-but-stale table, only a log line) and the gate can't tell a stale track
from a healthy WAIT. Add a HEALTH axis SEPARATE from the gate so ANY live/gated
track that stops updating surfaces LOUDLY on day 1.

forward_health.py (new): compute_health (pure state machine) + evaluate_health,
called by cockpit_forward AFTER ingest. States {HEALTHY, STALE(age), NO_REF,
BROKEN}, precedence BROKEN>STALE>NO_REF>HEALTHY:
- STALE: a `recompute` track whose source as_of OR its freeze table's max date
  (vrp → xsp_option_bars) is > Settings.stale_after_business_days (default 3)
  business days behind. The freeze axis catches the exact vrp mode: forward
  "updates" yet the frozen input is stale.
- NO_REF: a reconciliation-gated recompute track with no resolvable backtest
  reference (the gate can never reconcile) — the M6 static invariant made a real
  runtime check.
- record_mode carve-out: `observed` twins (vrp_exec/…) are externally-driven and
  legitimately DORMANT when execution is suspended — never staleness/no-ref
  alarmed (won't false-alarm the intentionally-dormant reference_only tracks).
- BROKEN: a per-track health-compute error is surfaced, never swallowed.

Storage: new strategy_health table (create_all — idempotent, Postgres-safe, no
ALTER), replace-all each run so a recovered track clears its stale row.
Rendering: FleetRow/StrategyDetail gain health/health_stale_days/health_reason;
a LOUD red health_badge (STALE 14d / NO REF / BROKEN) on Overview, the all-tracks
table, and the detail header — visually impossible to confuse with a WAIT.
cockpit_forward + evaluate_health log at ERROR with the unhealthy list.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 19:00:34 +02:00
jgrusewski
6c9d43774e fix(A): robust databento get_dataset_range parse + tz-aware clamp guard
The nightly OPRA freeze parsed metadata.get_dataset_range with a defensive
guess that only read a top-level "end" key and required a dict — a VALID
per-schema-nested response ({schema: {"start","end"}}) or a DatasetRange-like
object spuriously raised DatabentoRangeUnavailable, which vrp_nav's C1b wrapper
absorbs → silent recompute off a stale frozen table (the exact 2-week vrp rot).

- _dataset_range_ends(): parse ALL real 0.79 shapes (return type
  dict[str, str | dict[str, str]]): top-level {"start","end"}, per-schema
  nesting (max end across schemas), and a .end/.start attribute object. Only
  return [] (→ catchable DatabentoRangeUnavailable) for a genuinely empty /
  malformed response, never on a shape that carries the range.
- _to_utc_ts(): single tzinfo guard (localize if naive, convert if aware).
  Fixes _clamp_option_end's C1b gap where pd.Timestamp(x).tz_localize("UTC")
  raised TypeError on a tz-AWARE start/end.
- document the _option_available_end cache lifetime (one nightly run).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 18:54:14 +02:00
jgrusewski
515dec4b55 fix(H3): thread drop_incomplete_today so multistrat forward books today's close
multistrat._build dropped dates[-1] whenever it equalled UTC-today (to shed an
incomplete intraday bar). Correct for the exec path (execute-multistrat, 14:35
UTC, market OPEN) but WRONG for the nightly multistrat_nav (23:30 UTC, after the
US close): it first freezes today's FINAL close, then _build threw that day away
every night, leaving forward_summary.multistrat.as_of stuck one trading day
behind forever.

Thread a drop_incomplete_today flag (default True) through _build / book_series
/ target_weights / MultiStratStrategy. The nightly multistrat_nav constructs
MultiStratStrategy(..., drop_incomplete_today=False) — and since the SAME
build_strategy lambda backs both the forward track and its backtest ref, both
sides use the identical complete-today basis (no reconciliation divergence). The
exec target_weights path and growth_discipline keep the default True.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 18:38:28 +02:00
jgrusewski
3302ca5d04 fix(C2): make every *_nav failure-isolating so the gate is never skipped
cockpit_forward reads purely from the DB (forward_summary) and its deps are
ordering-only, so a skipped upstream stalls the whole gate/promotion/allocation
pipeline. Close the two pre-tracker offenders so no *_nav asset raises on a
data-provider failure:

- multistrat_nav: the snapshot_yahoo_closes loop runs OUTSIDE _run_paper_tracker;
  a Yahoo outage now logs ERROR and degrades to a no-op {forward_days:0,...}
  return (record unchanged) instead of raising and skipping the gate.
- _run_paper_tracker: broaden the caught set to also isolate the databento
  provider error base (BentoError/BentoClientError, imported lazily since
  databento is an optional extra). Bare Exception is NOT caught — genuine
  strategy-math bugs (KeyError/ValueError) still propagate loudly.

With every expected data-provider failure mode isolated, cockpit_forward always
runs (deps kept for ordering). See report for the decouple-approach rationale.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 18:36:48 +02:00
jgrusewski
afe488e6aa fix(C1): clamp OPRA query end to dataset range + vrp_nav freeze resilience
vrp_nav hard-failed every night: _freeze_day runs OUTSIDE _run_paper_tracker
and databento 422'd (data_end_after_available_end) because end=as_of+1day
exceeded OPRA's ~22:30-UTC available end at the 23:30 freeze. The raised
BentoClientError killed the asset before run_track/set_anchor, zeroing vrp's
forward anchor AND skipping the downstream cockpit_forward gate.

C1a: DatabentoDataProvider now fetches the OPRA available end via
metadata.get_dataset_range (cached once per instance) and clamps the exclusive
query `end` down to it in fetch_option_chain/fetch_option_chain_range/
fetch_option_bars, so no query requests past what OPRA has published. An empty
window (start >= clamped end) raises catchable DatabentoRangeUnavailable, never
a raw 422.

C1b: vrp_nav wraps _freeze_day so any freeze failure logs a WARNING and
continues to _run_paper_tracker, which recomputes VrpStrategy off the FROZEN
xsp_option_bars table — guarantees a forward row/anchor/ref even on a day the
freeze fails, and never skips the gate.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 18:34:57 +02:00
jgrusewski
95d47aaa05 Merge: remove stale fxhnt-forward CronJob residue 2026-07-14 17:16:09 +02:00
jgrusewski
2677bcd02a chore(infra): remove the stale fxhnt-forward CronJob residue
The fxhnt-forward CronJob was superseded by the deterministic-forward-state
Dagster assets (dagster.yaml cutover) and had been suspended/dead since before
0a — its args referenced the long-removed `fetch --crypto` + `forward-track`
CLI. Deleted the orphaned live CronJob + its dead NetworkPolicy (whose egress
still whitelisted Binance for a job that no longer runs), removed the netpol
stanza from fxhnt-cockpit.yaml, and refreshed the now-stale RBAC comment.
The batch/cronjobs RBAC rule stays (still used by the bybit/ibkr rebalancers +
compare-measured-precompute). Live objects already deleted via kubectl.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 17:16:09 +02:00
jgrusewski
94ad91977e Merge: drop the vestigial venue column from the paper-book tables
bybit is fxhnt's sole paper-book venue (Task 7f). The prod purge removed all
orphaned binance/legacy rows (236,829 rows across 7 tables), leaving them
bybit-only. This drops the now-constant `venue` column entirely: removed from
the 7 paper_* ORM models + ~50 read/write sites in paper_repo.py; new
paper_book_migration.py runs the one-time Postgres reverse migration
(DROP PK → DROP COLUMN venue → ADD PRIMARY KEY on the reduced key set;
paper_nav_summary gets a singleton id=1 PK). Idempotent + atomic + no-op after
first run. strategy_registry.venue (the strategy venue label) is untouched.

Full suite 1853 passed. Reviewed: migration DDL correct/atomic/idempotent;
no accidental co-predicate drops; bybit-book coverage intact.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 16:41:08 +02:00
jgrusewski
649ea25000 feat(paper): drop vestigial venue column from the 7 paper_* tables (Phase 0c)
bybit is fxhnt's sole paper book (Task 7f hardcoded every PaperRepo read/write
to it); the prod purge left every paper_* table bybit-only. Remove the now-constant
`venue` column at the SCHEMA level so the paper book is single-venue by construction.

- cockpit_models.py: drop `venue` from all 7 paper models; re-key the 5 composite-PK
  tables on their remaining columns; paper_nav_summary swaps its sole `venue` PK for a
  singleton surrogate `id` (=1); paper_trades drops the non-PK venue index+column.
- paper_book_migration.py (new): Postgres-only, idempotent reverse migration invoked by
  PaperRepo.migrate(). Guarded on the live `venue` column so it is a pure no-op after the
  first run (heavy DDL on every cockpit startup would crashloop). Sole home of the
  column-drop DDL, keeping paper_repo free of any discriminator logic.
- paper_repo.py: delete `_BYBIT_VENUE`/`_VENUE_TABLES`; strip every venue kwarg/predicate;
  PK lookups use (1) / (key). `grep -n venue paper_repo.py` → 0.
- test_no_binance_data.py: 7f venue-default guard → schema guard (no paper model has a
  `venue` column).

Full suite green: `uv run pytest -q` → 1853 passed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 16:34:17 +02:00
jgrusewski
f38b5ad55b Merge: Phase 0b — venue consolidation to Bybit+IBKR
Bybit (crypto 4-edge book) + IBKR (equity) are now the ONLY execution and
data venues. Binance and Alpaca fully removed from src (substring guards
enforce 0 hits). Crypto edges migrated Binance->Bybit; single-sourced
Bybit reconciliation cost (Settings.bybit_cost_bps=5.5); dead tracks retired
(crypto_tstrend standalone, stablecoin_rotation); registry state_file field
removed (DB anchor is the forward SSOT); dual-venue PaperRepo discriminator
collapsed to bybit-only.

Migrated tracks re-incept VISIBLY at version=2 (deterministic-forward-state).
Deploy fund (bybit_4edge/levered) + IBKR multistrat/sixtyforty/vrp forward
tracks preserved. 0a invariant test (every reconciliation-gated strategy has a
non-none backtest ref) stays green. Full suite 1853 passed.

POST-DEPLOY (mandatory, see PR/runbook):
  1. `fxhnt migrate-forward-anchors` — re-inception of xsfunding/unlock/
     positioning/bybit_4edge/levered at v2.
  2. Cluster: delete fxhnt-alpaca-rebalancer + fxhnt-multistrat-rebalancer
     CronJobs + alpaca-credentials secret.
  3. Follow-up schema migration: drop venue column from paper-table PKs +
     DELETE WHERE venue IN ('binance','legacy').

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 15:42:41 +02:00
jgrusewski
375cd108bc feat(0b): remove the vestigial dual-venue PaperRepo abstraction (Task 7f)
Task 7e only renamed venue="binance" -> "legacy" to pass the substring guard;
the dual-venue repo machinery was left in place even though bybit is the ONLY
live venue. PaperRepo now hardcodes every discriminated read/write to a
_BYBIT_VENUE = "bybit" module constant instead of a constructor venue param
(removed) -- reads/writes ONLY the bybit rows, so the orphaned legacy/binance
rows from the retired combined-crypto book (Task 7d) are never touched.
nav_summary's venue arg is dropped the same way (every caller always passed
"bybit"). cockpit_models keeps the venue column (dropping it from the PK is a
separate prod schema migration, documented in the report for the merge
runbook) with reworded comments reflecting the single-venue reality.

Test fallout: deleted test_paper_repo_venue.py (tested the now-removed
dual-venue isolation) and 3 "does-not-touch-binance" tests that constructed
PaperRepo(venue="binance"); mechanically dropped the venue kwarg everywhere
else. Added a guard assertion that PaperRepo.__init__ has no venue param.
Full suite green (1853 passed); binance substring guard still 0 hits.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 15:20:57 +02:00
jgrusewski
68970a793f feat(0b): true Binance 0% — remove the crypto_pit Binance fetch/ingest/research surface + substring guard (keep Databento futures + shared TrendRunner) 2026-07-14 14:57:08 +02:00
jgrusewski
ea66df5d7f docs(0b): refresh stale assets.py module docstring (crypto_bars deleted 7b; DB SSOT, no state files)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 10:40:12 +02:00
jgrusewski
62357d6cc2 feat(0b): remove vestigial registry state_file field (DB anchor is the forward SSOT)
Step-1 finding: migration_builders.py and the migrate-forward-anchors CLI
(cli.py:427) already carry their own local strategy_id -> state_file basename
map (track_builders()) from an earlier Task-8 pass -- neither reads
entry["state_file"] off STRATEGY_REGISTRY, so removing the field doesn't
touch them. Only three direct consumers needed changes:
 - registry.py: dropped "state_file" from all 11 entries + the header comment.
 - assets.py: _run_paper_tracker's state_file param was unused at steady
   state (its own docstring said so) -- dropped it + updated its 3 call
   sites (sixtyforty_nav, multistrat_nav, vrp_nav).
 - a handful of tests asserted on the registry's state_file key; updated to
   drop those assertions, and added tests/unit/test_no_registry_state_file.py
   as a permanent guard.

The JSON ForwardTracker + its .bak crash-recovery, and migrate-forward-anchors
itself, are untouched -- still the one-time JSON->DB seed / deployment tool.
2026-07-14 10:38:15 +02:00
jgrusewski
badded008c docs(0b): sweep stale comments referencing deleted crypto_funding/paper-backfill/Binance-klines (7c+7d review minors)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 10:31:37 +02:00
jgrusewski
0be7eb2fc6 feat(0b): retire the dead Binance-era combined-crypto /paper book (redundant with bybit_4edge); close the 7c Major 2026-07-14 10:21:53 +02:00
jgrusewski
5fc25a428b feat(0b): finish Binance-0% — repoint worst-basis to Bybit; drop vestigial stablecoin display + dormant MomentumVrp (Task 7c) 2026-07-14 09:48:00 +02:00
jgrusewski
994bb59641 test: parallelize (pytest-xdist -n auto) + optimize the slow tests
- pytest-xdist added to dev deps; addopts='-n auto' runs the ~1940-test suite across all
  cores (~6-9min -> ~1.5min); registered a 'slow' marker for a serial '-m "not slow"' loop.
- equity_backtest_runner liquidity fixture: trimmed the 3000-bar (8yr) LIVELONG/ILLIQUID
  histories to 800 bars (still >> min_history=300, multi-year for peak-yearly ranking) —
  the peak-yearly aggregation over 8yr was the cost: 50s/30s/19s tests -> ~5.8s each,
  assertions unchanged.
- paper_sim perf guards: made them parallel-robust (the wall-clock <1.6s bound flaked under
  -n auto CPU contention). full-history guard relaxed to <10s (it targets an O(D²) blowup =
  orders of magnitude, not a constant factor); cheap-reapply made RELATIVE (< full/10, ratio
  is contention-invariant). Both marked slow.

Full suite 1940 passed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 00:51:52 +02:00
jgrusewski
86aba147c9 feat(0b): remove Binance data + cross-venue research backends — crypto is 100% Bybit
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 00:30:20 +02:00
jgrusewski
4fd0f9454d chore(0b): remove orphaned _equity_spark (task 7a review minor — dead after the Binance paper branch removal)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 23:31:53 +02:00
jgrusewski
af68419909 feat(0b): cockpit is Bybit-only — remove the Binance venue toggle + paper book + sim compare view (Task 7a)
Removes the cockpit's (web-layer) use of Binance: the /paper venue toggle + Binance paper book view, the /paper/replay venue toggle, and the sim's binance_combined book + Binance-vs-Bybit compare view (_sim_compare.html deleted). Bybit is now the cockpit's only paper-book venue. Backend Binance data/research modules (compare_measured_cost.py, data adapters) are untouched -- Task 7b deletes those.
2026-07-13 23:16:07 +02:00
jgrusewski
fb9564c798 feat(0b): remove dead Binance execution surface (binance_ccxt + crypto-rebalance/positions/flatten CLI)
Binance is deprecated as a broker; kept only as a DATA source (Task 7 removes
that). Deletes binance_ccxt.py, crypto_execution.py (no non-Binance consumer),
the crypto-rebalance/crypto-positions/crypto-flatten CLI commands, and the
execution-only BinanceSettings class + field. Adds a narrow guard test.
2026-07-13 22:35:33 +02:00
jgrusewski
49e30b17ea chore(0b): scrub residual Alpaca labels (task 5 review minors — stale rebalancer comment + test fixtures)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 22:24:07 +02:00
jgrusewski
4cf6de17ee feat(0b): remove dead Alpaca execution; IBKR is the sole equity executor (Broker port kept)
Alpaca's rebalancer CronJob has been suspended with 401'd keys since Task 4's
venue consolidation; IBKR paper is already the live forward-track venue for
the multi-strat book. This removes the dead code/config/manifests instead of
leaving them to rot, and relabels the multistrat/multistrat_exec registry
venues from the vestigial "alpaca-paper" to "ibkr-equity" (they execute on
IBKR paper, not Alpaca — exec_venue itself is still computed dynamically at
runtime).

- Delete src/fxhnt/adapters/broker/alpaca.py, AlpacaSettings + the `alpaca`
  config field; move the one real setting it carried (min_order_notional,
  a generic "skip sub-min-notional dust" floor for any future
  fractional-capable broker) onto ExecutionSettings.
- Remove the `--broker` option and its alpaca branches from `execute` and
  `execute-multistrat` (cli.py) — IBKR is now implicit; IbkrBroker was
  already the module-level import, so the branch was pure dead code once
  Alpaca is gone.
- Delete infra/k8s/jobs/fxhnt-alpaca-rebalancer.yaml and
  infra/k8s/secrets/alpaca-credentials.yaml; delete the superseded
  infra/k8s/services/multistrat-rebalancer.yaml (PVC + NetworkPolicy +
  CronJob), replaced by fxhnt-ibkr-rebalancer. Drop the now-invalid
  `--broker ibkr` argument from fxhnt-ibkr-rebalancer.yaml and
  fxhnt-ucits-rebalancer.yaml (the flag no longer exists).
- Delete tests/integration/test_alpaca_broker.py (tested the deleted
  adapter directly); add tests/unit/test_no_alpaca.py as a standing guard
  (no "alpaca" substring anywhere in src/fxhnt + IbkrBroker still imports).
- Update test_registry_multistrat_exec.py + test_cockpit_ssot.py (venue
  label) and test_cli_execute_helpers.py (drop the broker_name kwarg and
  AlpacaBroker fake, use IbkrBroker instead) to match.

MANUAL cluster ops (not run by CI, record only):
  kubectl delete cronjob/fxhnt-alpaca-rebalancer secret/alpaca-credentials -n foxhunt
  kubectl delete cronjob/multistrat-rebalancer networkpolicy/multistrat-rebalancer pvc/multistrat-state -n foxhunt

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 22:17:04 +02:00
jgrusewski
d7a9c32690 feat(0b): retire crypto_tstrend standalone track + stablecoin_rotation (not tradeable on Bybit); book sleeve unchanged
Neither is tradeable on Bybit: crypto_tstrend as a standalone Binance-perp forward
track was a -0.21 marginal-Sharpe drag/crash-amplifier, and stablecoin_rotation's
FDUSD/USDP pairs aren't listed on Bybit. Removes the two *_nav assets, their
registry entries, the orphaned migration_builders builders, and the
Binance-only StableRotationForward wrapper (stablecoin_runner.py's
StableReversionRunner stays -- it's still live via the Bybit stablecoin eval
paths). The crypto_tstrend SLEEVE inside the bybit_4edge deploy book
(_DEFAULT_BYBIT_SLEEVES, Bybit data) is untouched.

Updates dependent tests: deletes 5 whose subject (the retired asset/registry
entry/module) no longer exists, and swaps the retired sid for a still-registered
one (unlock/xsfunding/sixtyforty) in tests that only used crypto_tstrend as a
generic example sid.
2026-07-13 21:56:23 +02:00
jgrusewski
a213edb800 fix(0b): single-source the Bybit reconciliation cost (bybit_cost_bps=5.5)
The standalone Bybit sleeve tracks (positioning/xsfunding/unlock) AND the two 4-edge
book tracks (bybit_4edge/bybit_4edge_levered) charged cost_bps_per_turnover=10.0 in
their forward nav assets, while their reconciliation ref (_persist_bybit_book) computed
the backtest at 5.5 — a forward-charged-more-than-ref mismatch that biases the gate
toward WAIT on a cost artifact (safe direction, but could permanently WAIT a real edge).
New Settings.bybit_cost_bps=5.5 (the measured liquid-book cost — the mirage finding) is
the SINGLE source for every Bybit forward track + its ref; registry cost_bps 10->5.5 +
definition_version 1->2 (visible deterministic re-inception). Equity/tradfi tracks keep
cost_bps_per_turnover (different venue). Invariant test locks all 5 Bybit sids. Full
suite 2058 green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 21:27:42 +02:00
jgrusewski
47419b398b docs(0b): insert Task 3 (align Bybit reconciliation cost basis, single-source 5.5); renumber 4-9
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 21:04:08 +02:00
jgrusewski
14f9894482 feat(0b): migrate unlock to a Bybit single-sleeve forward track
Reuses build_bybit_sleeve_forward_nav (Task 1). unlock_nav drops the standalone
Binance UnlockShortForward(UnlockCalendarLive) path, runs the single unlock sleeve
on bybit_features through the deterministic engine, loading the unlock calendar via
load_unlock_events (the same loader the Bybit book uses). unlock joins _BYBIT_BOOK_SIDS
+ bybit_report_sid_kwargs (ref via _persist_bybit_book); registry venue=bybit-perp,
backtest.kind=report. _DEFAULT_BYBIT_SLEEVES/deploy book untouched. Regression lock
now requires the unlock ref too. Full suite 2057 green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 20:51:59 +02:00
jgrusewski
5a3fca0de3 test(0b): tighten backtest-refs regression lock to require xsfunding ref (task 1 review minor)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 20:29:43 +02:00
jgrusewski
810ebdc105 feat(0b): migrate xsfunding to a Bybit single-sleeve forward track (venue-consistent ref)
Adds a generic build_bybit_sleeve_forward_nav(repo, store, sleeve, ...) builder in
assets.py, refactors build_positioning_forward_nav to delegate to it, and moves
xsfunding_nav off the standalone Binance XsFundingForward(BinanceXsFundingLive())
construction onto its own Bybit single-sleeve forward track (mirrors positioning
exactly). xsfunding still runs unchanged as a sleeve inside the bybit_4edge deploy
book (_DEFAULT_BYBIT_SLEEVES untouched).

- bybit_forward_track.py: bybit_report_sid_kwargs() gains an xsfunding case.
- cli.py: _BYBIT_BOOK_SIDS += "xsfunding" (routing only, series already computed).
- registry.py: xsfunding -> venue bybit-perp, backtest.kind sleeve->report, params
  mirror positioning's shape, record_mode observed->recompute (required for the
  deterministic-engine recompute path; confirmed by the exhaustive observed-mode
  pin in test_forward_definition.py, updated accordingly).
- migration_builders.py: replaced the xsfunding lambda:None stub (valid only under
  the old observed record_mode) with a real single-sleeve Bybit builder, mirroring
  positioning's, to avoid a latent crash if the one-time forward migration is ever
  re-run.

Tests: new tests/unit/test_bybit_sleeve_forward_builder.py (TDD, brief-verbatim);
full tests/unit + tests/integration sweep: 2056 passed, 0 failed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 20:25:03 +02:00