Commit Graph

175 Commits

Author SHA1 Message Date
jgrusewski
84c9fcd146 fix(alloc): per-strategy cap now binds (concentration limit, holds cash) instead of a renormalise no-op (B2a)
The equal-weight cap+renormalise was algebraically a no-op — n=1 got 100% weight despite a 50% cap.
Cap without renormalising: when 1/n > max_strategy_weight the book holds cash rather than over-
concentrating; common case (1/n <= cap) is plain equal weight. + tests that exercise the binding cap.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 21:17:52 +02:00
jgrusewski
497d4022db feat(alloc): compute_allocation — ex-ante vol-target on full-history vol, marginal gate, killswitch (B2a)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 21:10:54 +02:00
jgrusewski
5893604158 fix(alloc): deterministic tie-break in equal_weight_marginal_prune + accurate module docstring (B2a)
sorted(kept) makes the leave-one-out bench choice stable across process runs (a set's iteration order
is PYTHONHASHSEED-dependent) — a reproducibility fix for a capital-allocation function. Docstring no
longer claims compute_allocation/target_capital, which land in later tasks.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 21:07:23 +02:00
jgrusewski
7b886f0a05 feat(alloc): pure allocation helpers — full-history vol, equal-weight marginal prune, killswitch latch (B2a)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 21:02:12 +02:00
jgrusewski
9ec8eaab9c feat(alloc): versioned AllocationPolicy + content-derived hash + capital_ceiling (B2a)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 20:58:21 +02:00
jgrusewski
f821929ddf refactor(policy): gate functions take ResolvedPolicy; remove scattered gate constants (B1)
evaluate_reconciliation_gate + evaluate_gate now take a ResolvedPolicy (resolve_policy(POLICY,
gate_spec)); the module constants (MIN_FORWARD_DAYS / RECON_* ) + gate.py inline defaults are
removed — thresholds now come from the single GatePolicy. Verdict logic + reason strings
unchanged → byte-identical verdicts (full suite 1873 green proves it). _gate_verdict resolves once.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 14:16:18 +02:00
jgrusewski
cba9f6531b feat(policy): resolve_policy — per-track overrides on GatePolicy defaults, one resolution point (B1)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 13:37:00 +02:00
jgrusewski
5a99bb6329 feat(policy): versioned GatePolicy + content-derived policy_hash (B1)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 13:33:34 +02:00
jgrusewski
1745249cf4 refactor(forward): gate ETL + allocator read the DB record; retire state_reader JSON path
forward_ingest now evaluates the gate from the DB record the engine writes (all_summaries +
nav_history) instead of reading *_state.json via ForwardStateReader; _sleeve_return_series reads
nav_history too. ingest_forward_state drops state_dir. state_reader.py + its test deleted — the
DB anchor + record are the single source of truth (spec sub-project A). Gate verdict logic unchanged.
Docstrings updated to reflect the engine path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 10:02:28 +02:00
jgrusewski
a7533980da feat(forward): wire combined through the engine (recompute); drop track-level killswitch
combined_forward_nav now runs CombinedBookStrategy through forward_engine.run_track (DB anchor +
recompute) — the last track off the JSON ForwardTracker. The former ForwardTracker-level drawdown
killswitch overlay is intentionally dropped (a track throttle, not part of the edge; vestigial
research track, not capital). record_mode observed->recompute (version 2) -> clean re-inception.
Removes now-unused build_combined_forward_nav / CombinedBookForwardTracker registration in
migration_builders.py + the test_orchestration_assets.py test (its only exerciser). Also drops the
"combined" entry from the one-time migrate-forward-anchors builder map (record_mode is now
recompute, so the observed lambda:None placeholder would crash it; combined re-inceptions cleanly
through run_track with no JSON left to migrate).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 09:38:24 +02:00
jgrusewski
423aad7911 feat(forward): pure recompute_series + derive_nav (matches ForwardTracker formula)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 00:02:20 +02:00
jgrusewski
c3451560f5 fix(forward): resolve Task 1-3 review minors (enforce single-active anchor, validate dates, tighten tests)
- set_anchor self-guards the single-active-anchor invariant (archive prior active on re-inception)
- append_record/record_series validate + normalize ISO-8601 dates (fail loud on malformed)
- ordering test now inserts out-of-order; removed dead test imports/vars; clean Session import

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 23:57:05 +02:00
jgrusewski
15572f0e60 feat(forward): declared definition + content hash + per-track definition entries
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 23:27:36 +02:00
jgrusewski
f1916404b4 fix(gate): tolerate weekday-only gaps (equity tracks) via max_gap_days
The reconciliation gate's _has_gaps assumed calendar-continuous booking (crypto is
24/7). A weekday-only US-equity NAV (multistrat) has Fri->Mon 3-day steps + holiday
long weekends (<=4d) — NORMAL, but the gate would flag them as an execution gap at
day 20 -> perpetual false-WAIT (never reconciles). Now _has_gaps takes max_gap_days
from gate_spec (default 1 = crypto, unchanged); multistrat sets 4. A step of
0/negative (duplicate/out-of-order) is still always a hole.

Surfaced verifying the just-added multistrat gate track (14/20 building, real ref
CAGR 9.9% / Sharpe 1.48). Full suite green (1833).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 09:49:33 +02:00
jgrusewski
765402062a fix(gate): withhold PASS when no backtest reference (was a false-PASS)
The reconciliation gate degraded to a bare min-window + clean-exec timer when no
backtest_summary row existed, yet still returned PASS labelled 'reconciles with
backtest' — a hollow/false-PASS. For a real-capital gate the safe error is a
false-WAIT, never a false-PASS. Now backtest=None -> WAIT ('no backtest reference
to reconcile against — PASS withheld'). Reverses the prior degrade-to-PASS test.

Surfaced by positioning showing PASS +1.82% (one +15.3% anomaly day) with no
backtest_summary row; crypto_tstrend + stablecoin_rotation had the same hollow PASS.
Fix B (persist per-strategy backtest refs) makes their gates reconcile for real.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 20:51:07 +02:00
jgrusewski
12615cf90a refactor(cockpit): retire xvenue_carry forward track (H3) — light retire
xvenue_carry (cross-venue Bybit<->Deribit static carry) is VALIDATED-BUT-NOT-LIVE
with ~0 forward contribution (measured: +0.004%/day, dilutes book CAGR -18pp at
equal weight). Retire the forward-tracked strategy: registry entry, xvenue_carry_nav
asset + build_xvenue_carry_forward_nav, definitions wiring, cockpit backtest-ref
mapping, include_xvenue_carry persist path. KEEP the cross_venue_carry_eval research
lib + deribit_funding_bars ingestion (data stays warm for a possible cross-venue
execution revival). Asset count 21 -> 20; tests updated to assert the retire.
Full suite green (1816).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 20:22:30 +02:00
jgrusewski
5b3a173411 revert(gate): keep tight recon band (no dispersion-widening) — real-money gate must err toward WAIT, not PASS; keep 21d window
The dispersion-adaptive band widened the cumulative-return tolerance with the
forward's realized dispersion. But in prod the correlation sub-check does NOT run
(CockpitBacktestRefProvider passes daily_returns=None), so the cumulative band is
the ONLY always-on divergence check. Widening it there makes a real-capital gate
MORE likely to PASS a non-reproducing/underperforming forward with no correlation
backstop. The error asymmetry for a real-money gate: false-WAIT costs only time
(safe); false-PASS deploys on a dead edge (dangerous). The gate must err toward
WAIT, so the band must NOT be loosened.

Restore the fixed tight band `tol_frac * max(|expected|, min_band)` (0.5, 0.05):
remove RECON_DISPERSION_K, the _pstdev realized-dispersion term, and the
max(..., k*dispersion) widening. The correlation sub-check code is kept as-is
(harmless; fires only when a daily series is supplied, e.g. in tests). The lumpy
edge is handled the safe way — the bybit_4edge min_forward_days 14->21 bump stays
(a longer, more representative window lowers false-WAIT without loosening pass).

Tests: replace the adaptive-band tests with tight-band contract tests (band is the
fixed formula; a lumpy negative window below the fixed band WAITs with NO dispersion
widening; a steady mirage below the band WAITs on the cumulative check alone). The
21d-window test updates stay.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 13:26:31 +02:00
jgrusewski
e077521571 fix(gate): dispersion-adaptive recon band + 21d window for lumpy bybit_4edge
The 4-edge book's gains arrive in irregular clusters (big +days a median ~17 /
mean ~36 days apart), so a fixed 14-day cumulative-return band trips a false
"diverging (below expected)" WAIT ~15% of the time purely by spike-timing — the
edge is fine, no spike just landed in the window.

reconciliation_gate: make the shortfall band dispersion-adaptive —
  band = max(tol_frac*|expected|, min_band, RECON_DISPERSION_K * std(fwd)*sqrt(days))
with RECON_DISPERSION_K=2.0 (a shortfall must exceed ~2 window-sigma to count as
divergence; the historical dry/lumpy 14d 5th-pct ~-2.5% ≈ 1.65-sigma is absorbed).
The max() guarantees the band NEVER narrows below the old fixed floor — a steady
low-dispersion bleed still trips exactly as before, and the daily-return correlation
sub-check (RECON_MIN_CORR=0) remains the primary mirage-catcher, unaffected by
lumpiness.

registry: bump bybit_4edge min_forward_days 14 -> 21 (that book ONLY); a lumpy edge
needs a slightly longer window (median ~17d between spikes) to be representative.

Tests: 3 new safety cases (no false-negative on a lumpy-but-real window; mirage-catch
intact on an anti-correlated lumpy window; band never narrows below fixed + a true
sustained-shortfall+neg-corr mirage still WAITs). Updated the e2e + forward-ingest +
web tests for the 21-day bybit_4edge window.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 13:13:25 +02:00
jgrusewski
f649b5b267 fix(carry): winsorize per-symbol daily carry return at ±25%/day
The xsfunding (carry) sleeve booked +179.7% (book +44.69%) on 2025-02-02 —
a bad spot/perp close blew up the daily-close basis term (spot_ret − perp_ret),
not a capturable edge (next-largest day ever is +8%). At its cross-sectional
weight that lone print dominated Σ w·carry_ret/Σ|w| and inflated the backtest
baseline the forward gate reconciles against.

Winsorize the DAILY-CLOSE carry return `funding + basis` to ±_CARRY_RET_DAILY_CAP
(±25%/day) in carry_return_by_symbol — the SSOT both the backtest baseline and the
live book read — so the cap is retroactive (any curve recomputed on read) AND
prospective (a future glitch is capped, not dropped). The deliberate intraday
worst_basis liquidation charge (own −100% floor) is left UNwinsorized.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 12:51:39 +02:00
jgrusewski
78c625a99b feat(broker): add BybitExecution.open_orders for testnet idempotency guard
Wraps ccxt fetch_open_orders so the testnet reconcile can check for resting
orders before placing (design §4.3 pre-place guard). Adds adapter tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 16:17:41 +02:00
jgrusewski
d05b1697a7 feat(config): BybitExecutionConfig with triple-gate
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 14:24:49 +02:00
jgrusewski
3ed0cd83da feat(broker): ccxt Bybit testnet execution adapter
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 14:21:35 +02:00
jgrusewski
421317ae00 feat(exec): bybit testnet diff order planning (no overlay re-apply)
plan_orders diffs the raw bybit_4edge sleeve weights straight against
testnet positions: target_qty = weight·equity/mark snapped to qtyStep,
dust-filtered on minNotional/minOrderQty, delta vs current, reduce_only
on reductions/closes, gross capped at max_gross=1.0. Frozen DTOs
OrderIntent / InstrumentLimit (stable contract for Tasks 5/8). Pure.

Task-1 gate confirmed bybit_4edge is naive equal-weight with NO risk
overlay (see bybit_paper_book docstring + latest_raw_sleeve_weights), so
the design's effective_weights/overlay-reapply is dropped — the raw
weights ARE the effective weights.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 14:07:39 +02:00
jgrusewski
7fa06ced65 feat(exec): execution-gap mechanics verdict + indicative slippage
mechanics_verdict (PASS iff every post-rebalance position tracks its
effective target within tolerance + no error/gap rows; else FAIL naming
offending symbols) and indicative_slippage (per-tier mean slippage-vs-
assumed excluding low_confidence fills, reporting coverage). Frozen DTOs
PositionRow / SlippageFill / Verdict. Pure (design §5).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 14:05:18 +02:00
jgrusewski
bd73e02774 feat(exec): pure decomposed slippage model
execution_slippage_bps / timing_drift_bps / slippage_vs_assumed_bps —
pure float math (spec §3) for the Bybit testnet real-paper leg. BUY/SELL
sign, bps math, None-guards on non-positive mid/book_mark.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 14:04:03 +02:00
jgrusewski
0b59e8cabe fix(crypto-fetch): drop non-ASCII Binance perp symbols from the universe
Binance `exchangeInfo` can return junk meme symbols with non-ASCII (e.g. Chinese) characters
(`币安人生USDT`) that crash the klines GET with "'ascii' codec can't encode". Filter the universe to
ASCII-only symbols (still PERPETUAL / USDT-quote / TRADING as before).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 09:47:35 +02:00
jgrusewski
dfe001f969 perf(cockpit): materialize the headline nav-summary (stats + curve) — read 1 row, not 3650, per request
_overview_headline re-read the full ~3650-row bybit paper_nav series and re-ran
nav_backfill_stats + the multi-year curve SVG render on EVERY / and /paper load.
Under shared-DB contention (postgres is shared with dagster/jobs) that read spikes
and pushes the pages over the 1s budget. The stats+curve only change when the nav
changes (nightly).

MATERIALIZATION, not a cache: a new paper_nav_summary table (venue PK) stores the
stats + since/has_record + the pre-rendered curve SVG, versioned by the nav's latest
run_date. refresh_nav_summary computes it ONCE via the EXACT same helpers
(nav_backfill_stats + nav_sparkline), so the stored headline + curve are byte-for-byte
identical to the live compute. The read path runs a cheap 1-row version query (latest
run_date) and serves the stored row when the version matches — no full-nav read — and
self-heals (refreshes, the only full-nav read, at most once per nav change) when the
summary is missing/stale. The nightly nav-write path (build_bybit_paper_book +
bybit-paper-backfill CLI) refreshes proactively so the first request after a nav
update is already fast. ANY failure in the materialized path falls back to the current
live compute, so the headline never 500s and its numbers/curve are never wrong.

Verified byte-identical rendered / and /paper HTML (fast + self-heal paths) vs before.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-27 16:12:37 +02:00
jgrusewski
f8b5426a16 fix(cost): use real L1 quoted spread for the Bybit liquid book (was daily-CS, 104x too harsh)
The Bybit liquid book was costed off a daily-high/low Corwin-Schultz spread
proxy that over-states the true quoted spread ~104x (measured 1.4bp real vs
143.6bp CS), so the cockpit showed a 0.22-Sharpe cost-model artifact instead of
the book's true ~1.5-1.8 Sharpe.

- Part 1: `bybit_spread_refresh` samples the live L1 top-of-book several times
  (injectable sleep, no bare time.sleep) and stores the MEDIAN per coin in a new
  `bybit_real_spread` table (idempotent upsert + reader on PaperRepo). New CLI
  `fxhnt bybit-spread-refresh [--samples N]`.
- Part 2: `_bybit_measured_net` + the precompute take an optional
  `real_spread_by_coin` override (reuses the maker-recost `recost_net_series`
  plumbing at f=0 = exact measured formula); coins without a stored real spread
  fall back to CS. Threaded for bybit_4edge ONLY -- binance_combined stays on CS
  (its broad/illiquid mirage finding stands).
- Part 3: the cockpit caption honestly states "real L1 quoted spread
  (forward-realistic -- current liquidity; 2021-22 spreads were wider, so early-
  history cost is understated)"; the data-cost-mode="measured" marker is kept.
- Part 4: the daily compare-measured-precompute CronJob now runs
  `bybit-spread-refresh && compare-measured-precompute`; added 443 egress for
  api.bybit.com.

GROSS returns unchanged (cost-only fix). Tests: median-of-samples, missing/zero
skip, idempotent upsert, reader round-trip; override lowers cost-drag + lifts
Sharpe; partial-override CS fallback; precompute threads real spread for bybit
not binance; caption renders with marker intact. Full suite: 1687 passed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 23:19:51 +02:00
jgrusewski
11046441cf fix(forward): defensive Binance ticker parsing in unlock_calendar_live — unblocks the nightly gate
unlock_nav threw KeyError: 'quoteVolume' on ticker rows missing the field (newly-listed/settling
symbols); cockpit_forward deps=[...unlock_nav...] so it cascaded and froze the whole forward gate
(deploy book bybit_4edge stuck at 0/14). _perp/_funding_qvol now .get(...) or 0.0 + skip rows
without 'symbol', matching the existing defensive lastFundingRate handling.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 16:33:47 +02:00
jgrusewski
2e43c61e66 feat(cockpit): SSOT registry-driven console — deploy/research tiers, paper/live badges, one-entry fast-track
The registry is now the single source of truth for the cockpit: every STRATEGY_REGISTRY
entry declares `tier` (deploy|research) and `execution` (paper|live). The Overview renders
strictly from those tiers — DEPLOY (the Bybit fund: connected backfill headline + its
individual constituent edges + deploy-tier forward tracks) above a visually-dimmed RESEARCH
section ("not the fund") grouped by venue. Every track carries a PAPER/LIVE badge driven by
`execution`. Adding a strategy = ONE registry entry; it auto-appears in the correct section.

Reusable deploy/research grouping + exec-badge macros so the same structure can roll to the
Paper page next. Presentation + registry metadata only — no edge returns or gate logic changed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 08:44:28 +02:00
jgrusewski
f0ded1247f feat(cockpit): redesign Overview — connect the backfill record, provisional pre-gate Sharpes, cut clutter
Connect the bybit deploy book's multi-year BACKFILL (persisted paper_nav, venue="bybit") to the fund
headline so the page leads with the REAL record ($393,955 · +294% · real Sharpe · maxDD · since 2021)
plus a small live-readiness chip (gate days/min · status) — never the not-started forward track's
0.00 / 0 forward days.

- nav_stats.nav_backfill_stats: pure, unit-tested helper computing total return / annualised Sharpe
  (mean daily ret / pstdev × sqrt(365)) / maxDD / final equity from a persisted NAV equity series.
- Short pre-gate forward Sharpes are kept ("fun to watch") but rendered DIMMED with a "· pre-gate"
  marker; cleared tracks render bright. One caption explains it.
- Compact edges table (edge · class · record · forward · nav); not-started 0-day trackers collapsed
  into a "warming" subsection so they don't dominate.
- Cut clutter: removed the two prose paragraphs (folded into title= tooltips), removed the duplicate
  "Go to" footer, and replaced the grouped Live/Backtest nav with one tidy row
  (Overview · Paper · Replay · Backtest).
- Reusable Jinja macros (_macros.html: headline_card, provisional_num, track_row) for the
  cockpit-wide consistency roll-out. FleetRow gains gate_min_days for the forward "days/min" denominator.

Presentation + data-connection only — no edge return or gate logic changed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 08:07:42 +02:00
jgrusewski
d2429ef233 Merge: reconciliation-based forward gate (replaces blind 60-day wait)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 00:53:11 +02:00
jgrusewski
0a7ca60722 feat(gate): reconciliation-based forward gate (~2wk + forward-tracks-backtest + clean exec) replacing blind 60-day wait
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 00:46:10 +02:00
jgrusewski
19153c289a fix(config): default feature store to postgres (duckdb now explicit opt-in for tests/B3)
A warehouse-ingest Job that forgets FXHNT_FEATURE_STORE=postgres silently
wrote to a throwaway DuckDB file instead of the live postgres features
table, because get_feature_store defaulted to duckdb. Flip the default to
postgres (the live stack) so this can't recur; duckdb stays as the explicit
opt-in for the embedded store (tests + the B3 backtest). Only a default flip
— the duckdb branch and stores are unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 00:32:46 +02:00
jgrusewski
ff3fe62644 feat(data): capture real daily high/low through the crypto pipeline (was high=low=close)
Capture true daily OHLC ranges end-to-end through the binance/crypto data
pipeline so the warehouse `features` table stores real high/low instead of
the degenerate high=low=close. This unlocks real Corwin-Schultz measured
spreads for the binance book.

- crypto_fetch._klines: extract HIGH (kline idx2) and LOW (idx3) per row;
  return (day, close, high, low, qvol). fetch_crypto_pit writes high/low into
  the npz (keys: day, close, high, low, qvol, funding).
- normalize_crypto: new optional (high, low) args; uses real H/L when given,
  falls back to high=low=close otherwise (backward-compat for old npz/callers).
- Readers (ingest_crypto_pit + ingest-warehouse CLI crypto branch): read
  high/low from npz when present, else fall back to close — old npz without
  high/low still ingest, no crash.
- close (and thus combined-book returns) unchanged; only high/low are added.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 23:16:18 +02:00
jgrusewski
bdc56dc1ea feat(cockpit): /paper/sim compare mode — Binance vs Bybit books, same period + same cost, overlaid
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 20:34:52 +02:00
jgrusewski
060d747c7a chore: remove dormant strategy-factory subsystem (GENERATE→JUDGE→PROMOTE never activated) — keep combined_book_factory + all edges/live
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 11:59:45 +02:00
jgrusewski
f3fadcdfa8 fix(cockpit): vol-targeted comparable sleeve metrics (was misleading raw cumulative) + backtest-reference on single-edge detail pages
Issue 1: the bybit_4edge "Sleeves" breakdown showed each sleeve's RAW
UNSIZED cumulative return (positioning +4228%/42x) — unsized, and the
sleeves span different histories so the totals aren't comparable. Replace
with vol-targeted comparable metrics: scale each sleeve's daily series to
a common 20% annual vol (_SLEEVE_VOL_TARGET), then annualise (CAGR of the
vol-scaled series, ~= Sharpe x 0.20) with a maxDD on that consistent
basis. New vol_targeted_metrics() helper alongside _standalone_sharpe.
Breakdown restricted to the 4 book constituents.

Issue 2: single-edge detail pages (positioning, xvenue_carry) showed only
"0 forward days · +0.00%" while the forward track is young. Add a
"Backtest reference" card (Sharpe + vol-targeted annual return + maxDD,
same basis) sourced from the precomputed bybit_sleeve_ret table.
positioning -> its own sleeve; xvenue_carry -> the BTC/ETH static
cross-venue carry series, now also persisted (include_xvenue_carry) under
an auxiliary key by persist_bybit_sleeve_returns / the persist CLI. The
not-live flag stays visible; strategies with no source omit the section.

Read-only, precomputed series only (no recompute on the request path),
relative URLs preserved. Full suite green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 01:39:07 +02:00
jgrusewski
f41c7a04d7 feat(cockpit): surface positioning + cross-venue-carry as forward-tracked edges + Bybit book sleeve breakdown
Part A — two new forward-tracked edges (own rows + OOS forward records):
- registry: `positioning` (single Bybit positioning sleeve) + `xvenue_carry`
  (Bybit↔Deribit BTC/ETH static carry, with honest status="not-live"/note flag).
- `build_positioning_forward_nav` / `build_xvenue_carry_forward_nav` plain drivers
  + nightly `positioning_nav` / `xvenue_carry_nav` Dagster assets (wired into defs +
  cockpit_forward upstream). Both READ-ONLY (state JSON + forward_nav upsert only),
  gate=WAIT until 60 forward days.
- positioning daily ret = `sleeve_returns_from_store('positioning')` via the
  single-sleeve `naive_eqwt_daily_returns(..., ['positioning'])` path; xvenue daily
  ret = `cross_venue_carry_returns(mode='static')` via new `CrossVenueCarryStrategy`.
- not-live flag surfaced on the registry entry, the edges-table row, and the detail page.

Part B — Bybit 4-edge book sleeve breakdown on /strategy/bybit_4edge: per-sleeve
standalone backtest Sharpe + total return from the precomputed bybit_sleeve_ret
(reuses _sim_returns + _standalone_sharpe), book-type strategies only, READ-ONLY,
inline text, relative URLs.

Tests: registry entries + gate_spec + not-live flag + no-prune; both trackers build
forward_nav from a fake store, advance, idempotent, gate=WAIT, WriteTripwire READ-ONLY,
daily return matches the eval construction; cockpit edges table + not-live row/detail +
4-sleeve breakdown; leak-clean. Full suite green (1392 passed).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 00:59:33 +02:00
jgrusewski
98c8f950ad feat(vrp): defined-risk VRP (straddle + protective wings, BS-priced from DVOL) — caps the naked-short-vol tail
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 20:21:06 +02:00
jgrusewski
07e0daa669 feat(cockpit): Bybit 4-edge book — configurable backtest (precomputed sleeve returns) + live forward track, clearly labeled
Present the Bybit 4-edge book in the cockpit as BOTH a configurable backtest
AND the live forward track, clearly distinguished.

- bybit_sleeve_ret table + PaperRepo upsert/read: the PRECOMPUTED per-sleeve
  daily returns the sim reads (never recomputes the heavy 4-edge book on a web
  request). Mirrors paper_sleeve_ret's shape.
- bybit_4edge_nav asset also persists the 4 sleeves' full daily return series
  via persist_bybit_sleeve_returns (same live runners, memory-bounded, guarded
  so it never fails the forward track). Idempotent.
- /paper/sim book selector: binance_combined (overlay, existing default) and
  bybit_4edge (NAIVE equal-weight, NOT the overlay — A/B proved naive best OOS)
  via simulate_naive_eqwt; capital/start/cost_bps are the cheap in-memory knobs.
- Bybit view shows the configurable BACKTEST curve AND the live FORWARD track
  (forward_nav) labelled distinctly, with the forward gate status / N-of-60 days.
- Tests: naive eq-wt sim unit tests; asset persistence + idempotency + no-clobber
  + matches bybit_book_eval baseline; book-selector web smoke incl. a tripwire
  proving the request path never calls the heavy edge compute; Binance sim
  unchanged. Full suite green (1147 passed).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 17:27:10 +02:00
jgrusewski
6daf43b03c feat(unlock): unlock calendar in operational DB (networked) — removes the last PVC dependency for unlock
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 09:34:45 +02:00
jgrusewski
5cdce4c6b8 perf(bybit): bounded-concurrency ingest (ThreadPool) + rate-limit backoff (~5-10x faster, memory-bounded, resumable)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 22:45:14 +02:00
jgrusewski
274e974854 perf(sim): O(1)/day overlay loop + expensive/cheap split — simulate_book full run <1s (bit-identical)
Part A — make the per-day overlay loop O(1)-amortised (bit-EXACT, no numeric change):
- `_regime_lookback`: build the NaN matrix over only the last 2·L_min days (the rest of `rep`
  is never read) + look up the ≤20 tail days directly instead of scanning each sleeve's full
  dict. O(active·L_min) not O(active·D). Bit-identical (same numpy ops on the same values).
- `paper_book_returns` + `IncrementalRiskOverlay`: feed `adaptive_target_vol` a running list
  (`out_vals`) mirroring `out`'s chronological values, instead of rebuilding `list(out.values())`
  (O(D)) every day. Bit-identical (same values, same order; only the [-win:] tail is read).
  Keeps `paper_book_returns`'s exact `==` golden (the live==replay regime signal) untouched.

Part B — separate the expensive overlay from the cheap knobs (instant slider drags):
- `overlay_pass(returns, sleeves, controller, kelly, start, end) -> OverlayPass`: the EXPENSIVE
  half (per-day weights/eff_lev/killed/gross_ret/eff_w). kelly stays here — it does NOT factor
  out linearly (clamped by max_leverage + sizes the killswitch's regime signal).
- `apply_knobs(OverlayPass, capital, cost_bps) -> SimResult`: the CHEAP half (~3ms on full
  history); capital + cost_bps are the genuinely-cheap knobs. Bit-identical to full simulate_book.
- `simulate_book` composes the two internally; signature + behaviour unchanged.

Timing: full ~1473-day run ~1.7s -> ~1.15s; cheap capital/cost re-apply ~3ms (was ~1.2s/tick).

Tests (TDD): frozen pre-optimization golden (full SimResult, abs_tol 1e-9, 4 controller/kelly/
cost/capital combos); cheap-reapply == full simulate_book (bit-exact) over many knob combos;
overlay-pass capital/cost-independence; perf bounds (full <1.6s, cheap <50ms). Full suite green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 15:13:27 +02:00
jgrusewski
4e3938751b feat(sim): pure in-memory simulate_book engine (config -> equity curve + per-day state + metrics)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 14:32:52 +02:00
jgrusewski
b5301a0fff feat(carry): 1m worst_basis ingest (binance.vision) + intraday-liquidation charge in carry accounting (Parquet side-store)
Charges the carry book's INTRADAY basis-squeeze / liquidation tail that the
daily-close basis accounting was blind to (probed: LUNAUSDT 2022-05-12 intraday
worst_basis ≈ −163% vs daily-close −0.7%).

- application/worst_basis.py — PURE cumulative-MAE: {epoch_day: worst_basis} =
  min over the day's aligned minutes of Σ(spot_1m_ret − perp_1m_ret).
- adapters/data/binance_vision_klines.py — resumable/cached 1m monthly-kline
  fetch+parse → {epoch_ms: close}; µs→ms, missing month = empty (no crash),
  injectable fetch (no network in tests).
- adapters/persistence/worst_basis_store.py — Parquet SIDE-STORE (duckdb I/O,
  NOT the warehouse → no single-writer contention) with load/write round-trip.
- paper_book.carry_return_by_symbol — optional worst_basis_by_symbol overrides
  the daily-close basis with max(worst_basis, −1.0) (capped at total leg loss);
  absent → daily-close basis, bit-identical. Threaded through paper_backfill
  (worst_basis_by_date) + paper_snapshot (worst_basis) + assets, live==replay.
- CLI: fxhnt ingest-worst-basis --symbols <list|@file> --from --to (resumable).
- TDD: LUNA-pattern fixture, parquet round-trip, charge+cap+fallback, absent=
  bit-identical backfill curve. Full suite 812 passed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 13:38:51 +02:00
jgrusewski
caa4ae6f8e fix(paper): carry books full return (funding + daily basis), not funding-only — collapses fake Sharpe 12.55 -> ~1.56
The merged carry accounting booked the xsfunding sleeve return as funding
only (Σw·funding/Σ|w|), dropping the delta-neutral position's basis P&L
(spot_ret − perp_ret) where the LUNA-class tail lives — a fake Sharpe
12.55 / −0.7% maxDD. Book the FULL daily carry return per symbol =
(spot_ret − perp_ret) + funding for BOTH the persisted paper_sleeve_ret
AND the equity P&L contribution.

- funding_panel.basis_change_by_symbol: per-symbol daily basis change
  (spot_ret − perp_ret) with nearest-prior-day gap handling; missing leg
  → omitted (funding-only fallback, documented).
- paper_book.carry_return_by_symbol: single pure helper merging funding +
  basis into the full carry return, consumed by BOTH sleeve_daily_carry
  (return) and carry_pnl (equity) so they can't drift. carry_pnl/Σ|notional|
  == sleeve return invariant preserved with the basis term.
- backfill/snapshot/CLI/assets: build + thread basis per date alongside
  funding (warehouse spot+perp+funding panels); snapshot computes carry the
  same way → live == replay. basis absent → funding-only (back-compat).
- Caveat updated: basis now charged at DAILY-close resolution; remaining gap
  is the intraday 1m worst_basis liquidation tail (documented future work).
- Directional sleeves (crypto_tstrend/unlock/stablecoin) bit-identical
  (golden). Full suite green (773 passed).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 10:53:15 +02:00
jgrusewski
7d96ff2a00 feat(paper): combined risk-stack controller — K(D*) sizing + WEH structural-retirement gate + killswitch backstop
Add controller="combined" = K(D*) sizing + per-sleeve WindowedEdgeHealth
structural-retirement gate + the existing DrawdownKillSwitch backstop. The only
behavioural delta vs "killswitch" is the retirement gate filtering the active
sleeve set; with all sleeves healthy combined == killswitch bit-identically.

- RetirementGate (domain/edge_decay.py): pure hysteresis state machine over a
  WindowedEdgeHealth — retire after retire_window sub-retire_floor days, re-admit
  after readmit_window >= readmit_floor days; bootstrap-safe; to_dict/from_dict.
- Wire combined into both overlays (live==replay): paper_risk_overlay replays the
  gate from scratch; IncrementalRiskOverlay keeps per-sleeve gate state advanced
  over newly-appeared days. ISV over survivors; leverage + killswitch unchanged.
- Tests: pure state machine (retire/dip/re-admit/hysteresis/bootstrap/roundtrip),
  inert-on-healthy (combined==killswitch exact), synthetic dead-sleeve (combined
  retires B and beats killswitch NAV; re-admits on recovery), live==replay golden
  with a dying sleeve. Full suite green (756 passed).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 09:04:18 +02:00
jgrusewski
8f00406517 feat(paper): scale-invariant WindowedEdgeHealth detector + wire into edge_health controller (replaces broken PH-health)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 00:37:58 +02:00
jgrusewski
b31eb50a18 feat(paper): drawdown controller v1a — DD-budget Kelly + per-sleeve edge-health (controller flag, default killswitch)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 00:10:39 +02:00