Move the recompute-replay sim_curve_ret persist to after the `if sm is None: return`
guard so it is gated on the SAME condition as the reconciliation-gate backtest_summary
write. A too-short replay (< 2 rows, no gate ref) now writes NEITHER, never a degenerate
sim curve with no matching gate ref. Adds test_insufficient_replay_writes_neither_ref_nor_sim_curve.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Generalizes the Backtest page beyond the Bybit-only venue: bybit_4edge/bybit_4edge_levered
keep reading the precomputed bybit_sleeve_ret table; the IBKR paper strategies (multistrat,
vrp) now read their own precomputed recompute-replay curve from a new sim_curve_ret table,
written nightly by _persist_track_backtest_ref from the SAME rows the reconciliation-gate
ref is derived from. New sim_curves.sim_returns_for dispatches on registry_backtest_kind;
app.py wires an _ibkr_measured_context render path (reusing the existing pure
simulate_naive_eqwt engine) alongside the unchanged Bybit measured-cost path, with
plain-language pill labels for the two IBKR books.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
crypto_tstrend's standalone STRATEGY_REGISTRY entry was retired in Phase 0b
Task 4, but the still-live crypto_tstrend book SLEEVE in the bybit_4edge
per-edge breakdown (Overview + /paper) shared the same string id, so its
display-name lookup blanked to the raw id and linked to a dead
/strategy/crypto_tstrend (404).
Add _SLEEVE_DISPLAY_NAMES next to _DEFAULT_BYBIT_SLEEVES in
bybit_book_eval.py — the sleeve breakdown's own display-name source,
independent of STRATEGY_REGISTRY, so a future forward-track retirement can
never blank a live sleeve again. _deploy_individual_edges() now sources
display_name from that map and adds a `linkable` field (true only when the
sleeve id is also a real registry entry); deploy_constituent_row renders the
/strategy link only when linkable, plain text otherwise.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The vrp health axis flagged vrp=STALE because the OPRA freeze
(xsp_option_bars) was stuck: vrp_nav froze as_of=today, but today's
session is only partially available intraday, so the freeze failed with
`422 data_schema_not_fully_available` (distinct from the C1a clamp's
`data_end_after_available_end`). That made the freeze timing-dependent —
it could only succeed in a narrow post-settlement window and otherwise
silently degraded to recompute-off-stale-frozen.
Make it robust and timing-independent:
- databento: normalize the `data_schema_not_fully_available` 422 into the
single catchable DatabentoRangeUnavailable (surgical — only that 422;
any other client error propagates raw). Expose
last_available_option_date() as the step-back's starting candidate.
- assets: new _freeze_latest_session entry point walks back one TRADING
day (skip Sat/Sun) from OPRA's available-end until a COMPLETE session
freezes, bounded to 5 steps; returns/logs the frozen session date.
Never freezes a partial session's marks.
- No silent degradation: if no complete session is found in the bound it
raises DatabentoRangeUnavailable; the existing C1b wrapper in vrp_nav
catches it → recompute off frozen, and the health axis keeps vrp STALE
(loud), never a quiet no-op.
TDD: step-back-past-partial, weekend-skip, bound-exhausted-raises, 422
normalization (both chain + bars paths), unrelated-error-propagates,
last_available_option_date caching. Full suite green (one pre-existing,
unrelated seed-dependent gauntlet flake).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The "no forward data yet" carve-out in compute_health let a reconciliation-gated
recompute track that never produced data since inception (empty freeze table / no
forward_summary, old anchor t0 or no active anchor at all) read HEALTHY the same
way a genuinely brand-new track does. Thread the active forward_anchor.t0 through
evaluate_health so compute_health can tell "just inceptioned" (anchor within the
stale threshold -> HEALTHY) apart from "broken from inception" or "never anchored"
(-> STALE), closing the silent-degradation hole without false-alarming new tracks.
Also corrects the module docstring: BROKEN only covers per-track compute failures,
not the migrate/all_summaries/all_backtest_summaries/xsp_freeze_max_date reads
evaluate_health hoists outside the per-track try (those fail the whole
cockpit_forward Dagster asset instead).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
vrp rotted for 2 weeks because a data failure degraded QUIETLY (recompute off a
frozen-but-stale table, only a log line) and the gate can't tell a stale track
from a healthy WAIT. Add a HEALTH axis SEPARATE from the gate so ANY live/gated
track that stops updating surfaces LOUDLY on day 1.
forward_health.py (new): compute_health (pure state machine) + evaluate_health,
called by cockpit_forward AFTER ingest. States {HEALTHY, STALE(age), NO_REF,
BROKEN}, precedence BROKEN>STALE>NO_REF>HEALTHY:
- STALE: a `recompute` track whose source as_of OR its freeze table's max date
(vrp → xsp_option_bars) is > Settings.stale_after_business_days (default 3)
business days behind. The freeze axis catches the exact vrp mode: forward
"updates" yet the frozen input is stale.
- NO_REF: a reconciliation-gated recompute track with no resolvable backtest
reference (the gate can never reconcile) — the M6 static invariant made a real
runtime check.
- record_mode carve-out: `observed` twins (vrp_exec/…) are externally-driven and
legitimately DORMANT when execution is suspended — never staleness/no-ref
alarmed (won't false-alarm the intentionally-dormant reference_only tracks).
- BROKEN: a per-track health-compute error is surfaced, never swallowed.
Storage: new strategy_health table (create_all — idempotent, Postgres-safe, no
ALTER), replace-all each run so a recovered track clears its stale row.
Rendering: FleetRow/StrategyDetail gain health/health_stale_days/health_reason;
a LOUD red health_badge (STALE 14d / NO REF / BROKEN) on Overview, the all-tracks
table, and the detail header — visually impossible to confuse with a WAIT.
cockpit_forward + evaluate_health log at ERROR with the unhealthy list.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
cockpit_forward reads purely from the DB (forward_summary) and its deps are
ordering-only, so a skipped upstream stalls the whole gate/promotion/allocation
pipeline. Close the two pre-tracker offenders so no *_nav asset raises on a
data-provider failure:
- multistrat_nav: the snapshot_yahoo_closes loop runs OUTSIDE _run_paper_tracker;
a Yahoo outage now logs ERROR and degrades to a no-op {forward_days:0,...}
return (record unchanged) instead of raising and skipping the gate.
- _run_paper_tracker: broaden the caught set to also isolate the databento
provider error base (BentoError/BentoClientError, imported lazily since
databento is an optional extra). Bare Exception is NOT caught — genuine
strategy-math bugs (KeyError/ValueError) still propagate loudly.
With every expected data-provider failure mode isolated, cockpit_forward always
runs (deps kept for ordering). See report for the decouple-approach rationale.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
vrp_nav hard-failed every night: _freeze_day runs OUTSIDE _run_paper_tracker
and databento 422'd (data_end_after_available_end) because end=as_of+1day
exceeded OPRA's ~22:30-UTC available end at the 23:30 freeze. The raised
BentoClientError killed the asset before run_track/set_anchor, zeroing vrp's
forward anchor AND skipping the downstream cockpit_forward gate.
C1a: DatabentoDataProvider now fetches the OPRA available end via
metadata.get_dataset_range (cached once per instance) and clamps the exclusive
query `end` down to it in fetch_option_chain/fetch_option_chain_range/
fetch_option_bars, so no query requests past what OPRA has published. An empty
window (start >= clamped end) raises catchable DatabentoRangeUnavailable, never
a raw 422.
C1b: vrp_nav wraps _freeze_day so any freeze failure logs a WARNING and
continues to _run_paper_tracker, which recomputes VrpStrategy off the FROZEN
xsp_option_bars table — guarantees a forward row/anchor/ref even on a day the
freeze fails, and never skips the gate.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Task 7e only renamed venue="binance" -> "legacy" to pass the substring guard;
the dual-venue repo machinery was left in place even though bybit is the ONLY
live venue. PaperRepo now hardcodes every discriminated read/write to a
_BYBIT_VENUE = "bybit" module constant instead of a constructor venue param
(removed) -- reads/writes ONLY the bybit rows, so the orphaned legacy/binance
rows from the retired combined-crypto book (Task 7d) are never touched.
nav_summary's venue arg is dropped the same way (every caller always passed
"bybit"). cockpit_models keeps the venue column (dropping it from the PK is a
separate prod schema migration, documented in the report for the merge
runbook) with reworded comments reflecting the single-venue reality.
Test fallout: deleted test_paper_repo_venue.py (tested the now-removed
dual-venue isolation) and 3 "does-not-touch-binance" tests that constructed
PaperRepo(venue="binance"); mechanically dropped the venue kwarg everywhere
else. Added a guard assertion that PaperRepo.__init__ has no venue param.
Full suite green (1853 passed); binance substring guard still 0 hits.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Step-1 finding: migration_builders.py and the migrate-forward-anchors CLI
(cli.py:427) already carry their own local strategy_id -> state_file basename
map (track_builders()) from an earlier Task-8 pass -- neither reads
entry["state_file"] off STRATEGY_REGISTRY, so removing the field doesn't
touch them. Only three direct consumers needed changes:
- registry.py: dropped "state_file" from all 11 entries + the header comment.
- assets.py: _run_paper_tracker's state_file param was unused at steady
state (its own docstring said so) -- dropped it + updated its 3 call
sites (sixtyforty_nav, multistrat_nav, vrp_nav).
- a handful of tests asserted on the registry's state_file key; updated to
drop those assertions, and added tests/unit/test_no_registry_state_file.py
as a permanent guard.
The JSON ForwardTracker + its .bak crash-recovery, and migrate-forward-anchors
itself, are untouched -- still the one-time JSON->DB seed / deployment tool.
- pytest-xdist added to dev deps; addopts='-n auto' runs the ~1940-test suite across all
cores (~6-9min -> ~1.5min); registered a 'slow' marker for a serial '-m "not slow"' loop.
- equity_backtest_runner liquidity fixture: trimmed the 3000-bar (8yr) LIVELONG/ILLIQUID
histories to 800 bars (still >> min_history=300, multi-year for peak-yearly ranking) —
the peak-yearly aggregation over 8yr was the cost: 50s/30s/19s tests -> ~5.8s each,
assertions unchanged.
- paper_sim perf guards: made them parallel-robust (the wall-clock <1.6s bound flaked under
-n auto CPU contention). full-history guard relaxed to <10s (it targets an O(D²) blowup =
orders of magnitude, not a constant factor); cheap-reapply made RELATIVE (< full/10, ratio
is contention-invariant). Both marked slow.
Full suite 1940 passed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Removes the cockpit's (web-layer) use of Binance: the /paper venue toggle + Binance paper book view, the /paper/replay venue toggle, and the sim's binance_combined book + Binance-vs-Bybit compare view (_sim_compare.html deleted). Bybit is now the cockpit's only paper-book venue. Backend Binance data/research modules (compare_measured_cost.py, data adapters) are untouched -- Task 7b deletes those.
Binance is deprecated as a broker; kept only as a DATA source (Task 7 removes
that). Deletes binance_ccxt.py, crypto_execution.py (no non-Binance consumer),
the crypto-rebalance/crypto-positions/crypto-flatten CLI commands, and the
execution-only BinanceSettings class + field. Adds a narrow guard test.
Alpaca's rebalancer CronJob has been suspended with 401'd keys since Task 4's
venue consolidation; IBKR paper is already the live forward-track venue for
the multi-strat book. This removes the dead code/config/manifests instead of
leaving them to rot, and relabels the multistrat/multistrat_exec registry
venues from the vestigial "alpaca-paper" to "ibkr-equity" (they execute on
IBKR paper, not Alpaca — exec_venue itself is still computed dynamically at
runtime).
- Delete src/fxhnt/adapters/broker/alpaca.py, AlpacaSettings + the `alpaca`
config field; move the one real setting it carried (min_order_notional,
a generic "skip sub-min-notional dust" floor for any future
fractional-capable broker) onto ExecutionSettings.
- Remove the `--broker` option and its alpaca branches from `execute` and
`execute-multistrat` (cli.py) — IBKR is now implicit; IbkrBroker was
already the module-level import, so the branch was pure dead code once
Alpaca is gone.
- Delete infra/k8s/jobs/fxhnt-alpaca-rebalancer.yaml and
infra/k8s/secrets/alpaca-credentials.yaml; delete the superseded
infra/k8s/services/multistrat-rebalancer.yaml (PVC + NetworkPolicy +
CronJob), replaced by fxhnt-ibkr-rebalancer. Drop the now-invalid
`--broker ibkr` argument from fxhnt-ibkr-rebalancer.yaml and
fxhnt-ucits-rebalancer.yaml (the flag no longer exists).
- Delete tests/integration/test_alpaca_broker.py (tested the deleted
adapter directly); add tests/unit/test_no_alpaca.py as a standing guard
(no "alpaca" substring anywhere in src/fxhnt + IbkrBroker still imports).
- Update test_registry_multistrat_exec.py + test_cockpit_ssot.py (venue
label) and test_cli_execute_helpers.py (drop the broker_name kwarg and
AlpacaBroker fake, use IbkrBroker instead) to match.
MANUAL cluster ops (not run by CI, record only):
kubectl delete cronjob/fxhnt-alpaca-rebalancer secret/alpaca-credentials -n foxhunt
kubectl delete cronjob/multistrat-rebalancer networkpolicy/multistrat-rebalancer pvc/multistrat-state -n foxhunt
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Neither is tradeable on Bybit: crypto_tstrend as a standalone Binance-perp forward
track was a -0.21 marginal-Sharpe drag/crash-amplifier, and stablecoin_rotation's
FDUSD/USDP pairs aren't listed on Bybit. Removes the two *_nav assets, their
registry entries, the orphaned migration_builders builders, and the
Binance-only StableRotationForward wrapper (stablecoin_runner.py's
StableReversionRunner stays -- it's still live via the Bybit stablecoin eval
paths). The crypto_tstrend SLEEVE inside the bybit_4edge deploy book
(_DEFAULT_BYBIT_SLEEVES, Bybit data) is untouched.
Updates dependent tests: deletes 5 whose subject (the retired asset/registry
entry/module) no longer exists, and swaps the retired sid for a still-registered
one (unlock/xsfunding/sixtyforty) in tests that only used crypto_tstrend as a
generic example sid.
Reuses build_bybit_sleeve_forward_nav (Task 1). unlock_nav drops the standalone
Binance UnlockShortForward(UnlockCalendarLive) path, runs the single unlock sleeve
on bybit_features through the deterministic engine, loading the unlock calendar via
load_unlock_events (the same loader the Bybit book uses). unlock joins _BYBIT_BOOK_SIDS
+ bybit_report_sid_kwargs (ref via _persist_bybit_book); registry venue=bybit-perp,
backtest.kind=report. _DEFAULT_BYBIT_SLEEVES/deploy book untouched. Regression lock
now requires the unlock ref too. Full suite 2057 green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
End-to-end test at the real production seam CockpitBacktestRefProvider(repo).reference(sid, days)
-> evaluate_reconciliation_gate: proves the reconciliation gate is permanently stuck in the
"no backtest reference to reconcile against" WAIT with no backtest_summary row, and un-sticks
(reconciles to PASS) once one is persisted (Tasks 2/6's fix for bug A).
Refactor _persist_track_backtest_ref (assets.py) to delegate to the unified backtest_ref dispatcher
instead of re-implementing the recompute-replay path inline, and wire xsfunding's missing sleeve-kind
ref through it (bug A's asset half) using the Binance-scoped feature store. Replace the manual
fxhnt-bybit-sleeve-ret Job with a weekly-scheduled fxhnt-backtest-refs CronJob running
`fxhnt backtest-refs --all` (bug B) so the report-kind refs (bybit_4edge/bybit_4edge_levered/positioning)
no longer depend on a human remembering to apply a one-off Job.
The per-day backfill did ~6-8 OPRA API round-trips/day (~26k over 2013->), all latency
-> ~45h. Rewrote it batched: fetch_option_chain_range (one range definition/month, deduped)
+ one range ohlcv fetch/month (band-bounded via a coarse median-strike forward, wide enough
for intra-month moves incl. COVID), then every trading day processed LOCALLY (parity forward
+ band + upsert). Extracted the forward/band logic into pure helpers (_forward_from_marks,
_band_contracts, _band_rows_from_marks) shared by the per-day path (_xsp_forward_estimate,
_freeze_xsp_slice — refactored, behaviour unchanged, tests green) and _backfill_month. CLI
iterates per-month. ~1h, ~$70-100 (bulk pulls a wider slice than the tight daily band).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
_xsp_forward_estimate and _freeze_xsp_slice each independently called fetch_option_chain
(definition) per day, doubling OPRA definition cost on every nightly vrp_nav run AND the
2013-> backfill (~$75 -> ~$37 definition). New _freeze_day fetches the chain ONCE, derives
the parity forward from it, and freezes the band from the SAME chain. Both helpers now accept
an optional pre-fetched chain (default-fetch preserved, so existing callers/tests unaffected).
All 3 callers (vrp_nav, plan_and_record_vrp, backfill-xsp-opra) use _freeze_day. Cuts the
2013-> backfill from ~$90 to ~$55.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Freezes puts (spread band) + near-ATM calls (for the strategy's parity forward) into
xsp_option_bars, then recomputes the vrp track off the frozen table. _xsp_forward_estimate
uses min(sorted(...)) for cross-process determinism. Wires vrp_nav into the nightly job,
Definitions.assets, and cockpit_forward deps. (Controller committed after self-verifying:
subagent completed the work + augmentations but returned the malformed monitor-wait status.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Whole-branch Important: dashboard showed the stale hardcoded ceiling while the engine sizes against
the resolved (paper $1M / live $35k) ceiling. Task-5 Minor: ruff I001 import-sort.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 'combined' crypto-momentum book was retired; its legacy forward-tracking path is
now dead (the fxhnt-forward-cronjob was long since replaced by the Dagster deterministic
engine, and is already gone from the cluster). Removed: the 'forward-track' CLI command +
_FUTURES_UNIVERSE, the combined_book / combined_book_factory / combined_book_source
modules, CombinedBookStrategy + CombinedBookForwardTracker (KEEPING the load-bearing
ForwardTracker base used by the recompute engine + bybit tracks), the combined_book_*
config fields, and 5 combined-book-dedicated test files. MomentumVrpStrategy kept (a
tested strategy primitive, not a combined_book module).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The broad-Binance-universe momentum book (venue binance+glbx) was a survivorship/
microcap mirage superseded by the isolated crypto_tstrend edge + bybit_4edge. Removed
the combined_forward_nav Dagster asset (+ its cockpit_forward/paper_book_snapshot deps),
the registry entry (DB registry row auto-prunes via _seed_registry), the dashboard
headline preference, and the dead migration builder. CombinedBook modules retained
(still used by the forward-track CLI). Test fixtures repointed combined -> crypto_tstrend.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds two test cases:
1. Test that latest_testnet_nav() returns the most-recent row by run_date
2. Test that the Executed reality detail block renders with exec/modeled return figures
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds the per-edge lifecycle blocks (funding health, allocation, executed
reality) to the strategy detail page, plus a minimal latest_testnet_nav()
reader on BybitTestnetRepo to surface mean exec slippage for the bybit
executed twin.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
bybit_testnet_reconcile now sizes plan_orders to the B2a/B3 envelope
(min(current_allocation, equity)) instead of raw equity (prior commit).
These pre-existing fixtures never seeded a bybit_4edge allocation row, so
the envelope collapsed to 0 and 6 of these tests started asserting on a
flattened (zero-order) book. Seed an allocation well above the fake
equity by default so envelope == equity, preserving the sizing these
tests were written against.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A blocked RebalancePlan places zero orders but the account keeps its prior positions, so
plan_and_record must not advance the multistrat_exec_pos basis or book a forward_record in that
case — otherwise every subsequent executed return is computed against a phantom target book that
was never placed. Also wires CockpitBacktestRefProvider to alias multistrat_exec -> multistrat so
the reconciliation gate actually reconciles the executed track against the modeled book's
basis-matched backtest reference, as the registry comment already (incorrectly) claimed. Plus a
doc comment clarifying positions_after is the intended target basis, and type-annotated locals in
compute_exec_return to keep the inferred return type float instead of Any.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
plan_and_record read positions_after from the account BEFORE this run's rebalance, so the stored
multistrat_exec_pos was the prior book — every future exec-return was computed one rebalance-cycle
stale (an equity-accounting off-by-one). Now the target book (weights·nlv/price) is computed inside
plan_and_record after the rebalance; the positions_after param + the CLI's stale account read are gone.
+ a test that the stored book is the target book.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
