# Funding-history ingest + replayable xsfunding (3rd edge) — Design (Phase 2F-A4)

**Date:** 2026-06-22
**Status:** Design (approved)
**Repo:** fxhnt
**Follows:** A3h. Adds the 3rd validated edge (cross-sectional funding carry) to the paper replay by ingesting
historical funding rates into the warehouse.

## Motivation

The paper replay runs only 2 of the 3 validated edges; **xsfunding (funding-dispersion carry) is excluded —
"live-only"** — because its `current_weights` reads a live funding snapshot and historical funding rates are
**not in the warehouse** (unlock falls back to `funding=0`; the funding backtest reads the absent
`crypto_pit_dir`). The A3h OOS split showed the 2-edge book is **regime-concentrated** (H1 Sharpe ~0.9, H2
~5.6); a 3rd uncorrelated edge is the principled fix (diversification, not more tuning). This ingests funding
history → warehouse → makes xsfunding replayable → 3-edge book.

## Key facts (grounded in code)

- Warehouse = generic feature store `features(symbol, ts, feature, value)`; `ts` = epoch-SECONDS, daily rows
  use `ts = epoch_day·86400`; folded via `ts//86400`. `write_features_bulk` is idempotent per (symbol,
  ts-range). `crypto_close_panel` reads `feature='close'`; `read_panel(symbols, feature)` reads any feature.
  → **Funding needs NO new table — just `feature='funding'` rows + a mirror accessor.**
- xsfunding signal (`domain/cross_sectional_funding.py`, pure): Panel = `{sym:{day:(close,qvol,funding)}}`;
  `eligible_asof` (qvol + history floor) → `funding_score` (robust_z of trailing-`lookback_days` mean funding)
  → `construction_weights` (market-neutral long-low/short-high). Same panel shape unlock uses.
- Binance perp funding history: public `GET /fapi/v1/fundingRate?symbol=&startTime=&endTime=&limit=1000`
  (8h cadence, `{symbol, fundingTime(ms), fundingRate}`). httpx is in the cockpit image (`web` extra).

## Decisions

- **Storage = the existing feature store, `feature='funding'`** (daily, `ts=epoch_day·86400`,
  value = that UTC day's summed funding = the day's total carry). No schema change.
- **Ingest Binance perp `fundingRate` history**, 8h → **aggregated to daily (sum)**, for the warehouse's
  existing USDT-perp universe (`crypto_close_panel` symbols), via a never-raise-per-symbol httpx adapter +
  a CLI command + a Dagster asset (daily refresh keeps the live path consistent too).
- **Accessor** `crypto_funding_panel(suffix='USDT') -> {sym:{day:funding}}` mirroring `crypto_close_panel`.
- **`warehouse_funding_panel(close_panel, funding_panel, *, qvol_fallback)` -> `{sym:{day:(close,qvol,
  funding)}}`** (application): real funding now; qvol fallback as today. Serves BOTH xsfunding AND fixes
  unlock's `funding=0` fallback (bonus).
- **Replayable xsfunding sleeve**: a `PositionSource` whose `current_weights(as_of)` runs the domain pipeline
  (`eligible_asof`→`funding_score`→`construction_weights`) over the warehouse panel, using the SAME params as
  the live xsfunding (mode/quantile/lookback/min_qvol/min_history from the live config) — faithful replay,
  just sourced from history instead of the live snapshot.
- **Add xsfunding to the backfill sleeve set** → 3-edge replay. The overlay (A3h) + accounting (A3f) are
  unchanged; xsfunding is just another sleeve feeding the same `paper_risk_overlay`.
- **DuckDB single-writer:** ingestion is a one-shot (CLI/asset) that must not run concurrently with other
  warehouse writers (documented; mirror the existing ingest ops' single-writer discipline).

## Components

1. **`adapters/exchange/binance_funding_history.py`** — `BinanceFundingHistory.fetch(symbol, start_ms,
   end_ms) -> list[tuple[int, float]]` (fundingTime ms, rate); paginated (limit 1000, walk startTime);
   per-symbol try/except → `[]` on error (never abort the batch); last-good. httpx, public base
   `https://fapi.binance.com`.
2. **`DuckDbFeatureStore.crypto_funding_panel(*, suffix='USDT')`** — mirror of `crypto_close_panel` for
   `feature='funding'`.
3. **`application/funding_ingest.py`** — `ingest_funding(store, fetcher, symbols, start, end) -> int`:
   per symbol fetch → aggregate 8h→daily (sum per UTC epoch_day) → `FeatureRow(ts=day·86400,
   {'funding': daily})` → `write_features_bulk`. Returns symbols written. Per-symbol skip-on-empty.
4. **CLI `crypto-funding-ingest [--days N]`** + **Dagster asset `crypto_funding`** (daily) — wire (3) to the
   live warehouse store + the `crypto_close_panel` symbol universe + `BinanceFundingHistory`.
5. **`application/funding_panel.py`** `warehouse_funding_panel(close_panel, funding_panel, *,
   qvol_fallback=1e7) -> Panel` — assemble `(close, qvol, funding)`; funding 0.0 only where a symbol/day
   genuinely lacks an ingested rate (logged). Reused by the xsfunding sleeve AND unlock (replaces its
   `funding=0` fallback).
6. **Replayable xsfunding `PositionSource`** (`application/xsfunding_replay.py`): `current_weights(as_of)`
   = `construction_weights(funding_score(panel, eligible_asof(panel, day, ...), day, lookback), mode,
   quantile, ...)` with live params. `{}` when no eligible names that day.
7. **`cli.py _paper_backfill_sleeves`**: add the xsfunding replay sleeve (built from
   `warehouse_funding_panel`) to the sleeve set; drop the "xsfunding excluded" note → now 3 edges.
8. **Verify**: re-ingest + re-backfill in-cluster; run the OOS-half + cost analysis on the 3-edge book; check
   xsfunding contributes and the H1/H2 regime-concentration improves (more stable H1, higher blended Sharpe,
   lower DD) — the actual test of whether the 3rd edge helped.

## Risks / safety

- **External API:** rate-limited, paginated, per-symbol try/except → `[]` (a bad/missing symbol never aborts
  the run); idempotent writes (re-run replaces per (symbol, ts-range)). The `fundingRate` endpoint is light.
- **Read-only/additive to the book:** new feature rows + a new replayable sleeve; A3f accounting, A3h overlay,
  shadow signal, the other 2 sleeves all unchanged. xsfunding's LIVE path is untouched (replay is additive).
- **Faithful replay:** the replay sleeve uses the SAME pure domain pipeline + live params as live xsfunding;
  only the data source differs (warehouse history vs live snapshot). Causal (`eligible_asof`/`funding_score`
  use only `≤ asof_day`).
- **DuckDB single-writer:** ingest one-shot, not concurrent with other warehouse writers.
- **Funding aggregation:** 8h→daily SUM = the day's total carry (matches a daily-rebalanced book holding
  through the 3 funding stamps). Documented.
- **Honest caveat:** still gross/frictionless for the *new* sleeve too; the cost model (already estimated)
  applies to the 3-edge turnover at verify time.

## ADDENDUM (post-T4 finding): xsfunding is a CARRY edge — needs a carry return-source

**Finding (verified in code):** xsfunding's P&L is **delta-neutral carry**, not directional price. The domain
`book_return_xs = Σ wᵢ·fundingᵢ`; the live sleeve books `w·((spot_ret−perp_ret)+funding)`. The paper book's
per-sleeve return is price-MTM (`Σ qty·Δclose`) on a single perp-close series — which would measure the
*directional* price drift of funding-sorted perps (~3%/day noise) and completely miss the ~0.03%/day carry.
So xsfunding **cannot** be added via the price-MTM path; it needs a **carry return-source**. (Approved by user:
"extend the book.")

**Design:**
- New pure **`sleeve_daily_carry(prev_positions, funding_by_symbol) -> float | None`** (`paper_book.py`):
  `Σ (qty·entry_price)·funding / Σ|qty·entry_price|`. `qty·entry_price` recovers the signed unit-weight from
  the shadow book; normalized by gross. Mirrors `sleeve_daily_return` (funding instead of Δclose). `None` on
  zero notional. (= the domain `book_return_xs` per unit gross.)
- **Per-sleeve return mode:** `backfill_paper_book` + `PaperSnapshotService` compute each sleeve's daily
  return by `getattr(sleeve, "return_mode", "price")`: `"price"` → `sleeve_daily_return(prev_shadow, closes)`
  (unchanged); `"carry"` → `sleeve_daily_carry(prev_shadow, funding_today)`. `XsFundingReplay.return_mode =
  "carry"`. Both paths need the per-(day,symbol) funding (backfill: from `crypto_funding_panel`; snapshot:
  latest funding per symbol). Everything downstream consumes `paper_sleeve_ret` unchanged → live==replay.
- **Consequence (intended):** xsfunding's low carry-vol → large inverse-vol weight (risk-parity), then
  vol-target levers the blend — the correct risk-parity 3-edge behavior; the verify step tests whether it
  de-concentrates the H1/H2 regime risk. The directional sleeves (unlock, ts-trend) stay price-MTM unchanged.
- Revised tasks: **5a** `sleeve_daily_carry`; **5b** per-sleeve return-mode wiring (backfill+snapshot) + add
  xsfunding (carry) to the backfill sleeve set; **6** deploy/ingest/verify (does the carry edge help?).

## Out of scope

Switching live xsfunding off its live snapshot (replay is additive; live unchanged); spot-perp basis modeling
(xsfunding replay uses funding-only, as the funding-only fallback the backtest documents); walk-forward
re-selection of overlay params on the 3-edge book (separate); per-sleeve cost modeling.

## Acceptance criteria

- `BinanceFundingHistory.fetch`: paginates, parses `{fundingTime, fundingRate}`, per-symbol error → `[]`;
  unit-tested with a mocked httpx (multi-page + error).
- `crypto_funding_panel` reads `feature='funding'` → `{sym:{day:funding}}`; tested.
- `ingest_funding`: 8h→daily sum, idempotent write; tested with a fake fetcher + in-memory store (re-run =
  same rows).
- `warehouse_funding_panel`: assembles `(close,qvol,funding)` with real funding; tested.
- replayable xsfunding `current_weights(as_of)`: matches the domain pipeline on a fake panel (long-low /
  short-high, market-neutral); causal; `{}` on no-eligible; tested.
- backfill includes xsfunding (3 sleeves); CLI ingest + asset wired; full suite green.
- in-cluster: ingest funding → re-backfill 3-edge → xsfunding has positions/returns; OOS-half + cost rerun
  reported (does the 3rd edge de-concentrate H1/H2?); `/paper` + `/paper/replay` render; verified.
- read-only/additive (accounting/overlay/other sleeves); no `*.dbn` staged.
