# Funding-history ingest + replayable xsfunding (3rd edge) — Implementation Plan (Phase 2F-A4)

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development. TDD. `- [ ]` steps.

**Goal:** Ingest Binance perp funding-rate history into the warehouse, expose it, and add a faithful
replayable xsfunding sleeve so the paper book becomes a **3-edge** book.

**Architecture:** Funding stored as `feature='funding'` in the existing DuckDB feature store (no new table).
A never-raise httpx adapter fetches Binance `fundingRate` history; an ingest fn aggregates 8h→daily and bulk-
writes; a `crypto_funding_panel` accessor + `warehouse_funding_panel` assemble the `(close,qvol,funding)` panel
xsfunding/unlock need; a replayable xsfunding `PositionSource` runs the pure domain pipeline over it; the
backfill adds it. Read-only/additive to the A3f accounting + A3h overlay.

**Tech Stack:** Python 3.12, pytest, httpx, DuckDB. **Repo:** `~/Work/fxhnt`, branch `feat/funding-ingest-xsfunding`.
**Spec:** `docs/superpowers/specs/2026-06-22-funding-ingest-xsfunding-replay-design.md`.

**Verified context:**
- Feature store: `features(symbol, ts, feature, value)`, `ts=epoch_day·86400`; `write_features_bulk(items:
  list[(symbol, list[(ts, {feature:value})])])` idempotent per (symbol, ts-range); `crypto_close_panel(suffix)`
  reads `feature='close'`; `read_panel(symbols, feature)`.
- xsfunding domain (`domain/cross_sectional_funding.py`, pure): `Panel={sym:{day:(close,qvol,funding)}}`;
  `eligible_asof(panel, asof_day, *, min_qvol, min_history, vol_window=30)`; `funding_score(panel, eligible,
  asof_day, lookback_days=7)`; `construction_weights(scores, mode, *, quantile=0.2, borrowable=None)`.
- Live/backtest params: `min_qvol=1e6, min_history=30, lookback_days=7, quantile=0.2`. Replay mode =
  `"market_neutral"` (backtest-validated; `executable` needs a live borrowable set we can't reconstruct
  historically). qvol fallback `1e7` (≥ min_qvol → liquidity gate effectively off in replay, as with unlock).
- `PositionSource` = `Protocol{ current_weights(as_of: str) -> dict[str,float] }`. Backfill sleeves built in
  `cli.py:_paper_backfill_sleeves(s, panel)` (xsfunding currently excluded ~line 903).
- httpx is in the cockpit image. Binance public: `https://fapi.binance.com/fapi/v1/fundingRate`.

---

## Task 1: `BinanceFundingHistory` adapter

**Files:** Create `src/fxhnt/adapters/exchange/binance_funding_history.py`; Test `tests/unit/test_binance_funding_history.py`

- [ ] **Step 1: Failing test** (mock httpx — no network):
```python
# tests/unit/test_binance_funding_history.py
from fxhnt.adapters.exchange.binance_funding_history import BinanceFundingHistory

class FakeResp:
    def __init__(self, data): self._d = data
    def raise_for_status(self): pass
    def json(self): return self._d

class FakeClient:
    def __init__(self, pages): self.pages = pages; self.calls = []
    def get(self, url, params=None, timeout=None):
        self.calls.append(params); 
        return FakeResp(self.pages.pop(0) if self.pages else [])

def test_fetch_paginates_and_parses():
    p1 = [{"symbol": "BTCUSDT", "fundingTime": 1000, "fundingRate": "0.0001"}] * 1000
    p2 = [{"symbol": "BTCUSDT", "fundingTime": 2000, "fundingRate": "-0.0002"}]
    fc = FakeClient([p1, p2, []])
    out = BinanceFundingHistory(client=fc).fetch("BTCUSDT", 0, 9999)
    assert len(out) == 1001 and out[-1] == (2000, -0.0002)   # (fundingTime ms, rate float)

def test_fetch_error_returns_empty_not_raise():
    class BoomClient:
        def get(self, *a, **k): raise RuntimeError("429")
    assert BinanceFundingHistory(client=BoomClient()).fetch("X", 0, 1) == []
```
- [ ] **Step 2:** Run → FAIL.
- [ ] **Step 3:** Implement. `BinanceFundingHistory(client=None, base="https://fapi.binance.com")` (default
  `client=httpx.Client()`). `fetch(symbol, start_ms, end_ms)`: loop `GET /fapi/v1/fundingRate` with
  `{symbol, startTime, endTime, limit:1000}`; parse `(int(fundingTime), float(fundingRate))`; advance
  `startTime = last fundingTime + 1`; stop when a page has `<1000` rows or is empty or exceeds `end_ms`;
  wrap the whole loop in `try/except Exception → return what's collected (or [])`. Cap iterations (e.g. ≤200)
  as a runaway guard.
- [ ] **Step 4:** Run → PASS. **Step 5:** Commit `feat(funding): BinanceFundingHistory adapter (paginated, never-raise)`.

---

## Task 2: `crypto_funding_panel` accessor

**Files:** Modify `src/fxhnt/adapters/warehouse/duckdb_feature_store.py`; Test `tests/integration/test_crypto_funding_panel.py`

- [ ] **Step 1: Failing test**
```python
# tests/integration/test_crypto_funding_panel.py
from fxhnt.adapters.warehouse.duckdb_feature_store import DuckDbFeatureStore

def test_crypto_funding_panel_reads_funding_feature(tmp_path):
    st = DuckDbFeatureStore(str(tmp_path / "w.duckdb"))
    st.write_features("BTCUSDT", [(86400, {"funding": 0.0003}), (172800, {"funding": -0.0001})])
    st.write_features("ETHUSDT", [(86400, {"funding": 0.0005})])
    st.write_features("BTCUSDT", [(86400, {"close": 100.0})])     # close must NOT leak into funding panel
    p = st.crypto_funding_panel()
    assert p["BTCUSDT"] == {1: 0.0003, 2: -0.0001}                # ts//86400 = epoch_day
    assert p["ETHUSDT"] == {1: 0.0005}
```
- [ ] **Step 2:** Run → FAIL.
- [ ] **Step 3:** Add `crypto_funding_panel(*, suffix='USDT')` mirroring `crypto_close_panel` but
  `feature='funding'`.
- [ ] **Step 4:** PASS. **Step 5:** Commit `feat(funding): crypto_funding_panel accessor`.

---

## Task 3: `ingest_funding` + CLI + Dagster asset

**Files:** Create `src/fxhnt/application/funding_ingest.py`; Modify `src/fxhnt/cli.py` +
`src/fxhnt/adapters/orchestration/assets.py` (+ definitions); Test `tests/integration/test_funding_ingest.py`

- [ ] **Step 1: Failing test** (fake fetcher + real in-memory store):
```python
# tests/integration/test_funding_ingest.py
from fxhnt.adapters.warehouse.duckdb_feature_store import DuckDbFeatureStore
from fxhnt.application.funding_ingest import ingest_funding

class FakeFetcher:
    def fetch(self, sym, s, e):
        # two 8h stamps on epoch_day 1 (86400..) + one on day 2 -> daily SUM
        return [(86400_000, 0.0001), (86400_000 + 8*3600_000, 0.0002), (172800_000, -0.0003)]

def test_ingest_aggregates_8h_to_daily_sum_idempotent(tmp_path):
    st = DuckDbFeatureStore(str(tmp_path / "w.duckdb"))
    n = ingest_funding(st, FakeFetcher(), ["BTCUSDT"], start_ms=0, end_ms=10**13)
    assert n == 1
    p = st.crypto_funding_panel()
    assert abs(p["BTCUSDT"][1] - 0.0003) < 1e-12      # day1 = 0.0001+0.0002 (summed)
    assert abs(p["BTCUSDT"][2] - (-0.0003)) < 1e-12
    ingest_funding(st, FakeFetcher(), ["BTCUSDT"], start_ms=0, end_ms=10**13)   # re-run
    assert st.crypto_funding_panel()["BTCUSDT"] == p["BTCUSDT"]   # idempotent
```
- [ ] **Step 2:** Run → FAIL.
- [ ] **Step 3:** `ingest_funding(store, fetcher, symbols, *, start_ms, end_ms) -> int`: per symbol
  `fetcher.fetch(sym, start_ms, end_ms)`; aggregate by `epoch_day = ms // 86_400_000` summing rates;
  build `rows=[(day*86400, {"funding": daily})...]`; collect `(sym, rows)`; `store.write_features_bulk(items)`;
  return count of symbols with ≥1 row. Skip empty fetches.
- [ ] **Step 4:** PASS.
- [ ] **Step 5:** Wire a CLI `crypto-funding-ingest [--days 400]` in `cli.py`: open the live warehouse store
  (same path the paper-backfill uses via `get_settings()`), `symbols = list(store.crypto_close_panel())`,
  `BinanceFundingHistory()`, `start_ms = (today-days)·86_400_000`, call `ingest_funding`, echo count. Add a
  Dagster asset `crypto_funding` (mirror an existing crypto ingest asset; daily) in `assets.py` + register in
  `definitions.py`. Keep it a one-shot single-writer (note in the docstring).
- [ ] **Step 6:** `uv run pytest tests/integration/test_funding_ingest.py -q` + a CLI smoke test if one exists.
  Commit `feat(funding): ingest_funding (8h->daily) + CLI + Dagster asset`.

---

## Task 4: `warehouse_funding_panel` + replayable xsfunding sleeve

**Files:** Create `src/fxhnt/application/funding_panel.py` + `src/fxhnt/application/xsfunding_replay.py`;
Test `tests/integration/test_xsfunding_replay.py`

- [ ] **Step 1: Failing test**
```python
# tests/integration/test_xsfunding_replay.py
from fxhnt.application.funding_panel import warehouse_funding_panel
from fxhnt.application.xsfunding_replay import XsFundingReplay

def test_warehouse_funding_panel_assembles_tuple():
    closes = {"A": {1: 10.0, 2: 11.0}, "B": {1: 5.0, 2: 5.0}}
    funding = {"A": {1: 0.01, 2: 0.01}, "B": {1: -0.01, 2: -0.01}}
    panel = warehouse_funding_panel(closes, funding, qvol_fallback=1e7)
    assert panel["A"][1] == (10.0, 1e7, 0.01)

def test_xsfunding_replay_longs_low_funding_shorts_high():
    # 40 days history so eligibility/lookback clear; A high funding, B low funding
    closes = {s: {d: 10.0 for d in range(1, 41)} for s in ("A", "B")}
    funding = {"A": {d: 0.02 for d in range(1, 41)}, "B": {d: -0.02 for d in range(1, 41)}}
    panel = warehouse_funding_panel(closes, funding, qvol_fallback=1e7)
    w = XsFundingReplay(panel).current_weights("1970-02-10")   # epoch_day 40
    assert w.get("A", 0) < 0 and w.get("B", 0) > 0             # short high-funding A, long low-funding B
    assert abs(sum(w.values())) < 1e-9                          # market-neutral (dollar-neutral)
```
- [ ] **Step 2:** Run → FAIL.
- [ ] **Step 3:**
  - `funding_panel.py`: `warehouse_funding_panel(close_panel, funding_panel, *, qvol_fallback=1e7) ->
    {sym:{day:(close, qvol_fallback, funding_panel.get(sym,{}).get(day, 0.0))}}` over the close panel's
    (sym,day) keys. (Funding 0.0 only where genuinely missing.)
  - `xsfunding_replay.py`: `class XsFundingReplay` (implements `PositionSource`):
    `__init__(self, panel, *, min_qvol=1e6, min_history=30, lookback_days=7, quantile=0.2)`;
    `current_weights(as_of)`: `day = (date.fromisoformat(as_of) - date(1970,1,1)).days`;
    `elig = eligible_asof(panel, day, min_qvol=…, min_history=…)`; if not elig → `{}`;
    `scores = funding_score(panel, elig, day, lookback_days=…)`;
    `return construction_weights(scores, "market_neutral", quantile=…)`.
- [ ] **Step 4:** PASS. **Step 5:** Commit `feat(funding): warehouse_funding_panel + XsFundingReplay sleeve`.

---

## Task 5: wire xsfunding into the backfill (3-edge) + fix unlock funding

**Files:** Modify `src/fxhnt/cli.py` (`_paper_backfill_sleeves`); Test extend `tests/integration/test_paper_backfill*.py`

- [ ] **Step 1: Failing/asserting test** — build a fake warehouse (closes + funding) with a clear
  high/low-funding split; assert `_paper_backfill_sleeves` returns a sleeve set INCLUDING `"xsfunding"`, and
  that its `current_weights(<a date with history>)` is non-empty + market-neutral.
- [ ] **Step 2:** Run → FAIL.
- [ ] **Step 3:** In `_paper_backfill_sleeves`: build `funding_panel = store.crypto_funding_panel()`,
  `fp = warehouse_funding_panel(close_panel, funding_panel)`, add `sleeves["xsfunding"] =
  XsFundingReplay(fp)`. ALSO switch the unlock sleeve's panel to use `fp` (real funding) instead of the
  `funding=0` fallback (bonus from the spec). Update the "xsfunding excluded" comment → now included (3 edges).
- [ ] **Step 4:** PASS. **Step 5:** Commit `feat(funding): add xsfunding (3rd edge) to the paper backfill`.

---

## Task 6: Full suite + finish + deploy + ingest + re-backfill + 3-edge verify

- [ ] **Step 1:** `uv run pytest -q` → green. **Step 2:** no `*.dbn` staged.
- [ ] **Step 3:** **superpowers:finishing-a-development-branch** — merge → master, push (auto-deploy). Confirm
  workflow Succeeded + pods rolled.
- [ ] **Step 4:** Ingest funding in-cluster (one-shot, single-writer — ensure no other warehouse write runs):
  `kubectl exec deploy/dagster -c daemon -n foxhunt -- fxhnt crypto-funding-ingest --days 400`. Confirm it
  reports N symbols written; spot-check `crypto_funding_panel` is populated.
- [ ] **Step 5:** Re-backfill (now 3-edge):
  `kubectl exec deploy/dagster -c daemon -n foxhunt -- fxhnt paper-backfill --days 365`. Confirm sleeves now
  list `xsfunding` and it has positions.
- [ ] **Step 6: VERIFY (the real test)** — does the 3rd uncorrelated edge de-concentrate the regime risk?
  - per-sleeve standalone + in-book contribution (xsfunding should appear, ~uncorrelated to the other two);
  - the deployed nav curve (return / Sharpe / maxDD) + the **OOS H1/H2 split** — is H1 less weak and the
    blended Sharpe higher/steadier than the 2-edge book (which was H1 Sh ~0.9 / H2 ~5.6)?
  - the cost-drag rerun on the 3-edge turnover.
  - `/paper` + `/paper/replay` render with 3 sleeves. Report all numbers honestly (incl. whether xsfunding
    actually helped — if it didn't de-concentrate, say so).

---

## Self-review notes
- Funding = a new `feature='funding'`; no schema change. Ingest idempotent (per symbol,ts-range).
- Replay xsfunding uses the SAME pure domain pipeline + live params (market_neutral mode); causal.
- `warehouse_funding_panel` serves xsfunding AND fixes unlock's funding=0 fallback.
- Read-only/additive: A3f accounting, A3h overlay, shadow signal, other 2 sleeves untouched; live xsfunding
  path untouched (replay is additive).
- External API: per-symbol never-raise + idempotent; one-shot single-writer ingest.
- NEVER `git add -A`; verify no `*.dbn` staged before each commit.

## Acceptance criteria (from spec)
- BinanceFundingHistory paginates + never-raises; tested. ✅ T1
- crypto_funding_panel reads funding; tested. ✅ T2
- ingest_funding 8h→daily sum + idempotent + CLI + asset; tested. ✅ T3
- warehouse_funding_panel + XsFundingReplay (market-neutral, causal); tested. ✅ T4
- backfill includes xsfunding (3 edges) + unlock funding fixed; tested. ✅ T5
- in-cluster ingest → 3-edge re-backfill → OOS/cost verify (did the 3rd edge de-concentrate?); rendered. ✅ T6
- read-only/additive; no `*.dbn`. ✅ T6
