# Paper book: ISV-adaptive sleeve weighting — Design (Phase 2F-A3c)

**Date:** 2026-06-21
**Status:** Design (approved)
**Repo:** fxhnt
**Follows:** A3 replay + A3b equal-weight. Replaces naive equal-weight with ISV-adaptive risk weighting.

## Motivation

A3b sizes the paper book by **equal-weight** (1/N active sleeves). Equal-weight ignores risk: a high-vol
sleeve (ts-trend) gets the same book share as a low-vol one (stablecoin), so the blended curve is dominated
by whichever sleeve is most volatile. We want the replayable sleeves sized by **inverse-vol** (risk-parity),
and — per the user's call — with the weighting **meta-parameters themselves state-derived (ISV-adaptive)**
rather than hardcoded, applying the foxhunt ISV discipline (first-observation bootstrap, hard floors,
state-derived bounds, max-with-floor not blend). We want to *see the effect* in both live `/paper` and
`/paper/replay`.

## Key finding (drove the design)

The obvious return source — each sleeve's `forward_nav` series — **does not exist** for the replayable
sleeves. `forward_nav` (the live gate record) only has rows for `combined` (15d), `sixtyforty` (3d),
`xsfunding` (1d). `crypto_tstrend` / `stablecoin_rotation` / `unlock` have **zero** rows. So the return
series must come from the paper book's **own simulated per-sleeve daily returns**, computed causally as the
backfill walks dates chronologically — not from `forward_nav`.

## Decisions (from brainstorming)

- **Returns are the sleeves' own simulated daily P&L.** On date *d*, sleeve *s*'s return =
  Σ qty·(close@*d* − close@*d−1*) over *s*'s positions carried from *d−1*, ÷ *s*'s notional that day. The
  backfill walks dates chronologically and accumulates `returns_by_sleeve[s]` causally (each date weights on
  returns strictly *before* it → no look-ahead).
- **ISV-adaptive meta-params** (user chose "full ISV-adaptive" over mirroring the live `book_allocator`). The
  base inverse-vol math mirrors `book_allocator._inverse_vol_weights` (raw `1/std` normalize), but the cap is
  replaced by a two-sided water-fill so floor+cap resolve together; the three meta-params are state-derived
  with bootstrap + floors (detailed below).
- **Persist per-sleeve daily returns** in a new `paper_sleeve_ret(run_date, sleeve, ret)` table, written by
  BOTH backfill and nightly snapshot, so the snapshot (which sees only one new date/run) builds the identical
  trailing window the backfill does → live `/paper` and `/paper/replay` use the **same** ISV-adaptive logic.
  ("do for both — we see the effect.")
- **Cost accepted:** the adaptive vol path diverges from the live edge-book `book_allocator` (which uses
  fixed-window `np.std`). Live and replay stay consistent with *each other* (same helper).

## ISV-adaptive weighting (the three state-derived meta-params)

Given `returns_by_sleeve` (`{sleeve: {epoch_day: ret}}`, causal) and the day's `active_sleeves`. (Review
fixes #2/#4/#6 from the critical-review pass are folded in below.)

**A. Adaptive lookback `L_t`** (replaces fixed `VOL_LOOKBACK=60`) — **#2 disjoint-window signal**
- Regime signal compares **two DISJOINT windows** (not nested) so it actually moves under stationarity:
  pool the active sleeves' most-recent returns into `recent = last L_min` and `prior = the L_min before
  those`. `ρ = std(recent) / std(prior)` (if `prior` has < `L_min` points or `std(prior) ≤ 1e-12` →
  `ρ = 1.0`, i.e. hold). `ρ>1` = vol rising vs the preceding block → shorten the window to react.
- `L_t = round(clip(L_max / max(1.0, ρ), L_min, L_max))`. **Floor `L_min=10`**, **anchor `L_max=60`**
  (= live book). Until `≥ 2·L_min` obs exist (can't form disjoint windows), `ρ=1` → `L_t=L_max` → use all
  available (bootstrap). Vol-of-vol is the more general signal but needs more history than this book has;
  the disjoint recent-vs-prior ratio is the robust minimum that still deviates from 1 under a regime shift.

**B. Adaptive cold-start + floor** (replaces `MIN_OBS=20` cliff — ISV first-obs bootstrap + weight floor)
- **First-observation bootstrap:** a sleeve with `≥2` returns contributes an inverse-vol estimate; a sleeve
  with `<2` returns can't form a std → see #6 (mixed-history) below. No cliff, no NaN.
- **Weight floor is a bounded water-fill, NOT naive floor-then-renormalize (#4).** Naive renormalize can
  push a just-floored weight back below the floor, so we use the **same iterative clamp-and-renormalize loop**
  `_inverse_vol_weights` uses for the cap, but two-sided: each pass clamps every weight into
  `[w_floor, cap_t]`, redistributes the freed/needed mass across the unclamped sleeves, repeats until stable
  (≤ `N` passes). Feasible because `w_floor·N = φ = 0.5 ≤ 1 ≤ cap_t·N` (cap_t ≥ 1/N). `w_floor = φ/N_active`
  (`φ=0.5`, floor below equal-weight so the inverse-vol tilt survives).

**C. Adaptive concentration cap `cap_t`** (replaces `MAX_SLEEVE_WEIGHT=0.5`)
- Derived from the **effective number of bets** of the raw (pre-cap) inverse-vol weights:
  `N_eff = (Σw)² / Σw²` (participation ratio — a pure state measure of concentration).
- `cap_t = clip(κ / N_eff, 1.0/N_active, 1.0)`, `κ=2`. **Floored at `1/N_active`**, ceiled `1.0`. Fed as the
  upper bound of the two-sided water-fill in B (so cap + floor are resolved *together*, defined precedence:
  one joint clamp range `[w_floor, cap_t]`, not cap-then-floor). NOTE: at `N_active ≤ 3` the cap rarely binds
  (documented in the critical review #3); it's retained for when the book grows.

**#6 — mixed-history rule (some active sleeves have returns, some don't):** every active sleeve is in the
output. Sleeves with `≥2` returns are weighted by inverse-vol over the last `L_t` obs; active sleeves with
`<2` returns get **exactly `w_floor`** (max-uncertainty → minimum conviction, the ISV max-with-floor form),
and the remaining mass is water-filled across the has-history sleeves by inverse-vol. If **no** active sleeve
has `≥2` returns → equal-weight (1/N) over active.

**Structural constants** (shape anchors, each with a floor — not fit to data): `L_min=10`, `L_max=60` (live
anchor), `φ=0.5`, `κ=2`. The per-date `L_t`, `cap_t`, and weights all move with observed state.

**Order of operations** (per date): restrict to active → if no active sleeve has `≥2` returns, equal-weight
(1/N) → else compute `ρ→L_t` (disjoint windows), per has-history sleeve take last `L_t` obs and its inverse
std; compute `N_eff→cap_t` from the raw inverse-vol weights; assign `<2`-history active sleeves `w_floor`;
run the two-sided water-fill (clamp to `[w_floor, cap_t]` + renormalize, iterate) → final weights sum 1.

## Components

1. **Shared helper** `isv_adaptive_weights(returns_by_sleeve, active_sleeves, *, l_min=10, l_max=60,
   phi=0.5, kappa=2.0) -> {sleeve: float}` in `application/paper_book.py`. Pure function (no I/O). Computes
   the **raw inverse-vol weights** (`∝ 1/std` over the last `L_t` obs, normalized to sum 1 — same formula as
   `_inverse_vol_weights`' first stage, used to also derive `N_eff→cap_t`), then applies the **two-sided
   water-fill** for cap+floor (B/C) — it does NOT call `_inverse_vol_weights` with an internal cap (that
   would double-cap); it owns the single joint clamp. Adds A (`L_t`). Returns `{}` for no active sleeves;
   equal-weight (1/N) when no active sleeve has `≥2` returns.
1b. **Shared per-sleeve return fn (#5 — one definition, no drift)** `sleeve_daily_return(prev_positions,
   close_by_symbol) -> float | None` in `application/paper_book.py` (pure). The prior-date close is carried
   by `Position.entry_price` (positions are re-derived each date at that date's close), so no separate
   prev_close arg is needed. Returns `Σ qty·(close − entry_price) / Σ|qty·entry_price|` over the sleeve's
   prior-date positions, or `None` when there is no prior position / notional is 0 (unknown symbol →
   `close.get(sym, entry_price)` = 0 contribution). BOTH `backfill_paper_book` and `PaperSnapshotService`
   call this — they must NOT each re-derive the formula. Causality: a date's return uses that date's close,
   so it is computed AFTER sizing the date and used only to size the NEXT date.
2. **`paper_sleeve_ret` table** (`adapters/persistence/cockpit_models.py`): `(run_date, sleeve)` PK,
   `ret float`, `src_updated_at`. Timescale-friendly like `paper_nav`.
3. **PaperRepo** (`adapters/persistence/paper_repo.py`): `upsert_sleeve_ret(run_date, sleeve, ret)`
   (idempotent per (run_date, sleeve)) and `sleeve_returns(before=run_date) -> {sleeve: {epoch_day: ret}}`
   (trailing series strictly before a date, for the snapshot path).
4. **`backfill_paper_book`** (`application/paper_backfill.py`): walking dates in order, after pricing each
   date compute each active sleeve's daily return via `sleeve_daily_return(prior positions, prior closes,
   today closes)`, append (skip `None`) to in-memory `returns_by_sleeve`, and use
   `isv_adaptive_weights(returns_by_sleeve, active)` for that date's sizing (instead of `equal_weights`).
   Persist each non-`None` return via `upsert_sleeve_ret`. Day-0 of a sleeve has no prior position →
   `None` → no return (equal-weight cold-start covers it).
5. **`PaperSnapshotService`** (`application/paper_snapshot.py`): load the trailing series with
   `paper_repo.sleeve_returns(before=run_date)`, compute the run_date's own per-sleeve return via the SAME
   `sleeve_daily_return` (prior persisted positions + today's closes), append, then
   `isv_adaptive_weights(...)`; persist via `upsert_sleeve_ret`. Identical logic path to the backfill (#5).

## Data flow

Backfill (chronological) → per date: active sleeves' `current_weights` → prior-date per-sleeve return →
append to causal window → `isv_adaptive_weights` (ρ→L_t, N_eff→cap_t, first-obs bootstrap, w_floor) →
`persist_paper_book` + `upsert_sleeve_ret`. Nightly snapshot → read trailing returns from `paper_sleeve_ret`
+ compute today's → `isv_adaptive_weights` → persist. `/paper` and `/paper/replay` both reflect the weights.

## Risks / safety

- **Read-only/additive.** No change to sleeve `run()`/`advance()`, `combine_book`, or the gate record. Only
  the paper sizing + the new returns table/writes.
- **Look-ahead:** each date weights on returns strictly *before* it (causal). Bootstrap/short-history uses
  equal-weight; inverse-vol engages once a sleeve has a usable vol estimate.
- **Degenerate vol:** `_inverse_vol_weights` already returns equal-weight when total inverse-vol ≤ 0; the
  two-sided water-fill (clamp `[w_floor, cap_t]` + iterate) guarantees no sleeve is starved AND the floor
  actually holds post-renormalize (#4); `cap_t` floored at `1/N_active` so the range is always feasible.
- **Overfitting on the tiny 3-sleeve sample** (the caution raised when choosing full-ISV): contained by the
  hard floors (`L_min`, `w_floor`, `cap_t≥1/N`) and the equal-weight bootstrap — degrades to equal-weight,
  never to a degenerate/flat book.
- **Consistency:** the same `isv_adaptive_weights` helper + the same `paper_sleeve_ret` history feed both the
  backfill and the nightly snapshot.
- **Idempotent:** `upsert_sleeve_ret` keyed by (run_date, sleeve); re-run backfill replaces, no dup.

## Out of scope

`_marginal_prune` correlation pruning (the paper book uses the raw inverse-vol path); fractional-Kelly
leverage / vol-targeting the paper book (sleeve weights only, sum 1); EWMA-vol rewrite of the live
`book_allocator`; B alerts panel; back-computing xsfunding; the `_live_sleeves`↔backfill dedup (tech debt).

## Acceptance criteria

- `isv_adaptive_weights`: active-filter, equal-weight bootstrap (no `≥2`-history sleeve), adaptive `L_t` from
  the **disjoint** regime ratio, adaptive `cap_t` from `N_eff`, mixed-history rule, two-sided water-fill;
  unit-tested — (i) empty→`{}`, (ii) no-history→1/N, (iii) low-vol sleeve gets a higher weight than a
  high-vol one, (iv) rising-vol regime (disjoint `ρ>1`) shrinks `L_t` below `L_max` while a flat-vol series
  keeps `L_t=L_max`, (v) `cap_t` never below `1/N`, (vi) **after the water-fill every active weight is in
  `[w_floor, cap_t]` and the book sums to 1** (the iterative loop, not naive renormalize), (vii) a mixed
  book (one sleeve with history, one with `<2` returns) gives the history-less sleeve exactly `w_floor`.
- `sleeve_daily_return(prev_positions, close_by_symbol)` pure fn unit-tested (long & short positions,
  unknown-symbol→0 contribution, no-prior→`None`, zero-notional→`None`); both backfill + snapshot import it
  (no duplicated formula).
- `paper_sleeve_ret` table + `PaperRepo.upsert_sleeve_ret`/`sleeve_returns`; idempotent; tested.
- `backfill_paper_book` builds the causal per-sleeve return window, uses `isv_adaptive_weights`, persists
  returns; integration-tested with a fake warehouse over multiple dates (weights move off 1/N once history
  accrues; low-vol sleeve > high-vol sleeve weight).
- `PaperSnapshotService` reads the trailing returns + uses `isv_adaptive_weights`; tested.
- Re-run backfill in-cluster → `paper_sleeve_ret` populated; `/paper/replay` curve NON-flat and now
  risk-weighted; `/paper` live book uses the same weights; verified post-deploy.
- Full suite green; read-only/additive; no `*.dbn` staged.
