docs: crypto TS-trend sleeve spec + implementation plan
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
582
docs/superpowers/plans/2026-06-21-crypto-tstrend-sleeve.md
Normal file
582
docs/superpowers/plans/2026-06-21-crypto-tstrend-sleeve.md
Normal file
@@ -0,0 +1,582 @@
|
||||
# Crypto TS-Trend Paper-Track Sleeve — Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add a `crypto_tstrend` live paper-track sleeve (long/flat crypto TSMOM) that books daily paper NAV via the nightly Dagster combined-book job, with a configurable portfolio vol-target overlay and a binary Page-Hinkley edge-decay kill, gated `WAIT` until forward-proven.
|
||||
|
||||
**Architecture:** Recomputable-series `ForwardStrategy` (mirrors `CombinedBookStrategy`) that reuses the validated `TrendRunner.run()` unchanged on the existing `crypto_pit` panel, then applies a causal portfolio vol-target overlay + a `PageHinkleyDecay` binary kill incrementally over new days (state carried in the tracker's `extra`). No new data adapter.
|
||||
|
||||
**Tech Stack:** Python 3, pytest, numpy, Dagster 1.12, fxhnt (`TrendRunner`, `ForwardTracker`, `CryptoPitPanelSource`, `STRATEGY_REGISTRY`).
|
||||
|
||||
**Spec:** `docs/superpowers/specs/2026-06-21-crypto-tstrend-sleeve-design.md`
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- **Create** `src/fxhnt/domain/edge_decay.py` — `PageHinkleyDecay` (pure, reusable mean-shift detector).
|
||||
- **Create** `src/fxhnt/application/ts_trend_strategy.py` — `TsTrendForward` (recomputable ForwardStrategy + overlay + kill).
|
||||
- **Modify** `src/fxhnt/registry.py` — add `crypto_tstrend` to `STRATEGY_REGISTRY`.
|
||||
- **Modify** `src/fxhnt/adapters/orchestration/assets.py` — add `crypto_tstrend_nav` asset; add it to `cockpit_forward` deps.
|
||||
- **Modify** `src/fxhnt/adapters/orchestration/definitions.py` — import + add to `combined_book_job.selection` + `defs.assets`.
|
||||
- **Create** `tests/unit/test_edge_decay.py`, `tests/integration/test_tstrend_strategy.py`, `tests/unit/test_registry_tstrend.py`, `tests/integration/test_tstrend_wiring.py`.
|
||||
|
||||
---
|
||||
|
||||
## Task 0: Branch
|
||||
|
||||
- [ ] **Step 1: Create a feature branch** (fxhnt is on `master`; never commit feature work to the default branch)
|
||||
|
||||
```bash
|
||||
cd /home/jgrusewski/Work/fxhnt
|
||||
git checkout -b feat/crypto-tstrend-sleeve
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Commit the spec** (written during brainstorming, currently uncommitted)
|
||||
|
||||
```bash
|
||||
git add docs/superpowers/specs/2026-06-21-crypto-tstrend-sleeve-design.md docs/superpowers/plans/2026-06-21-crypto-tstrend-sleeve.md
|
||||
git commit -m "docs: crypto TS-trend sleeve spec + implementation plan"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 1: PageHinkleyDecay edge-decay detector
|
||||
|
||||
**Files:**
|
||||
- Create: `src/fxhnt/domain/edge_decay.py`
|
||||
- Test: `tests/unit/test_edge_decay.py`
|
||||
|
||||
- [ ] **Step 1: Write the failing tests**
|
||||
|
||||
```python
|
||||
# tests/unit/test_edge_decay.py
|
||||
import json
|
||||
|
||||
import numpy as np
|
||||
|
||||
from fxhnt.domain.edge_decay import PageHinkleyDecay
|
||||
|
||||
|
||||
def test_stationary_positive_never_kills():
|
||||
ph = PageHinkleyDecay(lam=0.05, reenter_days=10)
|
||||
rng = np.random.default_rng(0)
|
||||
alive = [ph.update(0.002 + 0.0005 * float(rng.standard_normal())) for _ in range(500)]
|
||||
assert all(alive)
|
||||
assert not ph.killed
|
||||
|
||||
|
||||
def test_mean_flip_kills():
|
||||
ph = PageHinkleyDecay(lam=0.05, reenter_days=10)
|
||||
for _ in range(200):
|
||||
ph.update(0.002)
|
||||
killed_at = None
|
||||
for i in range(200):
|
||||
if not ph.update(-0.01):
|
||||
killed_at = i
|
||||
break
|
||||
assert killed_at is not None
|
||||
assert ph.killed
|
||||
|
||||
|
||||
def test_rearm_after_hysteresis():
|
||||
ph = PageHinkleyDecay(lam=0.01, reenter_days=5)
|
||||
for _ in range(50):
|
||||
ph.update(0.001)
|
||||
for _ in range(50):
|
||||
if not ph.update(-0.02):
|
||||
break
|
||||
assert ph.killed
|
||||
states = [ph.update(0.0) for _ in range(5)]
|
||||
assert states[-1] is True
|
||||
assert not ph.killed
|
||||
|
||||
|
||||
def test_to_from_dict_roundtrip():
|
||||
ph = PageHinkleyDecay(lam=0.05)
|
||||
for _ in range(20):
|
||||
ph.update(0.001)
|
||||
d = json.loads(json.dumps(ph.to_dict()))
|
||||
ph2 = PageHinkleyDecay.from_dict(d)
|
||||
assert ph2.to_dict() == ph.to_dict()
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
Run: `cd /home/jgrusewski/Work/fxhnt && python -m pytest tests/unit/test_edge_decay.py -v`
|
||||
Expected: FAIL — `ModuleNotFoundError: No module named 'fxhnt.domain.edge_decay'`
|
||||
|
||||
- [ ] **Step 3: Implement `PageHinkleyDecay`**
|
||||
|
||||
```python
|
||||
# src/fxhnt/domain/edge_decay.py
|
||||
"""Page-Hinkley mean-SHIFT edge-decay detector with binary kill + hysteresis. Pure, no I/O.
|
||||
|
||||
Detects a sustained DOWNWARD shift in a return stream's mean (slow-bleed edge death a drawdown
|
||||
trigger misses). Shift-form (not level — level froze during normal operation in the RL context;
|
||||
see pearl_edge_decay_detector_phase1_validated_train_also_decays). Consumes the SHADOW (unkilled)
|
||||
return so it can detect recovery and re-arm while booked exposure is flat.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
|
||||
@dataclass
|
||||
class PageHinkleyDecay:
|
||||
delta: float = 0.0 # deadband below the running mean before drift accumulates
|
||||
lam: float = 0.05 # cumulative-deviation threshold to KILL (pre-registered, conservative)
|
||||
reenter_days: int = 10 # consecutive non-negative shadow returns required to re-arm
|
||||
n: int = 0
|
||||
mean: float = 0.0
|
||||
m_t: float = 0.0
|
||||
m_max: float = 0.0
|
||||
killed: bool = False
|
||||
recover: int = 0
|
||||
|
||||
def update(self, ret: float) -> bool:
|
||||
"""Feed one realized SHADOW daily return; return True if ALIVE (book), False if KILLED (flat)."""
|
||||
if self.killed:
|
||||
self.recover = self.recover + 1 if ret >= 0.0 else 0
|
||||
if self.recover >= self.reenter_days:
|
||||
self.n, self.mean, self.m_t, self.m_max = 0, 0.0, 0.0, 0.0
|
||||
self.killed, self.recover = False, 0
|
||||
return not self.killed
|
||||
self.n += 1
|
||||
self.mean += (ret - self.mean) / self.n
|
||||
self.m_t += (ret - self.mean + self.delta)
|
||||
if self.m_t > self.m_max:
|
||||
self.m_max = self.m_t
|
||||
if (self.m_max - self.m_t) > self.lam:
|
||||
self.killed, self.recover = True, 0
|
||||
return False
|
||||
return True
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {"delta": self.delta, "lam": self.lam, "reenter_days": self.reenter_days,
|
||||
"n": self.n, "mean": self.mean, "m_t": self.m_t, "m_max": self.m_max,
|
||||
"killed": self.killed, "recover": self.recover}
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, d: dict) -> "PageHinkleyDecay":
|
||||
return cls(**d)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they pass**
|
||||
|
||||
Run: `cd /home/jgrusewski/Work/fxhnt && python -m pytest tests/unit/test_edge_decay.py -v`
|
||||
Expected: PASS (4 passed)
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/fxhnt/domain/edge_decay.py tests/unit/test_edge_decay.py
|
||||
git commit -m "feat(domain): PageHinkleyDecay binary edge-decay detector"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2: TsTrendForward strategy
|
||||
|
||||
**Files:**
|
||||
- Create: `src/fxhnt/application/ts_trend_strategy.py`
|
||||
- Test: `tests/integration/test_tstrend_strategy.py`
|
||||
|
||||
- [ ] **Step 1: Write the failing tests**
|
||||
|
||||
```python
|
||||
# tests/integration/test_tstrend_strategy.py
|
||||
import json
|
||||
import math
|
||||
|
||||
import numpy as np
|
||||
|
||||
from fxhnt.application.forward_tracker import ForwardTracker
|
||||
from fxhnt.application.trend_runner import TrendRunner
|
||||
from fxhnt.application.ts_trend_strategy import TsTrendForward, _iso
|
||||
|
||||
|
||||
def _coin(n, start_day, drift, seed):
|
||||
rng = np.random.default_rng(seed)
|
||||
rets = drift + 0.01 * rng.standard_normal(n)
|
||||
close = 100.0 * np.cumprod(1.0 + rets)
|
||||
return {int(start_day + i): float(close[i]) for i in range(n)}
|
||||
|
||||
|
||||
def _panel(n=160):
|
||||
return {f"C{j}": _coin(n, 0, 0.001, seed=j) for j in range(6)}
|
||||
|
||||
|
||||
def _trend_returns(panel):
|
||||
return TrendRunner(panel, lookbacks=(20, 60, 120), vol_window=30, target_vol=0.10,
|
||||
gross_cap=1.0, long_short=False, cost_bps=10.0,
|
||||
periods_per_year=365).run()
|
||||
|
||||
|
||||
def test_inception_books_nothing_and_warms_state(tmp_path):
|
||||
panel = _panel()
|
||||
strat = TsTrendForward(lambda: panel, book_target_vol=0.15)
|
||||
path = str(tmp_path / "st.json")
|
||||
status = ForwardTracker(strat, path).step()
|
||||
assert status.forward_days == 0 # inception books nothing
|
||||
state = json.load(open(path))
|
||||
assert state["extra"]["ph"]["killed"] is False # fresh, alive (history can't pre-kill)
|
||||
assert len(state["extra"]["vol_buf"]) > 0 # overlay warm-started
|
||||
assert state["inception"] == state["last_date"]
|
||||
|
||||
|
||||
def test_subsequent_books_lev_times_raw_no_drift(tmp_path):
|
||||
panel1 = _panel()
|
||||
holder = {"p": panel1}
|
||||
# decay_lam high → never kills in this test; isolates the overlay + no-drift property
|
||||
strat = TsTrendForward(lambda: holder["p"], book_target_vol=0.15, lev_cap=3.0, decay_lam=1e9)
|
||||
path = str(tmp_path / "st.json")
|
||||
ForwardTracker(strat, path).step() # inception
|
||||
vb = json.load(open(path))["extra"]["vol_buf"]
|
||||
# extend the panel by exactly one day per coin
|
||||
last = max(next(iter(panel1.values())).keys())
|
||||
panel2 = {s: dict(series) for s, series in panel1.items()}
|
||||
for s in panel2:
|
||||
panel2[s][last + 1] = panel2[s][last] * 1.002
|
||||
holder["p"] = panel2
|
||||
status = ForwardTracker(strat, path).step()
|
||||
assert status.booked_today == 1
|
||||
res = _trend_returns(panel2)
|
||||
raw_last = float(res.returns[-1])
|
||||
date_last = _iso(int(res.dates[-1]))
|
||||
tv = float(np.std(vb))
|
||||
lev = min(3.0, (0.15 / math.sqrt(365)) / tv) if tv > 1e-12 else 1.0
|
||||
booked = next(d["ret"] for d in json.load(open(path))["days"] if d["date"] == date_last)
|
||||
assert booked == lev * raw_last # booked == lev*raw AND raw == TrendRunner (no drift)
|
||||
|
||||
|
||||
def test_killed_state_flattens_booking(tmp_path):
|
||||
panel1 = _panel()
|
||||
holder = {"p": panel1}
|
||||
strat = TsTrendForward(lambda: holder["p"], book_target_vol=0.15)
|
||||
path = str(tmp_path / "st.json")
|
||||
ForwardTracker(strat, path).step() # inception
|
||||
# force the carried PH into a killed, non-recovering state
|
||||
state = json.load(open(path))
|
||||
state["extra"]["ph"]["killed"] = True
|
||||
state["extra"]["ph"]["recover"] = 0
|
||||
json.dump(state, open(path, "w"))
|
||||
last = max(next(iter(panel1.values())).keys())
|
||||
panel2 = {s: dict(series) for s, series in panel1.items()}
|
||||
for s in panel2:
|
||||
panel2[s][last + 1] = panel2[s][last] * 0.99 # a down day (keeps it killed)
|
||||
holder["p"] = panel2
|
||||
status = ForwardTracker(strat, path).step()
|
||||
assert status.booked_today == 1
|
||||
state2 = json.load(open(path))
|
||||
assert state2["days"][-1]["ret"] == 0.0 # flattened while killed
|
||||
assert len(state2["extra"]["vol_buf"]) > 0 # shadow buffer still advances
|
||||
|
||||
|
||||
def test_extra_json_roundtrips(tmp_path):
|
||||
panel = _panel()
|
||||
strat = TsTrendForward(lambda: panel, book_target_vol=0.15)
|
||||
path = str(tmp_path / "st.json")
|
||||
ForwardTracker(strat, path).step()
|
||||
state = json.load(open(path))
|
||||
json.dumps(state["extra"]) # must not raise
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
Run: `cd /home/jgrusewski/Work/fxhnt && python -m pytest tests/integration/test_tstrend_strategy.py -v`
|
||||
Expected: FAIL — `ModuleNotFoundError: No module named 'fxhnt.application.ts_trend_strategy'`
|
||||
|
||||
- [ ] **Step 3: Implement `TsTrendForward`**
|
||||
|
||||
```python
|
||||
# src/fxhnt/application/ts_trend_strategy.py
|
||||
"""Live-booking crypto TS-trend (long/flat TSMOM) paper track — recomputable series.
|
||||
|
||||
Reuses the validated TrendRunner unchanged on the existing crypto_pit panel, then applies a CAUSAL
|
||||
portfolio vol-target overlay (the configurable book-vol knob) and a binary PageHinkleyDecay kill
|
||||
incrementally over new days, carrying overlay-buffer + detector state in the tracker's `extra`.
|
||||
Mirrors the ForwardStrategy contract: advance(last_date, extra) -> (rows, extra)."""
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime as dt
|
||||
import math
|
||||
from typing import Any, Callable
|
||||
|
||||
import numpy as np
|
||||
|
||||
from fxhnt.application.trend_runner import TrendRunner
|
||||
from fxhnt.domain.edge_decay import PageHinkleyDecay
|
||||
|
||||
_EPOCH = dt.date(1970, 1, 1)
|
||||
|
||||
|
||||
def _today() -> str:
|
||||
return dt.date.today().isoformat()
|
||||
|
||||
|
||||
def _iso(epoch_day: int) -> str:
|
||||
return (_EPOCH + dt.timedelta(days=int(epoch_day))).isoformat()
|
||||
|
||||
|
||||
class TsTrendForward:
|
||||
def __init__(self, load_panel: Callable[[], dict[str, dict[int, float]]], *,
|
||||
lookbacks: tuple[int, ...] = (20, 60, 120), vol_window: int = 30,
|
||||
internal_target_vol: float = 0.10, gross_cap: float = 1.0,
|
||||
cost_bps: float = 10.0, periods_per_year: int = 365,
|
||||
book_target_vol: float = 0.15, lev_cap: float = 3.0, vol_buf_window: int = 30,
|
||||
decay_lam: float = 0.05, decay_reenter_days: int = 10,
|
||||
clock: Callable[[], str] = _today) -> None:
|
||||
self._load_panel = load_panel
|
||||
self._lb = tuple(lookbacks)
|
||||
self._vw = vol_window
|
||||
self._itv = internal_target_vol
|
||||
self._gcap = gross_cap
|
||||
self._cost = cost_bps
|
||||
self._ppy = periods_per_year
|
||||
self._btv = book_target_vol
|
||||
self._levcap = lev_cap
|
||||
self._bufw = vol_buf_window
|
||||
self._lam = decay_lam
|
||||
self._reenter = decay_reenter_days
|
||||
self._clock = clock
|
||||
|
||||
def _raw_series(self) -> tuple[list[int], list[float]]:
|
||||
panel = self._load_panel()
|
||||
res = TrendRunner(panel, lookbacks=self._lb, vol_window=self._vw, target_vol=self._itv,
|
||||
gross_cap=self._gcap, long_short=False, cost_bps=self._cost,
|
||||
periods_per_year=self._ppy).run()
|
||||
return [int(d) for d in res.dates], [float(r) for r in res.returns]
|
||||
|
||||
def advance(self, last_date: str | None,
|
||||
extra: dict[str, Any]) -> tuple[list[tuple[str, float]], dict[str, Any]]:
|
||||
dates, raw = self._raw_series()
|
||||
|
||||
if not extra: # INCEPTION: warm overlay, fresh PH, book nothing
|
||||
vol_buf = raw[-self._bufw:] if raw else []
|
||||
ph = PageHinkleyDecay(lam=self._lam, reenter_days=self._reenter)
|
||||
rows = [(_iso(d), r) for d, r in zip(dates, raw)] # tracker freezes inception at latest, books none
|
||||
new_extra = {"ph": ph.to_dict(), "vol_buf": vol_buf,
|
||||
"last_day": (int(dates[-1]) if dates else None)}
|
||||
return rows, new_extra
|
||||
|
||||
ph = PageHinkleyDecay.from_dict(extra["ph"])
|
||||
vol_buf = [float(x) for x in extra.get("vol_buf", [])]
|
||||
cutoff = last_date or ""
|
||||
btv_daily = self._btv / math.sqrt(self._ppy)
|
||||
rows: list[tuple[str, float]] = []
|
||||
for d, r in zip(dates, raw):
|
||||
iso = _iso(d)
|
||||
if iso <= cutoff: # already booked / pre-inception
|
||||
continue
|
||||
tv = float(np.std(vol_buf)) if len(vol_buf) >= 2 else 0.0
|
||||
lev = min(self._levcap, btv_daily / tv) if tv > 1e-12 else 1.0
|
||||
alive = ph.update(r) # PH consumes shadow raw return
|
||||
rows.append((iso, lev * r if alive else 0.0))
|
||||
vol_buf.append(r) # shadow buffer always advances
|
||||
if len(vol_buf) > self._bufw:
|
||||
vol_buf = vol_buf[-self._bufw:]
|
||||
new_extra = {"ph": ph.to_dict(), "vol_buf": vol_buf,
|
||||
"last_day": (int(dates[-1]) if dates else extra.get("last_day"))}
|
||||
return rows, new_extra
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they pass**
|
||||
|
||||
Run: `cd /home/jgrusewski/Work/fxhnt && python -m pytest tests/integration/test_tstrend_strategy.py -v`
|
||||
Expected: PASS (5 passed)
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/fxhnt/application/ts_trend_strategy.py tests/integration/test_tstrend_strategy.py
|
||||
git commit -m "feat(application): TsTrendForward live paper-track strategy (overlay + decay kill)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Registry entry
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/fxhnt/registry.py` (inside the `STRATEGY_REGISTRY` dict)
|
||||
- Test: `tests/unit/test_registry_tstrend.py`
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
```python
|
||||
# tests/unit/test_registry_tstrend.py
|
||||
from fxhnt.registry import STRATEGY_REGISTRY
|
||||
|
||||
|
||||
def test_crypto_tstrend_registered():
|
||||
e = STRATEGY_REGISTRY["crypto_tstrend"]
|
||||
assert e["state_file"] == "crypto_tstrend_state"
|
||||
assert e["sleeve"] == "crypto-trend"
|
||||
assert e["venue"] == "binance-perp"
|
||||
assert e["gate_spec"] == {"min_days": 60, "min_total_return": 0.0, "min_sharpe": 0.5}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `cd /home/jgrusewski/Work/fxhnt && python -m pytest tests/unit/test_registry_tstrend.py -v`
|
||||
Expected: FAIL — `KeyError: 'crypto_tstrend'`
|
||||
|
||||
- [ ] **Step 3: Add the registry entry**
|
||||
|
||||
In `src/fxhnt/registry.py`, add this entry inside the `STRATEGY_REGISTRY` dict, immediately after the `"unlock"` entry:
|
||||
|
||||
```python
|
||||
"crypto_tstrend": {
|
||||
"display_name": "Crypto TS-momentum (long/flat daily trend, 20/60/120d)",
|
||||
"sleeve": "crypto-trend",
|
||||
"venue": "binance-perp", "state_file": "crypto_tstrend_state",
|
||||
"gate_spec": {"min_days": 60, "min_total_return": 0.0, "min_sharpe": 0.5},
|
||||
},
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `cd /home/jgrusewski/Work/fxhnt && python -m pytest tests/unit/test_registry_tstrend.py -v`
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/fxhnt/registry.py tests/unit/test_registry_tstrend.py
|
||||
git commit -m "feat(registry): register crypto_tstrend sleeve (gate WAIT 60d)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Dagster wiring
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/fxhnt/adapters/orchestration/assets.py` (add `crypto_tstrend_nav`; add to `cockpit_forward` deps)
|
||||
- Modify: `src/fxhnt/adapters/orchestration/definitions.py` (import + job selection + `defs.assets`)
|
||||
- Test: `tests/integration/test_tstrend_wiring.py`
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
```python
|
||||
# tests/integration/test_tstrend_wiring.py
|
||||
def test_asset_importable():
|
||||
from fxhnt.adapters.orchestration.assets import crypto_tstrend_nav
|
||||
assert crypto_tstrend_nav is not None
|
||||
|
||||
|
||||
def test_definitions_load_and_include_tstrend():
|
||||
# Importing `defs` constructs Definitions(...), which validates the asset graph — a missing
|
||||
# crypto_tstrend_nav (referenced by cockpit_forward's deps) would raise here.
|
||||
from fxhnt.adapters.orchestration.assets import crypto_tstrend_nav
|
||||
from fxhnt.adapters.orchestration.definitions import combined_book_job, defs
|
||||
assert crypto_tstrend_nav in defs.assets # in the Definitions assets list
|
||||
assert combined_book_job is not None # job built without error
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `cd /home/jgrusewski/Work/fxhnt && python -m pytest tests/integration/test_tstrend_wiring.py -v`
|
||||
Expected: FAIL — `ImportError: cannot import name 'crypto_tstrend_nav'`
|
||||
|
||||
- [ ] **Step 3: Add the asset in `assets.py`**
|
||||
|
||||
Add this asset definition immediately after the `xsfunding_nav` asset (around line 188):
|
||||
|
||||
```python
|
||||
@asset(deps=[crypto_bars]) # dep on crypto_bars so the crypto_pit panel is fresh before we read it
|
||||
def crypto_tstrend_nav(context: AssetExecutionContext) -> dict:
|
||||
"""Paper-forward crypto TS-momentum (long/flat daily TSMOM on the survivorship-free crypto_pit panel)."""
|
||||
from fxhnt.adapters.data.crypto_pit_panel import CryptoPitPanelSource
|
||||
from fxhnt.application.ts_trend_strategy import TsTrendForward
|
||||
s = get_settings()
|
||||
|
||||
def load_panel() -> dict:
|
||||
raw = CryptoPitPanelSource(s.crypto_pit_dir).panel() # {sym: {day: (close, qvol, funding)}}
|
||||
return {sym: {d: v[0] for d, v in series.items()} for sym, series in raw.items()}
|
||||
|
||||
return _run_paper_tracker(
|
||||
context, "crypto_tstrend_nav",
|
||||
lambda: TsTrendForward(load_panel, book_target_vol=0.15),
|
||||
"crypto_tstrend_state",
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Add `crypto_tstrend_nav` to the `cockpit_forward` deps**
|
||||
|
||||
In `assets.py` change the `cockpit_forward` decorator (line ~207) from:
|
||||
|
||||
```python
|
||||
@asset(deps=[combined_forward_nav, sixtyforty_nav, xsfunding_nav, unlock_nav])
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```python
|
||||
@asset(deps=[combined_forward_nav, sixtyforty_nav, xsfunding_nav, unlock_nav, crypto_tstrend_nav])
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Wire it in `definitions.py`**
|
||||
|
||||
Add `crypto_tstrend_nav,` to the import block from `fxhnt.adapters.orchestration.assets`:
|
||||
|
||||
```python
|
||||
from fxhnt.adapters.orchestration.assets import (
|
||||
cockpit_forward,
|
||||
combined_forward_nav,
|
||||
crypto_bars,
|
||||
crypto_tstrend_nav,
|
||||
futures_bars,
|
||||
sixtyforty_nav,
|
||||
unlock_nav,
|
||||
xsfunding_nav,
|
||||
)
|
||||
```
|
||||
|
||||
Add `crypto_tstrend_nav,` to BOTH the `combined_book_job` selection and the `defs` assets list (same line as `xsfunding_nav, unlock_nav,`):
|
||||
|
||||
```python
|
||||
xsfunding_nav, unlock_nav, crypto_tstrend_nav,
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Run tests to verify they pass**
|
||||
|
||||
Run: `cd /home/jgrusewski/Work/fxhnt && python -m pytest tests/integration/test_tstrend_wiring.py -v`
|
||||
Expected: PASS (2 passed)
|
||||
|
||||
- [ ] **Step 7: Commit**
|
||||
|
||||
```bash
|
||||
git add src/fxhnt/adapters/orchestration/assets.py src/fxhnt/adapters/orchestration/definitions.py tests/integration/test_tstrend_wiring.py
|
||||
git commit -m "feat(orchestration): wire crypto_tstrend_nav into nightly combined-book job + cockpit"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Full verification
|
||||
|
||||
- [ ] **Step 1: Run the full test suite**
|
||||
|
||||
Run: `cd /home/jgrusewski/Work/fxhnt && python -m pytest -q`
|
||||
Expected: all green (the ~467 existing + new tests)
|
||||
|
||||
- [ ] **Step 2: Type-check**
|
||||
|
||||
Run: `cd /home/jgrusewski/Work/fxhnt && python -m mypy src/fxhnt`
|
||||
Expected: clean (no new errors)
|
||||
|
||||
- [ ] **Step 3: Final commit (if any fixups were needed)**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "chore: crypto_tstrend sleeve — green tests + mypy clean"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Notes for the executor
|
||||
|
||||
- **Honest falsification:** the `PageHinkleyDecay` unit tests assert no-kill on a *synthetic stationary-positive* series (the defensible no-false-positive guarantee). On the REAL validated series it is expected and CORRECT for PH to kill during the 2022 crypto bear and re-arm in 2023 — that is the feature working, not a bug; do not add a hard assertion that it never kills on real history.
|
||||
- **No-drift is the load-bearing property:** `TsTrendForward` calls `TrendRunner.run()` directly, so the live signal IS the validated backtest signal. `test_subsequent_books_lev_times_raw_no_drift` guards this — keep it.
|
||||
- **Do not wire the tracker `kill_dd`** for this sleeve (would double-kill with the PH detector). The PH decay is the sleeve's edge-death guard.
|
||||
- **`s.crypto_pit_dir`** is the settings field already used by `xsfunding`/momentum assets; if it is absent in `get_settings()`, use the same path expression those assets use (verify against `assets.py` line ~196 / 202).
|
||||
```
|
||||
@@ -0,0 +1,235 @@
|
||||
# Crypto TS-Trend Paper-Track Sleeve — Design (rev. 2)
|
||||
|
||||
**Date:** 2026-06-21
|
||||
**Status:** Design (pending approval) — rev. 2 after critical review against fxhnt source
|
||||
**Repo:** fxhnt (Python agentic fund)
|
||||
|
||||
## Motivation
|
||||
|
||||
The "speed up profits" exploration (2026-06-20/21) falsified the high-frequency direction twice (crypto
|
||||
intraday cross-sectional momentum net SR −21→−0.67; cascade-reversion losers keep losing = continuation;
|
||||
see `pearl_crypto_intraday_no_edge_faster_is_feetrap`). The crypto edges are daily-or-slower and
|
||||
flow-based. The one validated, uncorrelated, **not-yet-deployed** return engine is crypto time-series
|
||||
trend (TSMOM, long/flat):
|
||||
|
||||
- Re-verified 2026-06-21 by running the actual `TrendRunner` on the real survivorship-free `crypto_pit`
|
||||
panel (180 coins, 2472 days): `long_short=False` → annualized Sharpe **+1.25** (memory ~1.23 ✓), in
|
||||
**8.2 s** wall-clock (full recompute — cheap enough to run nightly).
|
||||
- corr ~0 to the funding edge → adds return at diversified risk. It is a **directional crypto-beta-timing**
|
||||
engine (carries crypto beta when "on"), NOT market-neutral alpha. Intended: the book's directional sleeve.
|
||||
|
||||
Missing piece = the **live paper-forward track** so it accumulates a real forward record and graduates
|
||||
through the gate like `xsfunding` / `unlock`.
|
||||
|
||||
## Goal
|
||||
|
||||
Add `crypto_tstrend` as a live paper-track sleeve that books daily paper NAV from inception via the
|
||||
nightly Dagster combined-book job and surfaces in the cockpit with `gate=WAIT` (60 days, Sharpe ≥ 0.5)
|
||||
until forward-proven. No capital until the gate clears.
|
||||
|
||||
## Key design decisions (after critical review)
|
||||
|
||||
1. **Recomputable-series pattern, NOT the xsfunding snapshot/position-carry pattern.** Trend's return is
|
||||
just the realized price return of held positions, which `TrendRunner` already computes from a price
|
||||
panel. So the sleeve mirrors `CombinedBookStrategy` (forward_tracker.py:123): each night it recomputes
|
||||
the full series and returns rows; the tracker books only days strictly after `last_date`. This
|
||||
**reuses `TrendRunner.run()` unchanged** → zero live/backtest drift, no bespoke `advance` math.
|
||||
2. **Reuse the existing crypto_pit panel — no new data adapter.** `crypto_bars` (assets.py:43) already
|
||||
REFRESHes `crypto_pit` nightly (survivorship-free top-120-by-volume ∪ dead) and `CryptoPitPanelSource`
|
||||
(crypto_pit_panel.py) reads it: `{sym: {day: (close, qvol, funding)}}`. The sleeve reads close-only
|
||||
from it. The earlier scorecard worry ("trend needs a rolling-price-history live source, more plumbing")
|
||||
is wrong — the source already exists.
|
||||
3. **Static (configurable) vol-target, NOT adaptive ISV re-sizing.** Own ML/RL research: continuous
|
||||
trust-scaling on a single noisy edge whipsaws (Bongaerts FAJ 2020) and re-introduces the RL
|
||||
train→eval-collapse pathology. The legitimate adaptive-control home is the book-of-books continuous
|
||||
edge-decay layer — a SEPARATE later spec. This sleeve gets a SIMPLE BINARY decay kill only.
|
||||
4. **`target_vol` semantics made precise.** `TrendRunner` sizes **per-asset** (`p = sig·target_vol_daily/rv`)
|
||||
then caps GROSS at 1.0 → book vol is emergent (~86%), NOT equal to `target_vol`. The configurable
|
||||
book-vol knob is a SEPARATE **causal portfolio vol-target overlay** applied on top (this is the sizer
|
||||
whose −15%/−23%/−30% maxDD at 10/15/20% the 2026-06-21 probe measured). TrendRunner's internal
|
||||
per-asset `target_vol` stays at its validated default; the overlay normalizes book vol to the target.
|
||||
|
||||
## Architecture
|
||||
|
||||
Reuses generic forward-tracker / registry / Dagster-schedule / ingest / cockpit infra. New code = one
|
||||
domain helper (decay detector) + one strategy + wiring.
|
||||
|
||||
### 1. Edge-decay detector (NEW, reusable)
|
||||
|
||||
**File:** `src/fxhnt/domain/edge_decay.py` — **Class:** `PageHinkleyDecay`
|
||||
|
||||
Canonical Page-Hinkley **mean-shift** detector with binary kill + hysteresis (per
|
||||
`pearl_edge_decay_detection_is_a_missing_abstraction_layer`; shift-form, not level — level froze in the
|
||||
RL context per `pearl_edge_decay_detector_phase1_validated_train_also_decays`).
|
||||
|
||||
```python
|
||||
@dataclass
|
||||
class PageHinkleyDecay:
|
||||
delta: float = 0.0 # drift tolerance (per-day return units)
|
||||
lam: float = 0.05 # cumulative-deviation threshold to KILL (pre-registered, conservative)
|
||||
reenter_days: int = 10 # consecutive non-negative SHADOW days to re-arm (hysteresis)
|
||||
# mutable state (all JSON scalars): mean, n, m_t, ph_min, killed, recover
|
||||
def update(self, ret: float) -> bool:
|
||||
"""Feed one realized SHADOW daily return; return True if ALIVE, False if KILLED.
|
||||
ALIVE→KILLED when m_t - min(m_t) > lam (sustained downward mean shift).
|
||||
KILLED→ALIVE only after `reenter_days` consecutive ret >= 0 (then reset PH accumulators)."""
|
||||
def to_dict(self) -> dict: ...
|
||||
@classmethod
|
||||
def from_dict(cls, d: dict) -> "PageHinkleyDecay": ...
|
||||
```
|
||||
|
||||
Pre-registered (NOT tuned): `lam=0.05, delta=0.0, reenter_days=10`. Falsification: on the validated
|
||||
backtest series it must NOT kill during normal operation; must KILL on a synthetic mean-flip series.
|
||||
|
||||
### 2. Live-booking strategy (NEW)
|
||||
|
||||
**File:** `src/fxhnt/application/ts_trend_strategy.py` — **Class:** `TsTrendForward` (implements `ForwardStrategy`)
|
||||
|
||||
```python
|
||||
class TsTrendForward:
|
||||
def __init__(self, load_panel: Callable[[], dict[str, dict[int, float]]], *,
|
||||
lookbacks=(20, 60, 120), vol_window=30, internal_target_vol=0.10,
|
||||
gross_cap=1.0, cost_bps=10.0, periods_per_year=365,
|
||||
book_target_vol=0.15, lev_cap=3.0, vol_buf_window=30,
|
||||
decay_lam=0.05, decay_reenter_days=10,
|
||||
clock=_today) -> None: ...
|
||||
|
||||
def advance(self, last_date, extra) -> tuple[list[tuple[str, float]], dict]:
|
||||
"""Recomputable series + causal overlay + binary decay kill.
|
||||
|
||||
1. panel = load_panel() # {sym:{day:close}} (close extracted from CryptoPitPanelSource)
|
||||
2. res = TrendRunner(panel, lookbacks, vol_window, internal_target_vol, gross_cap,
|
||||
long_short=False, cost_bps, periods_per_year).run()
|
||||
→ raw_rets[], dates[] (dates = epoch-day each return is REALIZED on; ~8s)
|
||||
3. INCEPTION (extra empty / last_date None):
|
||||
- warm vol_buf = trailing `vol_buf_window` raw_rets from history (overlay warm-start ONLY)
|
||||
- PH = fresh ALIVE (history decay must NOT pre-kill the forward track)
|
||||
- return ALL rows [(iso(d), raw) ...] so tracker freezes inception at latest date
|
||||
(tracker books NONE on inception); persist extra {ph, vol_buf, last_day}
|
||||
4. SUBSEQUENT — for each d in dates with iso(d) > last_date, in chronological order:
|
||||
tv = std(vol_buf) (daily); lev = clip(book_target_vol/sqrt(ppy) / max(tv,eps), 0, lev_cap)
|
||||
alive = ph.update(raw_d) # PH consumes SHADOW raw return (continuous)
|
||||
booked = lev*raw_d if alive else 0.0
|
||||
rows.append((iso(d), booked)); vol_buf.append(raw_d) [trim]; last_day = d
|
||||
persist extra {ph: ph.to_dict(), vol_buf, last_day}; return ONLY new rows
|
||||
"""
|
||||
```
|
||||
|
||||
**Shadow semantics** (mirror the tracker's `kill_dd` shadow): the overlay buffer and PH always consume
|
||||
the *raw* (unkilled) return, so the kill can detect recovery and re-arm even while booked exposure is 0.
|
||||
**Causality:** `lev` uses the trailing buffer of returns *before* `d`; `raw_d` is realized over `[d, nxt]`
|
||||
by TrendRunner. No lookahead. **Path-stability:** PH + vol_buf are carried in `extra` and advanced only
|
||||
over NEW days, so panel top-N churn / historical revision cannot retro-change a booked kill decision (the
|
||||
tracker also books only strictly-new days, so past NAV is sticky regardless).
|
||||
|
||||
### 3. Registry entry
|
||||
|
||||
**File:** `src/fxhnt/registry.py`
|
||||
```python
|
||||
"crypto_tstrend": {
|
||||
"display_name": "Crypto TS-momentum (long/flat daily trend, 20/60/120d)",
|
||||
"sleeve": "crypto-trend", "venue": "binance-perp",
|
||||
"state_file": "crypto_tstrend_state",
|
||||
"gate_spec": {"min_days": 60, "min_total_return": 0.0, "min_sharpe": 0.5},
|
||||
},
|
||||
```
|
||||
|
||||
### 4. Dagster wiring
|
||||
|
||||
**File:** `src/fxhnt/adapters/orchestration/assets.py`
|
||||
```python
|
||||
@asset(deps=[crypto_bars]) # MUST dep on crypto_bars so the panel is fresh before we read it
|
||||
def crypto_tstrend_nav(context: AssetExecutionContext) -> dict:
|
||||
from fxhnt.adapters.data.crypto_pit_panel import CryptoPitPanelSource
|
||||
from fxhnt.application.ts_trend_strategy import TsTrendForward
|
||||
s = get_settings()
|
||||
def load_panel():
|
||||
raw = CryptoPitPanelSource(s.crypto_pit_dir).panel() # {sym:{day:(close,qvol,funding)}}
|
||||
return {sym: {d: v[0] for d, v in series.items()} for sym, series in raw.items()}
|
||||
return _run_paper_tracker(
|
||||
context, "crypto_tstrend_nav",
|
||||
lambda: TsTrendForward(load_panel, book_target_vol=0.15),
|
||||
"crypto_tstrend_state",
|
||||
)
|
||||
```
|
||||
|
||||
**File:** `src/fxhnt/adapters/orchestration/definitions.py` — add `crypto_tstrend_nav` to the
|
||||
`combined_book_job` selection.
|
||||
|
||||
**File:** `assets.py` — **add `crypto_tstrend_nav` to the explicit `cockpit_forward` dep list**
|
||||
(currently `@asset(deps=[combined_forward_nav, sixtyforty_nav, xsfunding_nav, unlock_nav])`, line ~207),
|
||||
else cockpit may run before the track materializes / omit it. **(latent-bug catch from review.)**
|
||||
|
||||
`book_target_vol` (and `decay_lam`/`decay_reenter_days`) are set at this composition root → re-tunable
|
||||
with no strategy-code change.
|
||||
|
||||
### 5. Drawdown kill-switch note (corrected)
|
||||
|
||||
`_run_paper_tracker` (assets.py:127) builds `ForwardTracker(strategy, path).step()` and **does NOT pass
|
||||
`kill_dd`** — so per-sleeve paper tracks (xsfunding, unlock, and this one) have **no** tracker drawdown
|
||||
kill. The strategy-level `PageHinkleyDecay` is therefore this sleeve's **primary** (and only) edge-death
|
||||
guard, not a complement. We deliberately leave the tracker `kill_dd` OFF to avoid a double-kill
|
||||
interaction; it remains available if a level-trigger is wanted later.
|
||||
|
||||
### 6. Tests (mirror xsfunding pattern + new)
|
||||
|
||||
- `tests/unit/test_edge_decay.py` — `PageHinkleyDecay`: stays ALIVE on stationary-positive series; KILLS
|
||||
on positive→negative mean-flip; RE-ARMS only after `reenter_days` non-negative days; `to_dict`/`from_dict`
|
||||
round-trip; does NOT kill on the validated long/flat backtest return series (falsification).
|
||||
- `tests/integration/test_tstrend_strategy.py` — fake `load_panel` (synthetic uptrend/downtrend panel):
|
||||
- inception → returns rows, tracker books none, extra warm-started (vol_buf full, PH alive);
|
||||
- subsequent day → booked == `lev*raw` (hand-computed), `lev` from carried buffer, causal;
|
||||
- **no-drift**: the raw return for a new day equals `TrendRunner.run()`'s tail value for that date;
|
||||
- decay kill → booked flattens to 0, vol_buf still advances (shadow), re-arms after hysteresis;
|
||||
- `extra` JSON round-trips through `json.dumps`/`loads`.
|
||||
- `tests/integration/test_orchestration_assets.py` — extend: `crypto_tstrend_nav` materializes a valid
|
||||
days-list state file from a synthetic panel; asset is reachable after `crypto_bars`.
|
||||
- `tests/unit/test_registry.py` — extend: `crypto_tstrend` present with required fields.
|
||||
|
||||
## Data flow
|
||||
|
||||
```
|
||||
nightly 23:30 UTC combined_book_job
|
||||
→ crypto_bars (REFRESH crypto_pit panel)
|
||||
→ crypto_tstrend_nav [deps=crypto_bars]
|
||||
→ TsTrendForward(load_panel, book_target_vol=0.15)
|
||||
→ TrendRunner.run(long_short=False, ppy=365) (~8s) → raw_rets, dates
|
||||
→ per new day: causal vol-overlay (lev) · PH shadow kill → booked
|
||||
→ ForwardTracker.step() → crypto_tstrend_state.json (days-list NAV)
|
||||
→ cockpit_forward [deps += crypto_tstrend_nav] → cockpit DB (gate=WAIT until 60d & Sharpe≥0.5)
|
||||
```
|
||||
|
||||
## Error handling
|
||||
|
||||
- Transient (network/disk): handled by `_run_paper_tracker` (logged, state unchanged, job not blocked).
|
||||
- Genuine bugs (ValueError/KeyError/AttributeError): propagate and fail loudly.
|
||||
- Empty panel / all symbols too short: TrendRunner returns empty `dates` → strategy books nothing that
|
||||
day (no crash). At inception with empty history: freeze at `_today_iso()`, empty extra warmed lazily.
|
||||
- Insufficient vol_buf (shouldn't happen post-warm-start): `lev` falls back to 1.0 (eps-guarded std).
|
||||
|
||||
## Risks / honest caveats
|
||||
|
||||
- **Directional beta, not alpha** — long crypto when trends are up; goes flat in bears but bled in 2022.
|
||||
corr~0 to funding is the diversification claim, not market-neutrality.
|
||||
- **Backtest → deployable haircut** — Sharpe ~1.25 backtest → realistically ~0.8–1.0 deployable; the
|
||||
forward track is the truth-teller (same role xsfunding's forward track plays for tail-basis).
|
||||
- **Crash co-movement** — in a crypto deleveraging cascade funding (short-convex) and trend (de-risks but
|
||||
lags fast cascades) can both hurt before trend flattens. Book-of-books sizing (later) handles this; this
|
||||
sleeve alone does not.
|
||||
- **Recompute redundancy** — full 8s recompute nightly is wasteful but correct and drift-free; acceptable
|
||||
at this cadence. If the panel grows large enough to matter, factor a single-day position helper later.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- `crypto_tstrend` registered; nightly job materializes `crypto_tstrend_state.json` (days-list shape);
|
||||
`cockpit_forward` depends on it and surfaces it with `gate=WAIT`, NAV booking from inception.
|
||||
- **No drift**: a test asserts the strategy's raw per-day return equals `TrendRunner.run()`'s value for
|
||||
the same date on a synthetic panel.
|
||||
- `PageHinkleyDecay`: no-kill on the validated window, kills on a synthetic mean-flip, re-arms with
|
||||
hysteresis, state persists through `extra` JSON.
|
||||
- Overlay is causal (uses only trailing returns) and `book_target_vol` controls realized book vol;
|
||||
the −15/−23/−30% maxDD at 10/15/20% targets reproduces the 2026-06-21 probe.
|
||||
- All new + existing tests green; mypy clean.
|
||||
- `book_target_vol`, `decay_lam`, `decay_reenter_days` adjustable at the asset composition root without
|
||||
touching strategy code.
|
||||
```
|
||||
Reference in New Issue
Block a user