Commit Graph

27 Commits

Author SHA1 Message Date
jgrusewski
5ed94ddc97 feat(surfer): cross-exchange funding frontier (a) — fails (carry trap re-found)
Fetched Bybit funding (OKX fetch failed, so Binance+Bybit) for 18 majors. Cross-venue
DISPERSION & Binance-premium signals all negative. Only positive = carry_multivenue +0.84
(OOS+0.80, corr_mom+0.01, marg_t+2.58) -- BUT Binance-only control on the SAME 18 majors =
+0.69, proving the edge is MAJOR-COIN SELECTION not the cross-venue angle (multi-venue adds
only +0.15 noise-reduction). This is the known cherry-picked carry trap (majors +0.8 ->
broad-clean -0.03). Almost re-fooled by the seductive number; the Binance-only control
disconfirmed it. Orthogonality doesn't rescue universe-fragility. Databento has no crypto/
on-chain data either. Both orthogonal frontiers closed. Product = residual-momentum + VRP.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 19:55:55 +02:00
jgrusewski
a7d6671dea feat(surfer): Reflexivity Harvester design — killed by marginal-alpha gate
Synthesized all findings into a novel crypto-native design (residual-momentum long +
exhaustion/death-weighted short + vol-regime gross-gate), spawned 4 research agents,
ran the decisive test the critic+exhaustion-researcher both named. Exhaustion-short is
momentum RE-SPELLED: corr +0.82 to residual momentum, marginal alpha t -0.37 (negative),
combined OOS +0.50 < momentum-alone +0.77 (adding it HURTS). Momentum ALREADY harvests
the crypto death-alpha (it already shorts dying coins -- the survivorship finding);
explicit death-features double-count. Regime-gate untestable (n~4 crashes). Ship Comp-1.
Product unchanged: residual-momentum (OOS +0.77, CPCVmed +0.82, DSR 0.57 deflated-by-50)
+ tail-managed VRP. The marginal-alpha gate killed our own creative design = discipline working.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 19:43:42 +02:00
jgrusewski
b664673df3 feat(surfer): Fear&Greed gate — sentiment regime-dependent, not robust
F&G market-timing: full-sample IC negative (fear->continuation), recent OOS positive,
IS/OOS sign-flip (-0.65->+0.75), 2026 flipped -0.8 = regime-dependent, not robust.
Free non-price is mostly market-wide timing (low-breadth, regime-prone); cross-sectional
on-chain (the valuable kind) needs paid data. Non-price free frontier mostly dead.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 19:35:37 +02:00
jgrusewski
a9d1af9ec8 feat(surfer): minute-horizon gate — deep ML (Mamba2/CfC/TLOB) has no fuel
Before building deep sequence models, tested whether cost-surviving signal exists at
the minute horizon for them to extract. 12 perps, 60d free 1m klines (OFI proxy from
takerBuyQuote/quoteVol), features -> forward 1/5/15min returns net of 5bp taker. ALL
features negative net (best gross ~0.2bp vs 5bp cost = 25x gap); reversal IC -0.027 and
vol-continuation +0.009 are real but ~25x too small; OFI/taker-imbalance ~zero predictive
power. -> No minute-horizon signal -> Mamba2/CfC/TLOB have nothing to extract -> don't
build (no model lifts IC 25x). Crossing-cost wall holds in crypto (sec-ES 100x, min-crypto
25x); maker=adverse-selection wall, full-LOB=colocation territory. Gate saved a multi-week
GPU build for $0/10min. Product remains the daily two-sleeve book.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 19:09:04 +02:00
jgrusewski
bc4cc46775 feat(surfer): residual (beta-stripped) momentum upgrade — net +0.40->+0.57 @$20M
Phase-1 upgrade applied to the PoC base sleeve: rank residual momentum (each coin's
return stripped of its rolling beta to the equal-weight market) instead of raw momentum.
Removes high-beta coins dominating the sort by market co-movement. Backtest: gross
+0.82->+0.99, net @$20M +0.40->+0.57, capacity $5M +0.60->+0.78, CPCV-med +0.47->+0.59,
combined two-sleeve book +1.56->+1.66. Same edge, cleaner construction. Live panel depth
bumped to cover the 60d beta window. No lookahead (beta + cumsum use closed bars).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 18:41:40 +02:00
jgrusewski
74b9d092ad feat(surfer): Phase 1 RL-gate — no rich signal set; RL not justified (+ resid-mom upgrade)
Tested 5 theoretically-distinct new crypto signals (residual-momentum, MAX/lottery,
idiosyncratic-vol, Amihud-illiquidity, acceleration) on PIT top-50 with full gauntlet
(OOS/CPCV/deflation/coin-bootstrap) + correlation to momentum. NO new DISTINCT validated
signal: survivors = momentum + residual-momentum (corr 0.95 = same edge). Others fail
(lottery/accel negative; ivol/amihud fail bootstrap-robustness, amihud closest miss
corr+0.20/boot0.75). DECISIVE: validated signal set is thin (one alpha + VRP diversifier)
-> RL/PPO has no rich interacting set to combine -> would not beat the linear baseline OOS
-> don't build the RL (the falsifiable gate returns NO; avoids the 65th-commit trap).
SMALL WIN: residual (beta-stripped) momentum +0.72 > raw +0.57 -- better base construction.
Product = simple two-sleeve book. RL needs non-price signals to ever justify itself.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 18:36:21 +02:00
jgrusewski
f986510099 feat(surfer): wire tail-managed VRP into PoC as second sleeve (two-sleeve book)
backtest now reports momentum + VRP (tail-managed, corr -0.04) + COMBINED book
(momentum 70% / VRP 30% risk) Sharpe +1.56, all years positive. paper marks BOTH
sleeves + combined equity each run; status shows all three + live Sharpes. VRP gate
('don't sell vol when DVOL rising/elevated') runs live -- currently 'flat (gated)'.
cwd-independence fixes: pit_sweep.load + _vrp_calib now use abs paths so the cron paper
run calibrates VRP correctly from $HOME (was silently falling back to zero-VRP). Live
panel depth bumped to cover the 60d VRP gate window. Honest: VRP +2.85 is the no-Greeks
proxy (real ~1-1.5) so combined is optimistic (~0.8-1.0 realistic) -- still > momentum alone.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 17:29:40 +02:00
jgrusewski
ad7feb61ff feat(surfer): tail-managed VRP — real second sleeve (with honest haircut)
'Don't sell vol when implied vol rising' (point-in-time, no lookahead verified) takes
crypto short-vol baseline +1.03 (skew -8.3, 2025 -0.26/2026 -0.77) to combo +2.89,
ALL years positive (2026 +2.26), worst -21sigma->-17sigma. Effect robust across gate
variants (rising/level/combo/loss_cap all fix 2025-26) = real economic effect (implied
leads realized), not one lucky rule. Confirms the -21sigma tail + recent decay were the
same mis-timing. Diversifier: corr -0.04, combined +2.57. HONEST HAIRCUT: +2.9 optimistic
(gate-selection best-of-5 + no-Greeks proxy idealization + residual -15sigma tail + 5y
sample) -> realistic ~1.0-1.5; but corr+edge robust -> combined ~1.37 > momentum 0.66.
Real second sleeve, size SMALL (tail reduced not gone). Best diversifier found.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 17:17:09 +02:00
jgrusewski
9642ad3156 feat(surfer): crypto VRP diversifier check — first real diversifier, but tail+decay
Vol-risk-premium (short-vol BTC+ETH, Deribit DVOL implied minus realized, short-variance
daily P&L proxy). VRP Sharpe +1.03, CORRELATION to momentum -0.03 (genuinely uncorrelated),
combined +1.22 (~doubles momentum-alone +0.66). FIRST genuine diversifier found -- has BOTH
edge AND low-corr (futures trend had corr but no edge -> diluted). CAVEATS: (1) catastrophic
negative skew -8.32, worst-day -21sigma (short-vol blowup risk; Sharpe flatters; must
tail-manage + size small) (2) recent decay -- negative 2025/2026 (+1.22 is a 2021-24 artifact).
Real and promising, categorically better than futures, but pursue carefully not naive deploy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 17:13:06 +02:00
jgrusewski
dbbe8f7b22 feat(surfer): multi-asset futures test — uncorrelated but no edge (dilutes)
Expanded futures universe 22->39 liquid CME roots ($16.39 credits, 16y daily). (A)
breadth did NOT unlock an edge: XS momentum + TS trend all negative/zero, DSR 0
(efficient market, unlike crypto). (B) diversifier check: futures-trend +0.01,
crypto-momentum +0.88, CORRELATION -0.01 (genuinely uncorrelated) but combined +0.63
< +0.88 -> an uncorrelated sleeve with zero standalone edge DILUTES, not diversifies.
A diversifier needs low-corr AND positive edge; futures trend has only the former.
Caveat: roll-zeroing understates futures trend/carry (floor work ~+0.05-0.14 w/ proper
rolls) -> marginal at best. Crypto momentum still the only real edge.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 17:08:40 +02:00
jgrusewski
ec308346b2 fix(surfer): cwd-independent PoC + fixed-phase weekly rebalance (cron-ready)
- Rebalance keyed on FIXED calendar phase (epoch-day%7==0 -> Thursdays) instead of
  drifting days[0]%K, so backtest and live agree on rebalance timing.
- STATE path absolute (relative to script) -> paper mode runs from any cwd (cron-safe).
- Backtest unchanged (net +0.40@$20M, CPCV-med +0.47).
Installed system crontab (daily 12:07): marks paper PnL daily, rebalances Thursdays,
logs to data/surfer/paper_runs.log. Forward-test seeded inception 2026-06-06.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 16:55:42 +02:00
jgrusewski
d6eb5b2ada feat(surfer): end-to-end Python PoC (backtest|regime|paper|status)
One-file PoC of the validated crypto XS-momentum edge. ONE shared compute_weights()
drives backtest AND live (no skew); no lookahead; funding-as-PnL; sqrt-impact cost.
Backtest reproduces research (gross +0.83, net +0.43@$20M, capacity curve, turnover
9.8%). Regime diagnostic = clean NEGATIVE (no feature predicts momentum favorability,
IC~0 IS/OOS) -> de Prado meta-labeling/regime overlay empirically unwarranted, runs
flat-sized. Paper mode = live forward-test (Binance ASCII perps, market-neutral book,
intended trades + paper PnL persisted). Python is the right tool for weekly crypto MFT;
foxhunt Rust/CUDA is HFT-ES-specific (concepts reused, plumbing not).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 16:44:41 +02:00
jgrusewski
4336a71e26 feat(surfer): diversifier hunt — no robust partner (TS-trend real but stale)
Directional TS-trend (fixed earlier demeaning bug) is a real edge (bootstrap frac>0=1.00)
with low corr to momentum (+0.12) and lifts full-sample Sharpe +0.67->+0.71 -- but per-year/
OOS expose it as STALE (value all in 2021 +2.65; recent 2024 -0.16/2026 -0.89), so it DRAGS
the book OOS to +0.24 vs momentum's strong recent. 2022 crisis-alpha hypothesis failed
(choppy bear whipsawed trend). Within-crypto diversifiers exhausted (carry/lowvol/reversal/
size/TS-trend all fail) -> momentum is a robust single-sleeve edge; real diversifier needs a
different market/data. Discipline: full-Sharpe said add, OOS said stale.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 16:24:55 +02:00
jgrusewski
99b747ad2c feat(surfer): turnover optimization resolves capacity — deployable momentum spec
Weekly rebalance + 5d weight-smoothing + top-30 liquid: gross Sharpe +0.80->+0.94
(smoothing cuts whipsaw), turnover 30%->10.6%, capacity dead-by-$20M -> net +0.76/$5M,
+0.58/$20M, +0.38/$50M, viable ~$100M. top-30 = breadth/liquidity optimum. The
small-capacity verdict was a daily-rebalance artifact. DEPLOYABLE SPEC: crypto XS
momentum, top-30 liquid perps, 20d signal, weekly rebal + 5d smoothing, market-neutral,
net Sharpe ~0.6-0.8 at $5-20M, survivorship-confirmed. Single edge (no validated diversifier).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 16:20:12 +02:00
jgrusewski
a2cbed4a83 feat(surfer): capstone — low-vol fails battery; momentum is small-capacity
(2) Low-vol FAILS its full battery (standalone ~0/neg all windows, coin-bootstrap
frac>0=0.45 coin-flip) -> its earlier diversifier value was a winsor artifact ->
NO validated second edge, momentum stands alone. (1) Slippage/capacity model
(sqrt-impact eta=1, daily rebalance): book gross +0.62 but net M +0.41 / M +0.22 /
0M -0.11 / 0M -0.50 -> real but small-capacity (~$1-5M), lives in smaller alts.
Favorable caveats: eta=1 conservative, daily rebalance of 20d signal wastes turnover.
Key untested lever: weekly rebalance + smoothing + top-20 liquid -> ~5x less turnover.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 16:17:33 +02:00
jgrusewski
7edeec9106 feat(surfer): proper (a)robust-construction + (b)diversifier research
(a) All 6 momentum constructions collapse to ~0.2 under aggressive +/-20% winsor
(momentum leans on large moves); volscaled/winsorinput degrade least, none robust to
the cap. (b) LOWVOL is genuinely uncorrelated (corr -0.13) and lifts the stressed book
+0.19->+0.34; TStrend correlated; carry/reversal/size don't help. Book=momentum+lowvol.
Honest deployable bracket ~0.2 (harsh winsor) to ~0.6 (fair); +/-20% blanket winsor
double-counts (death-spiral already excluded) so truth ~0.5. Deciding next: per-coin
ADV slippage model, not blanket winsor. Corrected the script's over-claiming verdict.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 16:12:55 +02:00
jgrusewski
ca3877c328 feat(surfer): execution-realism stress — honest momentum magnitude ~0.4-0.6
Realism layers on PIT mom_20 TOPK=50: robust to 20bp (+0.59) and liquidity floor
(top-50 already deep, no effect); death-spiral exclusion +0.60. Winsorize +/-20%/day
cuts to +0.50 and weakens 2021/22 -> edge leans partly on large moves. COMBINED
realistic stress +0.18 (CPCV~0); HARSH negative. winsor likely over-conservative ->
true deployable Sharpe ~0.4-0.6, not the 0.76 headline. Real but modest edge; recent
years strongest. Deploy needs signal-robustness refinement + diversifying 2nd edge.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 16:07:31 +02:00
jgrusewski
7495eaa286 feat(surfer): point-in-time survivorship test PASSED — momentum is real
Built survivorship-free universe: 135 perps incl 28 known-dead (LUNA/SRM/MATIC...),
universe rebuilt daily as top-K by trailing dollar-volume (dead coins in while trading,
drop out after crash), no lookahead. mom_20 TOPK=30: full +0.76, IS+0.85/OOS+0.56,
CPCV-med +0.73, DSR 0.86, POSITIVE EVERY YEAR 2020-2026 incl 2022 +0.72. Counterintuitive:
survivorship was HIDING the edge (survivor-only 2022 -0.40 -> PIT +0.72) because momentum
shorts the dying coins and profits from crashes. Decisive survivorship confirmation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 16:04:19 +02:00
jgrusewski
0d6d58428d feat(surfer): momentum robustness battery — crypto XS momentum is a robust real edge
Lookback sweep smooth+positive 5-120d (peak mom_20 DSR 0.95); coin-bootstrap 200
random half-universes frac>0=1.00, 5th-pct +0.34 (survivorship-proxy passed);
vol-scaled approx raw; survives 2x cost; IS+0.89/OOS+0.88. Crypto cross-sectional
momentum (>=45-coin breadth) passes every test that killed carry. One weakness:
negative 2019/2022 (momentum-crash) -> regime overlay. Develop-grade strongly met.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 15:18:59 +02:00
jgrusewski
0f1a04d16d fix(surfer): universe-robustness exposes carry as cherry-picked; momentum is robust
Broadened crypto universe 28 majors -> programmatic top-80 (64 usable), added
funding-coverage/history filter. Carry COLLAPSED (+0.82->-0.03) and stayed dead even
on broad-but-clean subset (fundcov>0.9, 45 coins): carry was cherry-picked to majors,
not robust. Cross-sectional MOMENTUM is the robust edge: XS_mom_30 IS+0.89/OOS+0.88,
CPCV-med +0.75, DSR 0.75, survives 2x cost, holds across 64- and 45-coin universes.
Momentum needs breadth (weak on 28 correlated majors). Same regime failure (2019/2022).
Methodology win: broadening killed a cherry-picked claim before a build.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 15:16:56 +02:00
jgrusewski
af5e6d523b feat(surfer): broaden crypto sweep + regime/cost robustness — carry crash found
13 crypto signals, deflated by cumulative 30. Carry family all develop-grade, survive
2x cost; per-year reveals THE failure mode = 2022 deleveraging carry-crash (carry_7
-1.2 in 2022, +0.4..+2.1 other years). Momentum OOS-negative (decayed); chasing rising
funding strongly negative. Carry is the robust core; its one tail (carry crash) is
exactly what the ML regime overlay is built to manage -> the two threads unite.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 15:10:40 +02:00
jgrusewski
cb402ab81f feat(surfer): crypto carry+momentum sweep — FIRST real edge (funding carry)
Batch 2 of the deflated signal sweep: cross-sectional funding carry + momentum on
28 major Binance USDT perps (2019-26, funding baked into return, 10bp cost, deflated
by cumulative 25 trials). XS_carry_7 = Sharpe +0.82, IS+0.77/OOS+0.93 (consistent),
CPCV-median +0.78 (robust across 45 paths), DSR 0.71. First signal all session that is
positive + IS/OOS-consistent + CPCV-median-positive. Develop-grade met, not yet deploy
(DSR<0.95, 5th-pct<0). Caveats: survivorship (current majors), confirm point-in-time.
Forward-paginated funding fetch (fundcov ~1.0).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 15:05:20 +02:00
jgrusewski
1283812c54 feat(surfer): deflated signal-sweep engine + Batch 1 (futures factor zoo = null)
Broad signal search judged against the FULL search (Deflated Sharpe deflated by
N_trials) — the multiple-testing discipline that makes breadth safe. Batch 1: 17
signals (XS/TS momentum, reversal, low-vol, seasonality) on 22-futures daily ->
ZERO survivors (best DSR 0.29); CPCV+deflation correctly kill the lucky cells.
SEAS_overnight invalid on 24h futures (equity-cash anomaly). Next batches: crypto, COT.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 14:54:06 +02:00
jgrusewski
1cf38232e8 fix(surfer): full-16y MFT settle — regime-adaptive surfer significantly negative
Pulled 16y ES ohlcv-1m continuous (year-chunked, $20 credits, no 504) + light
to_ndarray loader (to_df OOM'd at 20GB). On 1.1M 5-min bars the 1.3y +0.50/t=0.67
top-5% hint collapsed: regime-adaptive top-5% = -0.52 ticks/trade t=-2.51
(significantly negative); ALL cells/signals/convictions significantly negative.
Decisive: no capturable intraday directional edge for crossing/non-colocated setup.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 12:15:37 +02:00
jgrusewski
fa700c7b5f feat(surfer): MFT regime-adaptive quality test (intraday, GPU, 4GB-safe)
Tests the actual surfer thesis: intraday 5-min decisions/15-min holds, flat-by-close,
regime-adaptive (ride trend / fade range / stand aside), quality-over-quantity
(top-conviction only). Clean ES OHLCV-1m -> 5-min bars, per-trade net edge in ticks,
IS/OOS + t-stat. Result: regime-adaptive top-5% = +0.50 ticks/trade OOS, sign-consistent,
but t=0.67 (not significant); all else significantly negative. Adaptive+selective is the
only non-losing structure (thesis directionally right, within noise). 4GB-safe: cumsum
rolling-std (no unfold) + alloc cap.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 10:08:16 +02:00
jgrusewski
2bb2b50fd1 fix(surfer): correct roll handling (held-contract return stitching) + cost stress + dual gates
The +0.32 'edge' was a roll-zeroing artifact (discarded held-contract roll-day P&L).
Correct roll stitching + 2bp cost: floor Sharpe ~0.05 (sign) / 0.06 (cont) / 0.14
(cont+band), all NEGATIVE at 2x cost. Marginal, implementation-dominated edge.
Adds FLOOR_SIGNAL mode (sign|cont), no-trade band, 2x-cost stress, develop/deploy gates.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 01:43:06 +02:00
jgrusewski
8df1c7eea2 feat(surfer): Phase 0 GPU floor + validation harness — first real (weak) OOS trend edge
PyTorch-GPU diversified-trend floor (TSMOM 1/3/12mo + inverse-vol + 10% vol-target)
+ CPCV/Deflated-Sharpe validation, over 22 CME futures x 19.7y (Databento GLBX
ohlcv-1d via budget-capped fetcher, ~$32 credits). Verdict: Sharpe +0.32, CPCV
median +0.30, IS +0.36/OOS +0.08 (sign-consistent), Deflated Sharpe ~0.92 at honest
n_trials. Real but weak edge; fails deploy-grade gates (correct), passes edge-exists.
Includes roll-neutralization fix (max-vol outright + zero roll-day returns) that
eliminated 988%/day continuous-contract artifacts (RB vol 550%->31%). Plus
install_torch_gpu.sh. Data gitignored.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 01:31:37 +02:00