Files
foxhunt/scripts/surfer/fetch_equities.py
jgrusewski 2065c98c25 feat(surfer): small/mid-cap equity factor gate — no factor survives small-cap cost
DBEQ.BASIC all US equities daily 2023-2026 ($67.85, 17305 instruments). Small/mid-cap band
(excl top-50 mega, $2M floor, ~576 names/day), weekly rebal+smooth, illiquidity-scaled cost
(30-150bp). NO factor survives net: momentum negative even gross (2023-24 momentum-crash);
reversal gross +0.15 eaten by cost -> NET -0.86 (Amihud paradox); low-vol best but NET ~0
(DSR 0.03). Didn't even charge short-borrow. Confirms Databento tradeable universe (equities+
futures) is efficient+cost-walled -> no retail edge. Honest tension: cheap-to-trade markets too
efficient; the inefficient market with real edge (crypto) is the disliked one. Caveat: 3.2y only.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 20:24:43 +02:00

34 lines
975 B
Python

#!/usr/bin/env python3
"""Fetch all US equities daily OHLCV (DBEQ.BASIC) for the small/mid-cap factor test.
Budget-capped, get_cost-gated. Saves DBN to data/surfer/dbeq_ohlcv1d.dbn.
"""
import os
import sys
import databento as db
CAP = 70.0
DS, SCH = "DBEQ.BASIC", "ohlcv-1d"
START, END = "2023-03-28", "2026-06-05"
OUT = "data/surfer/dbeq_ohlcv1d.dbn"
def main():
c = db.Historical(os.environ["DATABENTO_API_KEY"])
if os.path.exists(OUT):
print("already downloaded"); return 0
cost = c.metadata.get_cost(dataset=DS, symbols=["ALL_SYMBOLS"], schema=SCH, start=START, end=END)
print(f"get_cost=${cost:.2f} cap=${CAP:.2f}")
if cost > CAP:
print("ABORT over cap"); return 1
data = c.timeseries.get_range(dataset=DS, symbols=["ALL_SYMBOLS"], schema=SCH, start=START, end=END)
os.makedirs("data/surfer", exist_ok=True)
data.to_file(OUT)
print(f"saved {OUT}")
return 0
if __name__ == "__main__":
sys.exit(main())