1222 Commits

Author SHA1 Message Date
jgrusewski
cb758c448f docs: ADR 0003 — research/trader split over SSOT feature-warehouse (architecture + port roadmap + conservation principle)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 02:02:04 +02:00
jgrusewski
7c4ab9b156 feat: SSOT feature-warehouse foundation + first feature TDD-ported (microprice)
Layered SSOT data architecture: domain/marketdata (silver — Mbp10Snapshot/BidAskLevel, faithful port of
Rust data contracts incl. the load-bearing 1e-9 price scaling) -> domain/features (gold — MicropriceFeatures
Block N+V, a FAITHFUL port of Rust ml-features/microprice.rs, matching its test oracle bit-for-bit incl.
f64::signum(0)=1.0) -> ports/warehouse.FeatureStore (point-in-time) -> adapters/warehouse DuckDbFeatureStore
(as_of = no lookahead) -> application/feature_pipeline (silver->gold transform, causal). 11 new tests
(8 microprice vs Rust oracle + pipeline/as-of). 31/31 total.

Verified via omnisearch: the Rust formula is the weighted mid-price (commonly mis-called 'Stoikov
microprice'); Stoikov's TRUE micro-price (Quant Finance 2018) is a recursive Markov-chain estimator,
empirically a better future-price predictor — to be ADDED as a separate feature (not a silent oracle
divergence), so the gauntlet A/Bs them. Conservation-of-money ('every cent accounted for, follow the
money') adopted as a first-class principle: a warehouse/trading invariant AND the lens that prioritizes
the order-flow features (OFI/VPIN/book-deltas/Kyle-λ).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 02:01:04 +02:00
jgrusewski
7bba744f06 feat: empirical window prior in agent hunter + validated factory with gpt-4.1 (OpenAI swap)
Local test with a capable model: no Claude API key in env, but OPENAI_API_KEY is set -> config-swap to
gpt-4.1 (FXHNT_AGENT_PROVIDER=openai), no architecture change (LLM-agnostic design pays off). gpt-4.1
reasons well (economic market-strategy fit, broad coverage) but first proposed SHORT trend windows that
whipsaw -> 0 specialists. Added the empirical lookback prior (trend ~100-250, mean_reversion ~10-40) to
the hunter prompt; re-run found 2 quality specialists from 6 candidates (mean_reversion GC trend_down|
high_vol, trend(120) NQ trend_up|low_vol) at global trials 6 not 18 — the discipline win of agentic
search with a capable model. 22/22 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 01:38:58 +02:00
jgrusewski
0ca2e5284a feat: LLM-driven hunters (fxhnt 2) — agents direct their own search
application/agent/hunter.AgentHunter: an LLM proposes the next batch of (market, strategy, window)
candidates, reasoning about strategy-market fit and the gaps the current population leaves — clamped to
the real universe + known strategies so hallucinations can't enter. FactoryRuntime now takes a
territory_provider (callable): fixed grids OR the agent's per-cycle proposals. CLI run --hunters agent.
Verified on local qwen2.5:3b: proposed 3 FOCUSED candidates (vs 18 grid) -> global trials 3 not 18 (the
discipline win of agentic search: lower multiple-testing burden) -> found buy_hold on GC in
trend_down|high_vol (gold flight-to-safety in panic). fxhnt 1+2 complete: [hunt] 6 new candidates → 1 specialists | global hypotheses 9 | by regime {'trend_down|high_vol': 1}
[book] as of 2026-06-08 | 0 active, 2 dormant | weights (flat)
on cron = the standing agentic force that constantly seeks regime-specialists and runs them side by side.
22/22 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 01:20:53 +02:00
jgrusewski
c41dc088e9 feat: continuous factory runtime (fxhnt 1) — the standing 'constantly seeking' force
application/runtime.FactoryRuntime.cycle: hunt new regime-specialists (fleet) then assemble today's
regime-adaptive book (regime execution) in one pass. CLI run (single cycle = cron-friendly, like the
foxhunt paper trackers; --interval-minutes for a foreground daemon). Distinct-hypothesis DEDUP added to
the store (SeenCandidateRow + is_seen/mark_seen) and wired into the fleet, so re-running across cycles
does NOT re-inflate the global trial count or re-do work — honest multiple-testing accounting over time.
Verified: cycle 1 = 18 new candidates / global 18; cycle 2 = 0 new / global stays 18. 22/22 tests.
(Schedule [hunt] 0 new candidates → 0 specialists | global hypotheses 18 | by regime {}
[book] as of 2026-06-08 | 0 active, 3 dormant | weights (flat) via cron for the always-on force. Next: LLM-driven hunters = fxhnt 2.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 01:17:37 +02:00
jgrusewski
ae9f9965ad feat: regime-conditional execution (factory layer 5) — the loop closes
application/regime_execution.RegimeConditionalExecutor: reads specialists from the store, detects TODAY's
regime per market, activates only those whose regime is live, nets their latest positions into per-market
target weights (gross-capped). CLI book. The multi-strategy book is now regime-adaptive — trend specialists
run in calm uptrends, mean-reversion in panic, automatically in/out by regime. Live demo: as of 2026-06-08
all 3 discovered specialists are DORMANT (NQ in trend_up|HIGH_vol not its calm-trend regime; GC in calm
downtrend not its panic regime) -> book correctly FLAT. Discipline: don't trade out-of-regime. 21/21 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 21:40:47 +02:00
jgrusewski
16fdc43650 feat: fleet orchestrator (factory layer 3) — the standing research force
application/fleet: Territory (a hunter's mandate: family x universe x param grid) + FleetOrchestrator —
searches each territory, evaluates every candidate REGIME-conditionally (layer 1), writes the regime-
specialists it finds to the lifecycle store (layer 2) with honest global trial accounting. CLI hunt.
Hunters sequential now (local GPU serializes), interface ready for concurrent/agent-proposed later.
Live on cached futures: 2 hunters, 18 candidates -> 3 specialists (NQ trend in trend_up|low_vol;
GC mean_reversion in trend_down|high_vol — a non-trend specialist the full sample would miss). 20/20 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 19:28:08 +02:00
jgrusewski
adbc47b40b feat: strategy-lifecycle store (factory layer 2) — the persistent backbone
domain/factory (StrategyRecord: regime-tagged specialist + lifecycle status discovered/forward/deployed/
retired; RegimeFit; deterministic strategy_id) | ports.StrategyStore | adapters SqlStrategyStore (upsert/
get/list-by-status&regime/set_status + GLOBAL trial counter for honesty-at-scale) + StrategyRow/FactoryMetaRow
ORM | CLI factory. The fleet writes discovered specialists here; execution reads DEPLOYED ones; the global
hypothesis count lives here. Demo: NQ+6E trend specialist (trend_up|low_vol DSR 0.984) seeded + forward-tracking.
19/19 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 19:24:07 +02:00
jgrusewski
72cdbc9eec feat: regime layer — the foundation of the agentic strategy factory (ADR 0002)
ADR 0002 designs the real vision: a standing fleet of agents continuously breeding regime-specialist
strategies into an adaptively-allocated book, with multiple-testing-at-fleet-scale as the constraint the
whole architecture is built around (structured search + regime-conditioning + forward-validation +
global-N accounting = the moat). domain/regime: VolTrendClassifier (causal trend x volatility state) +
evaluate_by_regime (judge a strategy WITHIN each regime). Demo on real NQ: trend(200) is MARGINAL on the
full sample but a clear OOS-confirmed SPECIALIST in trend_up|low_vol (DSR 0.984, OOS +0.93), correctly
rejected in range/high-vol regimes — the architecture's thesis proven. 16/16 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 19:20:23 +02:00
jgrusewski
661b56db79 feat: honest trial-accounting — agent deflates by its own effective search breadth
StructuredResearchAgent now collects every book it tests (dedup'd, returns kept via BookEvaluation.
is/oos_returns) and at the end re-judges ALL of them at the EFFECTIVE number of independent books
searched (effective_trials over their return streams) — the same multiple-testing discipline the grid
discovery uses, applied to the agent's OWN search. A book wins only if it clears the full-search bar.
Verified on local qwen2.5:3b: tested 4 books (~4.0 effective), NQ+6E trend SURVIVES at n_trials=4
(DSR 0.971) while weaker combos correctly reject. LLM proposes; gauntlet + multiple-testing dispose.
14/14 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 16:55:07 +02:00
jgrusewski
2f6df0ffda style(cli): move misplaced import to the top of the module (E402)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 16:40:34 +02:00
jgrusewski
25036e12c9 feat: structured proposal-loop agent — robust on local models, finds first survivor
application/agent/structured.StructuredResearchAgent: a LangGraph StateGraph that constrains the LLM to
one job (emit a structured ProposalBatch of books) while the graph runs each through the portfolio
gauntlet deterministically and loops. Robust where free-ReAct failed on the weak 3B model. CLI agent
--mode structured (default) | react. VERIFIED on local qwen2.5:3b: reliably drove the loop (6.5s) and
found the platform's FIRST passing book -> trend on 6E+NQ (EUR/USD x Nasdaq, uncorrelated): Sharpe 0.82,
OOS 0.86, maxDD -10%, DSR 0.971 PASS; correctly rejected weak combos (ZN+CL DSR 0.68). LLM proposes,
gauntlet disposes. 14/14 tests. (Caveat: n_book_trials fixed at 10 — should track the agent's full
search breadth for complete trial-accounting honesty.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 16:39:30 +02:00
jgrusewski
e628daba52 feat: agentic research layer — LangGraph agent + LLM port + compute tools
application/agent: ResearchAgent (LangGraph create_react_agent) + build_tools (list_strategies,
evaluate_candidate, evaluate_book wrapping the services) | adapters/llm.build_chat_model (config-
swappable: local Ollama now, cluster/API later) | AgentSettings | CLI agent. The LLM proposes; the
deterministic gauntlet inside the tools disposes. Verified: tools work + the model emits tool-calls
(architecture correct). Local qwen2.5:3b is too weak for the reliable multi-step loop (model bottleneck,
swappable via config). Also fixed flaky test seeds (hash()->deterministic). 14/14 tests. Bonus: tools
surfaced GC+NQ+ES trend book = Sharpe 1.06 OOS 1.47 DSR 0.949.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 16:22:59 +02:00
jgrusewski
84b2708af2 fix(test): book-eval test passed a BacktestResult to compute_stats; use .stats.sharpe (14/14 green)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 16:07:46 +02:00
jgrusewski
b4ecc4898a feat: portfolio-level validation — judge the BOOK, not the individual sleeves
domain/portfolio.combine_returns (date-aligned weighted sum of sleeve returns) + application
PortfolioEvaluationService (backtest each sleeve, combine, gauntlet on the book) + CLI validate-book
+ DRY _data_provider helper. The unit of alpha is the diversified book: live futures-trend book
(6 markets) = Sharpe 0.81, OOS 0.91, maxDD -9%, DSR 0.947 (n=1) vs individual sleeves' DSR 0.02-0.04
-> the book is vastly more robust, borderline-significant (exactly what a real CTA book is). 14/14 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 16:06:56 +02:00
jgrusewski
5f5979ec68 feat: effective-N deflation correction — correlation-adjusted trial count
Blind grid deflated by RAW N over-penalises: trend-150/200/250 on the same asset + buy-hold on
correlated markets are NOT independent tests. Added gauntlet.effective_trials (design-effect:
N_eff = N/(1+(N-1)*rho_bar), rho_bar = mean off-diagonal correlation of candidate returns). Discovery
now deflates by the effective N. Live: 48 futures candidates -> 11.2 effective trials (bar ~1.7 -> ~1.0
Sharpe). Genuine edges (NQ/GC trend, OOS 1.2-1.35) now near-but-still-below the individual bar -> the
PORTFOLIO is the right unit (next). 12/12 tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 16:02:50 +02:00
jgrusewski
747f98a404 feat: ingest command + databento settings wired — decouple slow ingestion from fast search
Databento historical fetches are slow (~30-90s/symbol for continuous futures over 8y) — so ingestion
must be decoupled from search: fetch+cache a universe once, search it many times fast. Added fxhnt
ingest (pre-fetch into the analytical store), --data-source on discover (yahoo|databento), and the
missing Settings.databento field. This is the data-layer-value lesson made concrete (the user intuited
it). 11/11 tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 14:48:52 +02:00
jgrusewski
b9bc157683 feat: discovery search — the agentic search army (deflated over the FULL search)
application/discovery: CandidateGenerator (seam for an LLM agent later) + GridGenerator + DiscoveryService.
The discipline that makes it honest at scale: every candidate judged with n_trials = total searched and
sr_variance = variance across ALL candidates' Sharpes -> the deflated-Sharpe bar accounts for the whole
search. Shared run_id (domain), per_period_sharpe helper. CLI . Verified on real data: 32
candidates -> 0 survivors (CORRECT: trend-on-SPY passes at n_trials=1 but not at 32 -> multiple-testing
correction working, the anti-false-positive core proven at scale). Perf: persistent DuckDB connection
+ only survivors persist timeseries (was connect-per-op, 87s I/O). 11/11 tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 14:27:24 +02:00
jgrusewski
f79b4e7173 feat: breadth layer — more strategies (mean_reversion, buy_hold) + Databento adapter
3 strategy templates now (trend, mean_reversion contrarian z-score, buy_hold baseline) — gauntlet
correctly rejects mean_reversion on trending SPY (strategy-market fit working). DatabentoDataProvider
behind DataProvider port: survivorship-free daily bars across futures/equities (GLBX 2010+, equities
2018+), with a COST GUARD (estimate via metadata + refuse above cap before download — the $3.5k lesson
as code). Verified live: guard blocks when cap too low; ES.c.0 fetch ok (620 bars @ $4812). data extra.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 14:12:26 +02:00
jgrusewski
fb642317f9 feat: IBKR broker adapter + execute CLI — full pipeline runs end-to-end
adapters/broker/ibkr (IbkrBroker implements Broker via ib_async: managed connection, paper detection
via DU* prefix, whole-share orders, gross-position-value for the data-consistency gate) | IbkrSettings
config | domain/portfolio.book_from_runs (survivor library -> executable Book) | CLI
(composition root wiring IBKR + ExecutionService, dry-run default). Verified live against IBKR paper
(NLV $1.04M): research -> gauntlet -> survivors -> book -> netting -> plan. 10/10 tests still green.
ib_async is an optional extra (ibkr).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 14:05:31 +02:00
jgrusewski
7fb14c47a0 feat: multi-strategy execution layer (domain netting + Broker port + ExecutionService)
Layer-by-layer: domain/portfolio (Book + StrategyAllocation + compute_target_weights — nets many
sleeves, incl. same market, into per-market targets capped at gross leverage) | ports/broker (Broker
contract + Order/AccountState DTOs) | application/execution (ExecutionService: net → reconcile vs
broker → whole-share orders with entry-floor/hysteresis, behind the hard-won gates: paper-guard,
data-consistency/conservative-NLV, leverage-cap) | config ExecutionSettings. Tested with fake
data+broker (no network/IBKR): netting, leverage cap, buy-from-flat, in-band no-op, live-block,
inconsistent-data-block. 10/10 tests green. IBKR adapter (behind Broker port) is the next step.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 13:52:32 +02:00
jgrusewski
58bbb4520c feat: fxhnt foundation — hexagonal architecture + proven gauntlet + vertical slice
Enterprise clean-rebuild (no foxhunt code). Hexagonal/ports-and-adapters: pure domain (gauntlet
math, strategies, backtest, models) | ports (DataProvider, repositories) | adapters (Yahoo data,
SQLAlchemy operational [Postgres/SQLite], DuckDB analytical) | application (ResearchService, DI) |
CLI composition root. Gauntlet-first: Deflated Sharpe (Bailey-LdP) built + falsification-tested
(kills best-of-N-on-noise, keeps real premium). Full vertical slice runs end-to-end on real data:
data -> strategy(trend) -> backtest(net of costs) -> IS/OOS gauntlet -> persistence. 4/4 tests green.
Postgres+DuckDB split, pydantic contracts, typed, DRY via one-contract-per-port. ADR + README.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 12:21:47 +02:00