docs(exec): design spec — live crypto execution layer (Binance USDM perps, ccxt)
Research-grounded (rate limits 2400wt/1200ord per min + 418 ban risk, clientOrderId idempotency, LOT/PRICE/ MIN_NOTIONAL filters, maker-first+POV slicing saves 50-90bps, ccxt testnet). Hexagonal: crypto_exchange port + binance_ccxt adapter w/ rate-limit governor, pure smart_router (maker->IOC, slicing, slippage abort), full risk layer (gross/per-symbol/leverage caps + edge-decay×Kelly scaler + kill-switch + drawdown CB), reconcile + TCA, testnet-first w/ allow_live gate. 5 milestones. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
@@ -0,0 +1,153 @@
|
||||
# Live crypto execution layer (Binance USDⓈ-M perps) — design
|
||||
|
||||
**Date:** 2026-06-14
|
||||
**Status:** Design approved (research-grounded), ready for implementation plan
|
||||
**Parent:** the deployable combined book — crypto X-sectional momentum sleeve (83%) needs live execution
|
||||
|
||||
## Problem
|
||||
|
||||
The cockpit forward-validates the combined book on paper. To eventually trade the crypto alpha (dollar-neutral
|
||||
cross-sectional momentum across ~135 Binance USDⓈ-M perps, daily rebalance) we need a production-grade
|
||||
execution layer: turn target weights into real fills on a live venue, with the safety + cost discipline a
|
||||
systematic desk requires. **Testnet first; real capital only after the Phase-1 forward gate clears.**
|
||||
|
||||
## Goals
|
||||
|
||||
- A hexagonal, fxhnt-native execution layer: a second `Broker`/exchange adapter (Binance USDM via **ccxt**)
|
||||
plus a pure, liquidity-aware **smart order router**, full **pre/post-trade risk controls**, and a
|
||||
reconciliation + transaction-cost-analysis (TCA) loop.
|
||||
- **Testnet-first**: identical code path; live requires an explicit `allow_live` gate.
|
||||
- Best realistic execution quality for a daily-rebalanced multi-perp book (maker-first + taker fallback +
|
||||
liquidity slicing), with measured fill quality.
|
||||
- Honest cost + risk: rate-limit-safe, idempotent, capacity-aware, kill-switchable.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Not HFT/microsecond. The book rebalances once daily; fills over minutes are fine.
|
||||
- No new alpha — execution only. The signal is the existing `xs_momentum_book`.
|
||||
- Not coupling to foxhunt's Rust risk controllers — fxhnt gets a **Python** edge-decay×Kelly scaler.
|
||||
- Spot, options, cross-exchange routing — out (single venue v1; ccxt keeps multi-exchange open).
|
||||
|
||||
## Research-grounded production concerns (do NOT underestimate)
|
||||
|
||||
Verified via research (Binance USDⓈ-M docs, ccxt, execution-algo literature):
|
||||
|
||||
1. **Rate limits.** Binance USDM: **2,400 request-weight/min per IP** and **1,200 orders/min per account**
|
||||
(plus short-window order caps). Exceeding → HTTP 429; repeated/no-backoff → **418 IP ban**. With 135
|
||||
symbols × (limit place → cancel → IOC cross) the order count is the binding constraint. **Mandatory:** a
|
||||
rate-limit governor — ccxt `enableRateLimit`, a local token bucket, reading `X-MBX-USED-WEIGHT-*` /
|
||||
`X-MBX-ORDER-COUNT-*` response headers, exponential backoff on 429, hard stop before 418.
|
||||
2. **Idempotency.** Every order carries a deterministic `clientOrderId` (e.g. `fxhnt-<rebalance_id>-<symbol>-<leg>`)
|
||||
so a retry after a timeout never double-places. Reconcile by clientOrderId.
|
||||
3. **Exchange filters.** Round quantity to `LOT_SIZE.stepSize`, price to `PRICE_FILTER.tickSize`, and skip
|
||||
anything below `MIN_NOTIONAL` (~5 USDT) and `MARKET_LOT_SIZE`. Cache `exchange_info` per rebalance.
|
||||
4. **Position/margin setup.** Explicit **one-way** position mode; `reduceOnly` on reducing/closing legs;
|
||||
set leverage + margin mode (isolated default) per symbol once. Funding settles every 8h (track in PnL).
|
||||
5. **Execution cost is real.** Pro desks cut 50–90 bps with maker-first + slicing vs pure taker. Measure it:
|
||||
a **TCA** record per rebalance (fill price vs arrival/decision mid = implementation shortfall) so we know
|
||||
whether execution is eating the alpha — especially on thin small-caps.
|
||||
6. **Testnet specifics.** ccxt `set_sandbox_mode(True)` + **separate** testnet API keys
|
||||
(`testnet.binancefuture.com`). Live = same adapter, sandbox off, real keys, `allow_live=true`.
|
||||
|
||||
## Architecture (hexagonal, fxhnt)
|
||||
|
||||
```
|
||||
ports/crypto_exchange.py Perp exchange contract (Protocol) + DTOs:
|
||||
ExchangeInfo(stepSize,tickSize,minNotional per symbol) · OrderBook(bids/asks)
|
||||
· PerpPosition(symbol, signed_qty, entry, leverage) · AccountState(nlv,...)
|
||||
· ChildOrder(symbol, side, qty, type, price?, reduce_only, post_only, client_id)
|
||||
· Fill(client_id, symbol, side, qty, price, fee, ts)
|
||||
methods: exchange_info() · order_book(symbol) · positions() · account_state()
|
||||
· set_leverage/set_position_mode · place(ChildOrder)->Fill|status · cancel(id)
|
||||
domain/execution/
|
||||
rounding.py PURE: round_qty(stepSize), round_price(tickSize), meets_min_notional()
|
||||
smart_router.py PURE: (target deltas + market snapshot) -> [ChildOrder] schedule
|
||||
maker-first limit@touch -> IOC cross fallback; POV slice vs depth; slippage-abort
|
||||
risk_gates.py PURE: pre-trade checks (allow_live, gross cap, per-symbol cap, leverage cap, kill-switch)
|
||||
+ dynamic scaler: edge-decay(θ, Page-Hinkley) × Kelly-fraction × capacity caps
|
||||
capacity.py PURE: ADV-based per-symbol max notional (don't be >X% of a thin market)
|
||||
tca.py PURE: implementation-shortfall per fill (vs arrival mid) -> cost bps
|
||||
application/
|
||||
crypto_execution.py ORCHESTRATES: book -> target$ -> scaler -> deltas -> gates -> router ->
|
||||
rate-limited place loop -> collect fills -> reconcile -> persist (cockpit DB)
|
||||
reconciliation.py intended vs exchange positions (by clientOrderId + net) -> drift -> log / halt
|
||||
adapters/exchange/binance_ccxt.py ccxt binanceusdm adapter (testnet via sandbox); rate-limit governor;
|
||||
header-aware backoff; filter cache; idempotent clientOrderId
|
||||
cli.py: crypto-rebalance (--dry-run | --execute) · crypto-flatten (kill-switch) · crypto-positions
|
||||
config: BinanceSettings (api_key/secret env, testnet=true, leverage, gross_cap, per_symbol_cap,
|
||||
participation_rate, max_slippage_bps, allow_live=false, kill_switch=false)
|
||||
```
|
||||
|
||||
The **Broker** boundary stays the contract; this adds a richer perp-aware `crypto_exchange` port alongside it
|
||||
(perps need exchange_info, order book, leverage, signed positions, reduce-only — beyond the equities `Broker`).
|
||||
|
||||
## Data flow
|
||||
|
||||
```
|
||||
xs_momentum_book -> target weights (dollar-neutral, ~135 perps)
|
||||
-> risk scaler: gross *= clamp(edge_decay θ) × Kelly_fraction ; per-symbol *= capacity cap
|
||||
-> target $ per symbol -> deltas vs current exchange positions (signed)
|
||||
-> pre-trade gates (allow_live · kill_switch · gross cap · per-symbol cap · leverage · session drawdown CB)
|
||||
-> smart_router per symbol:
|
||||
skip if |Δ| < max(min_notional, hysteresis×NLV)
|
||||
size = min(target, participation_rate×ADV, depth_cap×book_depth) ; slice if target exceeds
|
||||
child = post-only limit @ touch (maker) ; wait T ; if unfilled -> cancel -> IOC cross (taker)
|
||||
abort symbol if fill price drifts > max_slippage_bps vs arrival mid ; reduce_only on closes
|
||||
-> rate-limited place loop (token bucket + X-MBX headers + 429 backoff; deterministic clientOrderId)
|
||||
-> collect Fills -> TCA (shortfall bps) -> reconciliation (intended vs actual; halt on drift>tol)
|
||||
-> persist fills + realized + TCA to the cockpit DB (dashboard shows live execution + cost)
|
||||
```
|
||||
|
||||
## Risk controls (full)
|
||||
|
||||
- **Pre-trade gates** (`risk_gates`): `allow_live` (default false — testnet/dry-run otherwise) · kill-switch
|
||||
flag (file/DB; `crypto-flatten` closes all reduce-only) · gross-exposure cap (×NLV) · per-symbol notional
|
||||
cap · leverage cap · session-drawdown circuit breaker (halt new opens if intraday PnL < −X%).
|
||||
- **Dynamic sizing**: target gross scaled by **edge-decay θ** (Page-Hinkley on realized per-trade PnL — a
|
||||
Python port of the foxhunt concept) × **Kelly fraction** (capped, e.g. ≤0.25) × **capacity caps** (ADV).
|
||||
Edge decays → gross shrinks automatically; edge dies → θ→0 → flat.
|
||||
- **Post-trade**: reconciliation compares intended vs exchange positions (and unfilled clientOrderIds); drift
|
||||
beyond tolerance → log + halt (no silent divergence). TCA flags execution eating > N bps of expected edge.
|
||||
|
||||
## Error handling
|
||||
|
||||
- 429 → exponential backoff honoring `Retry-After`; approach the 418 threshold → hard stop the batch.
|
||||
- Network/timeout on place → look up by `clientOrderId` before any retry (never double-place).
|
||||
- Order rejected (filter/margin) → log, skip that symbol, continue the batch (one bad symbol ≠ abort).
|
||||
- Partial fill → router books the filled qty, re-sizes the remainder for the IOC leg.
|
||||
- Any reconciliation drift > tolerance or kill-switch set → halt + alert; never "best-effort" past a risk gate.
|
||||
|
||||
## Testnet & safety
|
||||
|
||||
Everything runs first on **Binance futures testnet** (ccxt `set_sandbox_mode(True)`, separate testnet keys).
|
||||
`crypto-rebalance --dry-run` prints the full child-order schedule + TCA estimate with **zero** orders. Live
|
||||
requires `testnet=false` AND `allow_live=true` AND kill-switch clear — three independent conditions. Secrets
|
||||
are k8s secrets, never logged.
|
||||
|
||||
## Testing (TDD)
|
||||
|
||||
- Pure units offline with fixtures: `rounding` (step/tick/minNotional edge cases), `smart_router`
|
||||
(maker→IOC, slicing vs depth, slippage abort, partial fill, dust skip), `risk_gates` (every gate + the
|
||||
edge-decay×Kelly×capacity scaler), `capacity` (ADV caps), `tca` (shortfall math), `reconciliation`
|
||||
(drift detection). No network.
|
||||
- `binance_ccxt` adapter against **testnet** behind a pytest marker (real testnet fills): place/cancel/IOC,
|
||||
filter rounding, leverage/position-mode, rate-limit header parsing.
|
||||
- End-to-end on testnet: rebalance a tiny book → fills → reconcile → cockpit DB rows.
|
||||
|
||||
## Milestones
|
||||
|
||||
- **M1 — Exchange port + ccxt adapter:** `crypto_exchange` port + DTOs; `binance_ccxt` (exchange_info,
|
||||
order_book, positions, account_state, set_leverage/position_mode, place/cancel) with the **rate-limit
|
||||
governor**, header-aware backoff, filter cache, idempotent clientOrderId; `set_sandbox_mode` testnet wiring.
|
||||
- **M2 — Smart router (pure):** `rounding` + `smart_router` (maker-first → IOC, POV slicing, slippage abort).
|
||||
- **M3 — Risk layer (pure):** `risk_gates` + `capacity` + edge-decay(θ)×Kelly scaler + kill-switch +
|
||||
session-drawdown CB.
|
||||
- **M4 — Orchestrator + observability:** `crypto_execution` + `reconciliation` + `tca` + cockpit-DB
|
||||
persistence of fills/realized/TCA; CLI `crypto-rebalance`/`crypto-flatten`/`crypto-positions`.
|
||||
- **M5 — Testnet end-to-end:** real testnet fills, reconciliation, TCA; validate against a tiny live book.
|
||||
|
||||
## Open follow-ups (post-v1)
|
||||
|
||||
- Limit-chase/iceberg refinements; cross-exchange routing (ccxt makes Bybit additive).
|
||||
- Wire the cockpit to show live execution + TCA alongside the paper-forward track.
|
||||
- Capacity/sizing calibration from realized testnet slippage before any real-capital sizing.
|
||||
Reference in New Issue
Block a user