diff --git a/crates/backtesting/README.md b/crates/backtesting/README.md index 8f343d714..d6b8505cb 100644 --- a/crates/backtesting/README.md +++ b/crates/backtesting/README.md @@ -1,48 +1,24 @@ -# Backtesting Crate +# backtesting -## Overview +Strategy backtesting engine for simulating trading strategies against historical market data. -The `backtesting` crate provides a robust and configurable engine for simulating trading strategies against historical market data. It enables quantitative analysts and developers to evaluate strategy performance, optimize parameters, and validate hypotheses before live deployment. +## Key Types + +- `Backtester` — main backtesting engine +- `BacktestConfig` — simulation configuration (time range, instruments, slippage, commissions) +- `BacktestResults` — performance metrics (Sharpe, max drawdown, alpha, beta, Sortino) ## Features -* **Historical Data Replay:** Efficiently replays market data from Parquet files, supporting various data granularities (ticks, order book snapshots, candles). -* **Comprehensive Performance Metrics:** Calculates key performance indicators such as Sharpe Ratio, Maximum Drawdown, Alpha, Beta, Sortino Ratio, and more. -* **Realistic Slippage Modeling:** Configurable slippage models (e.g., fixed, percentage, volume-based) to accurately reflect real-world execution costs. -* **Commission Modeling:** Supports various commission structures (e.g., fixed per trade, percentage of value, per share/contract) for accurate P&L calculation. -* **Detailed Trade Analytics:** Generates in-depth reports on individual trades, cumulative P&L, win/loss ratios, and trade duration analysis. -* **Pluggable Strategy Interface:** Defines a clear interface for users to implement and integrate their custom trading strategies seamlessly. +- Historical data replay from Parquet files (ticks, order book snapshots, candles) +- Configurable slippage models (fixed, percentage, volume-based) +- Commission modeling (fixed, percentage, per-contract) +- Pluggable strategy interface ## Usage ```rust use backtesting::{Backtester, BacktestConfig}; -use common::types::InstrumentId; -use std::path::PathBuf; - -let config = BacktestConfig { - start_time: "2023-01-01T00:00:00Z".parse().unwrap(), - end_time: "2023-01-02T00:00:00Z".parse().unwrap(), - data_path: PathBuf::from("./historical_data/"), - instruments: vec![InstrumentId::new("BTCUSD".to_string())], - // ... other configuration like slippage, commissions -}; - -// let mut backtester = Backtester::new(config); -// let strategy = MySimpleStrategy::new(); // Initialize your strategy -// backtester.run(&strategy).expect("Backtest failed"); - -// let results = backtester.get_results(); -// println!("Sharpe Ratio: {}", results.sharpe_ratio); -// println!("Max Drawdown: {}", results.max_drawdown); +let config = BacktestConfig { /* ... */ }; +let results = backtester.run(&strategy)?; ``` - -## Testing - -```bash -cargo test --package backtesting -``` - -## Documentation - -Full API documentation is available at [docs.rs/backtesting](https://docs.rs/backtesting). diff --git a/crates/common/README.md b/crates/common/README.md index 3adcd3d62..43bb2ff2f 100644 --- a/crates/common/README.md +++ b/crates/common/README.md @@ -1,42 +1,24 @@ -# Common Crate +# common -## Overview +Shared types, error handling, and utilities for the Foxhunt HFT ecosystem. -The `common` crate provides a foundational set of shared types and utilities essential for building high-frequency trading applications within the Foxhunt ecosystem. It encapsulates core data structures, error handling patterns, and common helpers used across various components. +## Key Types -## Features +- `FoxhuntError` — unified error enum with `Result` alias +- `ModelType` — canonical ML model enum (10 primary variants) +- `CircuitBreakerTrait` — async circuit breaker interface +- `TlsProtocolVersion`, `ClientIdentity` — TLS configuration types +- Market data types (`Tick`, `OrderBook`) and order types (`LimitOrder`, `MarketOrder`) -* **Market Data & Order Types:** Defines standardized structs for market data (e.g., `Tick`, `OrderBook`) and various order types (e.g., `LimitOrder`, `MarketOrder`). -* **High-Precision Time Utilities:** Offers utilities for working with nanosecond-resolution timestamps and duration calculations critical for HFT. -* **Robust Error Handling:** Implements a custom `FoxhuntError` enum and `Result` type for consistent and traceable error management across the system. -* **Data Validation Helpers:** Provides functions for validating common HFT parameters such as prices, quantities, and instrument IDs. -* **Serialization/Deserialization:** Includes helpers and derive macros for efficient data serialization (e.g., using `serde`) for inter-process communication or persistence. -* **Instrument & Asset Definitions:** Standardized types for defining trading instruments, assets, and exchanges. +## Modules + +- `model_types` — canonical `ModelType` enum re-exported by ml/ and model_loader/ +- `resilience` — `CircuitBreakerTrait` async trait +- `tls` — TLS types and configuration +- `questdb` — QuestDB client configuration ## Usage ```rust -use common::types::{Order, OrderSide, Price, Quantity, InstrumentId}; -use common::errors::FoxhuntError; - -fn create_limit_order(instrument: InstrumentId, price: Price, quantity: Quantity) -> Result { - if price.value() <= 0.0 || quantity.value() <= 0.0 { - return Err(FoxhuntError::ValidationError("Price and quantity must be positive".to_string())); - } - Ok(Order::new_limit(instrument, OrderSide::Buy, price, quantity)) -} - -let instrument = InstrumentId::new("BTCUSD".to_string()); -let order = create_limit_order(instrument, Price::new(60000.0), Quantity::new(0.5)); -println!("{:?}", order); +use common::{FoxhuntError, ModelType}; ``` - -## Testing - -```bash -cargo test --package common -``` - -## Documentation - -Comprehensive API documentation is available at [docs.rs/common](https://docs.rs/common). diff --git a/crates/config/README.md b/crates/config/README.md new file mode 100644 index 000000000..cf7174e1a --- /dev/null +++ b/crates/config/README.md @@ -0,0 +1,18 @@ +# config + +Configuration management for Foxhunt HFT trading system. + +## Key Types + +- `FoxhuntConfig` — top-level configuration struct +- `MlConfig` — ML model and training configuration +- `RiskConfig` — risk management parameters +- `DatabaseConfig` — PostgreSQL connection settings +- `StorageConfig` — S3/object storage configuration + +## Usage + +```rust +use config::FoxhuntConfig; +let config = FoxhuntConfig::load("config/foxhunt.toml")?; +``` diff --git a/crates/ctrader-openapi/README.md b/crates/ctrader-openapi/README.md new file mode 100644 index 000000000..71f057d2d --- /dev/null +++ b/crates/ctrader-openapi/README.md @@ -0,0 +1,14 @@ +# ctrader-openapi + +cTrader Open API client — Protobuf over TCP+TLS for order routing, account queries, and market data. + +## Key Types + +- `CtraderConnection` — TLS-wrapped TCP connection to cTrader +- `CtraderCodec` — Protobuf message framing +- `RateLimiter` — request rate limiting +- `AuthManager` — OAuth2 authentication flow + +## Usage + +Broker gateway integration for order routing via cTrader Open API protocol. diff --git a/crates/data/README.md b/crates/data/README.md index f0e421b7f..2dea0f723 100644 --- a/crates/data/README.md +++ b/crates/data/README.md @@ -4,39 +4,31 @@ Market data ingestion, broker integration, and feature extraction. ## Providers -- **Databento** -- Historical and real-time market data via DBN format -- **Benzinga** -- News and fundamentals feed +- **Databento** — historical and real-time market data via DBN format +- **Benzinga** — news and fundamentals feed ## Broker Integrations -- **IB TWS** -- Interactive Brokers TWS/Gateway socket connection -- **ICMarkets** -- FIX 4.4 protocol integration +- **IB TWS** — Interactive Brokers TWS/Gateway socket connection +- **ICMarkets** — FIX 4.4 protocol integration ## Key Modules -- **brokers** -- Broker adapters (IB TWS, ICMarkets FIX) -- **providers** -- Data provider clients (Databento, Benzinga) -- **dbn_uploader** -- DBN file upload and processing -- **parquet_persistence** -- Parquet read/write for tick and bar data -- **replay** -- Market data replay for backtesting -- **training_pipeline** -- Data preparation for ML model training -- **features** -- Technical indicator and feature computation -- **unified_feature_extractor** -- Normalized feature vectors across providers -- **validation** -- Data quality checks and schema validation +- `brokers` — broker adapters (IB TWS, ICMarkets FIX) +- `providers` — data provider clients (Databento, Benzinga) +- `parquet_persistence` — Parquet read/write for tick and bar data +- `replay` — market data replay for backtesting +- `training_pipeline` — data preparation for ML model training +- `features` — technical indicator and feature computation +- `validation` — data quality checks and schema validation ## Cargo Features -| Feature | Default | Description | -|---------------|---------|------------------------------------| -| `databento` | yes | Databento provider support | -| `benzinga` | yes | Benzinga provider support | -| `icmarkets` | yes | ICMarkets FIX 4.4 integration | -| `redis-cache` | no | Redis caching layer | -| `ib` | no | Interactive Brokers TWS adapter | -| `mock` | no | Mock providers for testing | - -## Testing - -```bash -SQLX_OFFLINE=true cargo test -p data --lib -``` +| Feature | Default | Description | +|---------------|---------|-------------------------------| +| `databento` | yes | Databento provider support | +| `benzinga` | yes | Benzinga provider support | +| `icmarkets` | yes | ICMarkets FIX 4.4 integration | +| `redis-cache` | no | Redis caching layer | +| `ib` | no | Interactive Brokers adapter | +| `mock` | no | Mock providers for testing | diff --git a/crates/database/README.md b/crates/database/README.md index 9fcc05dc3..973f8530b 100644 --- a/crates/database/README.md +++ b/crates/database/README.md @@ -1,50 +1,24 @@ -# Database Crate +# database -## Overview +PostgreSQL persistence layer for the Foxhunt HFT system. -The `database` crate manages the persistent storage layer for the Foxhunt HFT system, primarily utilizing PostgreSQL. It handles schema definitions, migrations, and provides utilities for storing and querying critical trading data, including time-series market data and audit logs. +## Key Types + +- `DatabasePool` — connection pool management +- Schema definitions for trading events, market data, and audit logs +- Migration management utilities ## Features -* **PostgreSQL Schema & Migrations:** Defines database schemas for trading events, market data, and user configurations, managed via an integrated migration system. -* **Event Streaming & Audit Log:** Provides interfaces for recording and querying all significant system events, ensuring a comprehensive audit trail for compliance and post-trade analysis. -* **Optimized Time-Series Storage:** Implements efficient storage and indexing strategies for high-volume, time-series market data. -* **Query Utilities:** Offers a set of helper functions and ORM-like abstractions for common data retrieval and manipulation tasks. -* **Connection Pooling:** Manages database connections efficiently using a connection pool to minimize overhead and improve throughput. -* **Data Archiving & Retention:** Includes mechanisms for managing data lifecycle, such as archiving old data or implementing retention policies. +- PostgreSQL schema and migrations +- Event streaming and audit log +- Optimized time-series storage and indexing +- Connection pooling for high-throughput access +- Data archiving and retention policies ## Usage ```rust -use database::models::{TradeEvent, NewTradeEvent}; use database::connection::establish_connection; -use common::types::{InstrumentId, Price, Quantity}; -use chrono::Utc; - -// This would typically come from a connection pool -let mut conn = establish_connection().expect("Failed to connect to database"); - -let new_trade = NewTradeEvent { - timestamp: Utc::now(), - instrument_id: InstrumentId::new("ETHUSD".to_string()), - price: Price::new(3000.50), - quantity: Quantity::new(1.2), - side: "BUY".to_string(), - // ... other fields -}; - -// Example: Insert a new trade event -// let inserted_trade = database::crud::create_trade_event(&mut conn, new_trade) -// .expect("Failed to insert trade event"); -// println!("Inserted trade: {:?}", inserted_trade); +let conn = establish_connection()?; ``` - -## Testing - -```bash -cargo test --package database -``` - -## Documentation - -Detailed API documentation is available at [docs.rs/database](https://docs.rs/database). diff --git a/crates/market-data/README.md b/crates/market-data/README.md new file mode 100644 index 000000000..f3acf6c13 --- /dev/null +++ b/crates/market-data/README.md @@ -0,0 +1,15 @@ +# market-data + +Market data repository for the Foxhunt HFT trading system. + +## Key Types + +- `MarketDataRepository` — PostgreSQL-backed market data storage +- Price, orderbook, indicator, and model data access methods + +## Usage + +```rust +use market_data::MarketDataRepository; +let repo = MarketDataRepository::new(pool).await?; +``` diff --git a/crates/ml-data/README.md b/crates/ml-data/README.md new file mode 100644 index 000000000..88e90bde4 --- /dev/null +++ b/crates/ml-data/README.md @@ -0,0 +1,17 @@ +# ml-data + +Production-ready ML data management for HFT trading systems. + +## Key Types + +- `MlDataConfig` — ML data repository configuration +- `FeaturesRepository` — feature storage and retrieval +- `ModelsRepository` — model metadata and versioning +- `PerformanceRepository` — model performance tracking +- `TrainingRepository` — training run management + +## Usage + +```rust +use ml_data::{MlDataConfig, FeaturesRepository}; +``` diff --git a/crates/ml/README.md b/crates/ml/README.md index 898305b15..d6adf0067 100644 --- a/crates/ml/README.md +++ b/crates/ml/README.md @@ -1,49 +1,32 @@ # ml -Machine learning models for Foxhunt. +10-model ML ensemble for the Foxhunt HFT system, built on Candle v0.9.1. ## Models -- **DQN (Rainbow)** -- Deep Q-Network with prioritized experience replay, dueling heads, noisy nets, double Q-learning -- **PPO** -- Proximal Policy Optimization with GAE, LSTM policies, clip-higher option -- **TFT** -- Temporal Fusion Transformer for multi-horizon time series forecasting -- **Mamba2** -- State space model for efficient sequence prediction -- **Liquid Networks** -- Biologically inspired neural networks for non-stationary data -- **TLOB** -- Transformer-based Limit Order Book analysis -- **Flash Attention** -- Optimized attention implementation - -## Training - -Two paths per model: - -1. **Standalone trainer** -- direct training loop (e.g., `DQN::train`, `PpoTrainer`) -2. **UnifiedTrainable adapter** -- wraps models for the hyperopt pipeline (e.g., `DQNTrainableAdapter`, `UnifiedTrainablePPO`) - -## Inference - -`InferenceAdapterBridge` connects models to the ensemble coordinator in `adaptive-strategy`. Each model exposes an `InferenceAdapter` trait for prediction. - -## Backend - -- **Candle v0.9.1** -- `VarMap`, `AdamW`, `loss.backward()`, `GradStore`, `opt.step(&grads)` -- **CUDA required for training** -- tested on RTX 3050 Ti 4GB, max batch size 230 -- **CPU inference** supported - -## Hyperopt - -`ArgminOptimizer` (Particle Swarm Optimization) with per-model adapters: -DQN, PPO, ContinuousPPO, TFT, Mamba2. Uses `ParameterSpace` trait for continuous parameter mapping. - -## ModelType Enum - -15 variants: `CompactDQN`, `DistilledMicroNet`, `DQN`, `RainbowDQN`, `MAMBA`, `TFT`, `TGGN`, `LNN`, `TLOB`, `PPO`, `Transformer`, `Mamba`, `LiquidNet`, `TGNN`, `Ensemble`. +- **DQN (Rainbow)** — deep Q-network with prioritized replay, dueling heads, noisy nets +- **PPO** — proximal policy optimization with GAE, LSTM policies, clip-higher +- **TFT** — temporal fusion transformer for multi-horizon forecasting +- **Mamba2** — state space model for sequence prediction +- **Liquid Networks** — biologically inspired networks for non-stationary data +- **TLOB** — transformer-based limit order book analysis +- **KAN** — Kolmogorov-Arnold networks +- **xLSTM** — extended LSTM architecture +- **TGGN** — temporal graph neural network +- **Diffusion** — diffusion-based generative model ## Key Modules -`dqn`, `ppo`, `tft`, `mamba`, `liquid`, `tlob`, `flash_attention`, `ensemble`, `evaluation`, `inference`, `trainers`, `hyperopt`, `checkpoint`, `preprocessing`, `data_loaders`, `features`, `model_factory`, `training_pipeline`, `regime_detection`, `stress_testing`, `validation`, `bridge`, `common`, `metrics`. +- `ensemble` — model ensemble coordination and confidence aggregation +- `hyperopt` — PSO-based hyperparameter optimization with per-model adapters +- `trainers` — unified training loops (DQN, PPO, supervised) +- `inference` — `InferenceAdapter` trait for prediction +- `checkpoint` — model checkpointing and restoration +- `evaluation` — walk-forward evaluation pipeline -## Testing +## Usage -```bash -SQLX_OFFLINE=true cargo test -p ml --lib # ~2009 tests +```rust +use ml::dqn::DQN; +use ml::ppo::PpoTrainer; ``` diff --git a/crates/model_loader/README.md b/crates/model_loader/README.md new file mode 100644 index 000000000..6de57353e --- /dev/null +++ b/crates/model_loader/README.md @@ -0,0 +1,17 @@ +# model_loader + +ML model loading and caching infrastructure. + +## Key Types + +- `AsyncModelLoader` — async model loading with LRU cache +- `LoadedModelInfo` — loaded model metadata and weights +- `ModelType` — re-exported from common (10 model variants) + +## Usage + +```rust +use model_loader::AsyncModelLoader; +let loader = AsyncModelLoader::new(cache_size); +let model = loader.load("path/to/checkpoint").await?; +``` diff --git a/crates/risk-data/README.md b/crates/risk-data/README.md new file mode 100644 index 000000000..3b5528390 --- /dev/null +++ b/crates/risk-data/README.md @@ -0,0 +1,16 @@ +# risk-data + +Risk data repository for high-frequency trading risk management. + +## Key Types + +- `RiskDataConfig` — repository configuration +- `VarRepository` — Value-at-Risk data storage +- `LimitsRepository` — position and risk limit management +- `ComplianceRepository` — compliance record storage + +## Usage + +```rust +use risk_data::{RiskDataConfig, VarRepository}; +``` diff --git a/crates/risk/README.md b/crates/risk/README.md index bac163ea8..fbfec1d8f 100644 --- a/crates/risk/README.md +++ b/crates/risk/README.md @@ -2,39 +2,23 @@ Enterprise risk management for HFT. -## Kill Switch +## Key Types -`AtomicKillSwitch` provides immediate trading cessation, coordinated via Redis. Supports local, remote, and Unix socket triggers. +- `RiskEngine` — central risk evaluation and enforcement +- `AtomicKillSwitch` — atomic trading halt with Redis coordination +- `KellySizer` — Kelly criterion position sizing +- `StressTester` — extreme market scenario simulation +- `ComplianceValidator` — SOX, MiFID II regulatory checks +- `DrawdownMonitor` — peak-to-trough equity tracking +- `CircuitBreaker` — threshold-based trading pauses +- `CorrelationMonitor` — cross-asset correlation tracking ## Value at Risk -Four VaR methods plus Expected Shortfall: - -- Historical Simulation -- Monte Carlo -- Parametric (variance-covariance) -- Expected Shortfall (CVaR) - -## Key Types - -| Type | Purpose | -|------|---------| -| `RiskEngine` | Central risk evaluation and enforcement | -| `AtomicKillSwitch` | Atomic trading halt with Redis coordination | -| `KellySizer` | Kelly criterion position sizing | -| `StressTester` | Extreme market scenario simulation | -| `ComplianceValidator` | SOX, MiFID II regulatory checks | -| `DrawdownMonitor` | Peak-to-trough equity tracking | -| `CircuitBreaker` | Threshold-based trading pauses | -| `CorrelationMonitor` | Cross-asset correlation tracking | +Four VaR methods plus Expected Shortfall (CVaR): +historical simulation, Monte Carlo, parametric (variance-covariance). ## Config Presets -- `development_config()` -- relaxed limits for local testing -- `production_config()` -- 5ms safety check timeout, strict position limits - -## Testing - -```bash -SQLX_OFFLINE=true cargo test -p risk --lib # ~209 tests -``` +- `development_config()` — relaxed limits for local testing +- `production_config()` — 5ms safety check timeout, strict position limits diff --git a/crates/storage/README.md b/crates/storage/README.md index a8f3917c4..5bc844322 100644 --- a/crates/storage/README.md +++ b/crates/storage/README.md @@ -1,53 +1,24 @@ -# Storage Crate +# storage -## Overview +S3-compatible object storage client for models, datasets, and training artifacts. -The `storage` crate provides robust integration with Amazon S3 for durable and scalable storage of models, large datasets, and other critical artifacts. It includes features for caching, versioning, data integrity verification, and efficient file management. +## Key Types + +- `S3Client` — upload, download, and manage S3 objects +- `S3Config` — bucket, region, and credential configuration +- Local caching layer for frequently accessed models +- Checksum verification (MD5, SHA256) on upload/download ## Features -* **S3 Client Integration:** Seamlessly integrates with AWS S3 for uploading, downloading, and managing objects. -* **Model Caching:** Implements a local caching layer for frequently accessed models, reducing latency and S3 API calls. -* **Model Versioning:** Supports tracking and managing different versions of machine learning models or configuration files stored in S3. -* **Checksum Verification:** Ensures data integrity by automatically verifying checksums (e.g., MD5, SHA256) during uploads and downloads. -* **Compression/Decompression Utilities:** Provides built-in support for compressing and decompressing files (e.g., Gzip, Zstd) to optimize storage and transfer costs. -* **File Management API:** Offers a high-level API for common S3 operations like listing objects, deleting, and managing prefixes. +- Model versioning and artifact management +- Compression/decompression (gzip, zstd) +- High-level file management API (list, delete, prefix operations) ## Usage ```rust use storage::{S3Client, S3Config}; -use std::path::PathBuf; - -let config = S3Config { - bucket_name: "foxhunt-models".to_string(), - region: "us-east-1".to_string(), - // ... other AWS credentials or profile settings -}; - -// let client = S3Client::new(config).expect("Failed to create S3 client"); - -let local_file_path = PathBuf::from("./my_model.bin"); -let s3_key = "models/v1/my_model.bin"; - -// Example: Upload a file to S3 -// client.upload_file(&local_file_path, s3_key, true /* with checksum */) -// .expect("Failed to upload model"); -// println!("Model uploaded to s3://{}/{}", config.bucket_name, s3_key); - -// Example: Download a file from S3 -// let download_path = PathBuf::from("./downloaded_model.bin"); -// client.download_file(s3_key, &download_path, true /* with checksum */) -// .expect("Failed to download model"); -// println!("Model downloaded to {:?}", download_path); +let client = S3Client::new(config)?; +client.upload_file(&path, "models/v1/model.bin", true)?; ``` - -## Testing - -```bash -cargo test --package storage -``` - -## Documentation - -Complete API documentation is available at [docs.rs/storage](https://docs.rs/storage). diff --git a/crates/trading-data/README.md b/crates/trading-data/README.md new file mode 100644 index 000000000..4816ef4e6 --- /dev/null +++ b/crates/trading-data/README.md @@ -0,0 +1,16 @@ +# trading-data + +Trading data repository layer for the high-frequency trading system. + +## Key Types + +- `PostgresOrderRepository` — PostgreSQL-backed order storage +- `OrderRepository` trait — order CRUD operations +- Position and execution tracking + +## Usage + +```rust +use trading_data::PostgresOrderRepository; +let repo = PostgresOrderRepository::new(pool).await?; +``` diff --git a/crates/trading_engine/README.md b/crates/trading_engine/README.md index 057e321e4..527fb3493 100644 --- a/crates/trading_engine/README.md +++ b/crates/trading_engine/README.md @@ -1,71 +1,27 @@ -# Trading Engine Crate +# trading_engine -## Overview +High-performance execution core for HFT — RDTSC timing, SIMD processing, lock-free data structures. -The `trading_engine` crate provides the high-performance core infrastructure essential for High-Frequency Trading (HFT) operations. It focuses on ultra-low latency execution, precise timing, and efficient order management to handle demanding market conditions. +## Key Types + +- `TradingEngine` — central execution engine +- `OrderManager` — order lifecycle management (placement, execution, cancellation) +- `IbAdapter` — Interactive Brokers connectivity +- `IcMarketsAdapter` — ICMarkets FIX 4.4 connectivity ## Features -* **Extreme Performance Optimization**: Utilizes RDTSC for precise timing, CPU affinity for dedicated core execution, and SIMD instructions for vectorized data processing. -* **Robust Order Management**: Manages the lifecycle of orders, from placement to execution and cancellation, ensuring accuracy and low-latency updates. -* **Flexible Execution Engine**: Implements a highly optimized engine capable of processing trading strategies and executing orders across various venues. -* **Multi-Broker Connectivity**: Seamlessly integrates with multiple brokers, including Interactive Brokers and ICMarkets, via specialized adapters. -* **Event-Sourced Architecture**: Employs event sourcing for deterministic state reconstruction, coupled with comprehensive metrics and persistent storage. -* **Concurrent Lock-Free Data Structures**: Leverages advanced lock-free data structures to minimize contention and maximize throughput in multi-threaded environments. - -## Architecture - -The `trading_engine` is structured around several key components: - -* **Execution Core**: The central logic for strategy evaluation and trade decision-making. -* **Order Manager**: Handles all order-related operations, maintaining order state and communicating with broker adapters. -* **Broker Adapters**: Abstract interfaces and concrete implementations for connecting to specific trading venues (e.g., `IbAdapter`, `IcMarketsAdapter`). -* **Performance Utilities**: Modules for RDTSC access, CPU core pinning, and SIMD instruction sets. -* **Event Store**: A mechanism for recording all significant events, enabling replay and auditability. -* **Metrics System**: Collects and reports performance and operational statistics. -* **Persistence Layer**: Stores critical state and event data for recovery and analysis. -* **Concurrency Primitives**: Custom lock-free queues, rings, and other data structures. +- RDTSC for sub-microsecond timing, CPU affinity for dedicated core execution +- SIMD-accelerated data processing +- Lock-free concurrent queues and ring buffers +- Event-sourced architecture for deterministic state reconstruction +- Multi-broker connectivity (IB TWS, ICMarkets) ## Usage -To initialize the trading engine and place a simple order: - ```rust -use trading_engine::{ - engine::TradingEngine, - order::{Order, OrderSide, OrderType}, - broker::BrokerType, -}; - -#[tokio::main] -async fn main() -> Result<(), Box> { - let mut engine = TradingEngine::new(); - engine.connect_broker(BrokerType::InteractiveBrokers).await?; - - let order = Order { - symbol: "ESZ23".to_string(), - side: OrderSide::Buy, - order_type: OrderType::Limit, - quantity: 1, - price: Some(4500.0), - // ... other order details - }; - - let order_id = engine.place_order(order).await?; - println!("Placed order with ID: {}", order_id); - - Ok(()) -} +use trading_engine::engine::TradingEngine; +let mut engine = TradingEngine::new(); +engine.connect_broker(BrokerType::InteractiveBrokers).await?; +let order_id = engine.place_order(order).await?; ``` - -## Testing - -To run the tests for the `trading_engine` crate: - -```bash -cargo test --package trading_engine -``` - -## Documentation - -Comprehensive API documentation is available at [docs.rs/trading_engine](https://docs.rs/trading_engine). diff --git a/crates/training_uploader/README.md b/crates/training_uploader/README.md new file mode 100644 index 000000000..96fb2367c --- /dev/null +++ b/crates/training_uploader/README.md @@ -0,0 +1,13 @@ +# training_uploader + +K8s sidecar that uploads training artifacts to S3 and reports completion via gRPC. + +## Usage + +Deployed as a sidecar container alongside training jobs. Watches for training +artifacts (checkpoints, metrics) and uploads them to the configured S3 bucket. + +## Configuration + +- `S3_BUCKET` — target bucket for artifacts +- `TRAINING_SERVICE_URL` — gRPC endpoint for completion reporting diff --git a/crates/web-gateway/README.md b/crates/web-gateway/README.md index cdb27429e..68e1eaa78 100644 --- a/crates/web-gateway/README.md +++ b/crates/web-gateway/README.md @@ -1,58 +1,31 @@ # web-gateway -REST and WebSocket gateway for the Foxhunt web dashboard. Built with Axum 0.7.9, it proxies HTTP requests to backend gRPC services and bridges gRPC streams to WebSocket clients. +REST and WebSocket gateway for the Foxhunt web dashboard. Axum 0.7.9 proxying HTTP to backend gRPC services. -## Configuration +## Key Types -All settings are read from environment variables. - -| Variable | Default | Description | -|---|---|---| -| `GATEWAY_LISTEN_ADDR` | `0.0.0.0:3000` | HTTP listen address | -| `JWT_SECRET` | _(required, min 32 chars)_ | HMAC key for JWT validation | -| `CORS_ORIGINS` | `http://localhost:5173` | Comma-separated allowed origins | -| `TRADING_SERVICE_URL` | `https://localhost:50051` | Trading gRPC upstream | -| `BACKTESTING_SERVICE_URL` | `https://localhost:50052` | Backtesting gRPC upstream | -| `ML_TRAINING_SERVICE_URL` | `https://localhost:50053` | ML training gRPC upstream | - -## Routes - -All API routes are under `/api`. Authentication is required unless noted. - -| Tier | Rate Limit | Endpoints | -|---|---|---| -| Public | 10 req/min | `POST /api/auth/login` | -| Trading | 200 req/min | `/api/trading/*`, `/api/risk/*`, `/api/ml/*`, `/api/performance/*`, `/api/config/*` | -| Compute | 30 req/min | `/api/training/*`, `/api/backtest/*`, `/api/tune/*` | -| WebSocket | 50 msg/min | `GET /api/ws` (JWT via `token` query param) | -| Health | none | `GET /health`, `GET /ready` | - -## Security - -- JWT authentication middleware (32-character minimum secret enforced at startup) -- Per-IP rate limiting across three tiers -- 1 MB request body size limit -- CORS with explicit methods (`GET`, `POST`, `PUT`, `DELETE`) and headers (`Authorization`, `Content-Type`) -- Response headers: HSTS, `x-frame-options: DENY`, `x-content-type-options: nosniff`, `referrer-policy: strict-origin-when-cross-origin` -- WebSocket topic whitelist (8 valid topics), per-connection rate limit, max 20 subscriptions -- `X-Request-Id` propagation (generated if absent, echoed on response) +- `AppState` — shared state (gRPC channels, WS broadcast, config) +- `GatewayConfig` — configuration loaded from environment variables ## Modules -| Module | Purpose | -|---|---| -| `auth` | JWT validation and login handler | -| `config` | `GatewayConfig` loaded from environment | -| `error` | Unified error types and HTTP error responses | -| `grpc` | Tonic clients and gRPC stream bridges | -| `rate_limit` | Per-IP token-bucket rate limiter (3 tiers) | -| `routes` | Axum router construction and all HTTP handlers | -| `state` | `AppState` (gRPC channels, WS broadcast, config) | -| `ws` | WebSocket upgrade, topic subscriptions, keepalive | +- `auth` — JWT validation and login handler +- `routes` — Axum router and HTTP handlers +- `grpc` — Tonic clients and gRPC stream bridges +- `ws` — WebSocket upgrade, topic subscriptions, keepalive +- `rate_limit` — per-IP token-bucket rate limiter (3 tiers) +- `error` — unified error types and HTTP error responses -## Running +## Rate Limits -```sh -export JWT_SECRET="your-secret-at-least-32-characters-long" -cargo run -p web-gateway -``` +| Tier | Limit | Endpoints | +|----------|-------------|----------------------------------------------------| +| Public | 10 req/min | `POST /api/auth/login` | +| Trading | 200 req/min | `/api/trading/*`, `/api/risk/*`, `/api/ml/*` | +| Compute | 30 req/min | `/api/training/*`, `/api/backtest/*`, `/api/tune/*`| + +## Security + +- JWT (32-char min secret), CORS, 1MB body limit, HSTS +- WebSocket topic whitelist (8 topics), per-connection rate limit, max 20 subs +- `X-Request-Id` propagation