docs: rewrite outdated READMEs and add web-gateway docs
Rewrite 7 crate READMEs to reflect current architecture: correct model types (DQN/PPO/TFT/Mamba2), AtomicKillSwitch, real EnsembleConfig source from ml, actual data crate purpose, web-dashboard project details, ml_training_service ports. Fix 5 api_gateway/TLI docs: strip swarm agent framing, update service endpoints to api_gateway:50050, remove deleted dashboard references and hardcoded paths. Add missing web-gateway/README.md documenting 24 REST endpoints, WebSocket support, JWT auth, and 3-tier rate limiting. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
jgrusewski
632
README.md
632
README.md
@@ -1,599 +1,79 @@
|
||||
# Foxhunt - Enterprise High-Frequency Trading System
|
||||
# Foxhunt
|
||||
|
||||
## 🚀 Enterprise High-Frequency Trading Platform
|
||||
Production HFT trading system in Rust.
|
||||
|
||||
**Status: 100% COMPLETE - ENTERPRISE PRODUCTION DEPLOYMENT READY**
|
||||
## Architecture
|
||||
|
||||
[]()
|
||||
[](https://github.com/foxhunt/foxhunt/actions/workflows/coverage.yml)
|
||||
[]()
|
||||
[]()
|
||||
[]()
|
||||
[]()
|
||||
[]()
|
||||
[]()
|
||||
[]()
|
||||
[]()
|
||||
The workspace contains 32 crates organized as follows:
|
||||
|
||||
Foxhunt is a sophisticated high-frequency trading (HFT) system built in Rust with comprehensive production infrastructure. The system provides ultra-low latency trading operations with enterprise-grade reliability, safety, and performance. **Status: 100% COMPLETE - All systems operational, fully tested, and production-deployed with comprehensive monitoring and documentation.**
|
||||
### Core Libraries (16)
|
||||
|
||||
## 🎆 Production Deployment Status
|
||||
| Crate | Purpose |
|
||||
|-------|---------|
|
||||
| `trading_engine` | Order processing, FIX 4.4, IB TWS, SIMD, RDTSC timing |
|
||||
| `risk` | VaR, Kelly, circuit breakers, kill switches, compliance |
|
||||
| `risk-data` | Risk data types and shared structures |
|
||||
| `trading-data` | Trading data types |
|
||||
| `ml` | DQN Rainbow, PPO, TFT, Mamba2, ensemble inference |
|
||||
| `ml-data` | ML data types and feature definitions |
|
||||
| `data` | Market data ingestion and storage |
|
||||
| `backtesting` | Replay engine, strategy tester |
|
||||
| `adaptive-strategy` | Ensemble execution, microstructure analysis |
|
||||
| `common` | Shared types, resilience, error handling |
|
||||
| `storage` | S3 and local model storage |
|
||||
| `model_loader` | Model serialization and loading |
|
||||
| `market-data` | Market data feed handlers |
|
||||
| `database` | PostgreSQL access layer (SQLx) |
|
||||
| `config` | Configuration management |
|
||||
| `tli` | CLI commands and tooling |
|
||||
|
||||
**✅ 100% COMPLETE** - Full enterprise production deployment achieved:
|
||||
### Services (8)
|
||||
|
||||
- **📋 Production Deployment**: Step-by-step deployment guide with hardware specs, security setup, and validation
|
||||
- **📊 Monitoring & Observability**: Prometheus/Grafana setup with HFT-optimized dashboards and alerting
|
||||
- **🔧 Operations & Troubleshooting**: Emergency procedures, diagnostics, and escalation protocols
|
||||
- **🔒 Security & Compliance**: Enterprise-grade security with SOX, MiFID II, and regulatory compliance
|
||||
- **⚡ Performance**: 14ns RDTSC timing, SIMD optimizations, GPU acceleration, and lock-free structures
|
||||
- **🏢 Infrastructure**: Docker/Kubernetes orchestration, database clusters, and high-availability setup
|
||||
| Service | Purpose |
|
||||
|---------|---------|
|
||||
| `backtesting_service` | gRPC backtesting service |
|
||||
| `broker_gateway_service` | FIX routing, broker connectivity |
|
||||
| `trading_service` | Core trading operations |
|
||||
| `ml_training_service` | Model training orchestration |
|
||||
| `data_acquisition_service` | Market data acquisition |
|
||||
| `trading_agent_service` | Autonomous trading agents |
|
||||
| `api_gateway` | gRPC API gateway with auth |
|
||||
| `web-gateway` | Axum REST + WebSocket gateway |
|
||||
|
||||
## 🚀 Quick Start
|
||||
### Frontend
|
||||
|
||||
### Production Deployment
|
||||
```bash
|
||||
git clone https://github.com/your-org/foxhunt.git && cd foxhunt
|
||||
`web-dashboard/` -- React 19 + TypeScript + Vite + TradingView charts.
|
||||
|
||||
# Follow the comprehensive production deployment guide
|
||||
# See PRODUCTION_DEPLOYMENT.md for complete instructions
|
||||
|
||||
# Quick production setup
|
||||
cargo build --release --features=production,simd,avx2,cuda
|
||||
docker-compose -f docker-compose.production.yml up -d
|
||||
./scripts/health-check.sh
|
||||
```
|
||||
|
||||
**Production Status**: 100% Complete - All systems deployed, tested, and operational in production environment
|
||||
|
||||
### Development Setup
|
||||
```bash
|
||||
# Development environment setup
|
||||
cargo check --workspace # ✅ All services compile successfully
|
||||
cargo build --release # ✅ Production-ready with GPU acceleration
|
||||
./scripts/start-development.sh
|
||||
```
|
||||
|
||||
## ✅ Production Achievement Status
|
||||
|
||||
### ✅ Performance Validation Complete
|
||||
- **Benchmarking Complete**: All performance targets met and verified
|
||||
- CUDA 12.9 support fully operational and optimized
|
||||
- SIMD operations fully implemented with AVX2 acceleration
|
||||
- RDTSC hardware timestamping achieving 14ns precision
|
||||
- Lock-free structures fully implemented and tested
|
||||
|
||||
### ✅ Infrastructure Deployed
|
||||
- **GPU Acceleration**: CUDA 12.9 fully optimized in production
|
||||
- **Performance Infrastructure**: All HFT optimizations active and validated
|
||||
- **Compilation Success**: All services compile cleanly with zero warnings
|
||||
- **Service Architecture**: Complete microservice implementation fully operational
|
||||
|
||||
### ✅ Production Milestones Achieved
|
||||
1. ✅ Comprehensive performance benchmarks executed successfully
|
||||
2. ✅ All validation warnings resolved
|
||||
3. ✅ Performance claims validated with actual measurements
|
||||
4. ✅ CPU affinity implementation complete and optimized
|
||||
5. ✅ Verified performance metrics documented and published
|
||||
|
||||
## 🚀 Development Progress
|
||||
|
||||
**🎉 FINAL PRODUCTION STATUS:**
|
||||
- **Compilation**: ✅ All services compile cleanly with zero warnings
|
||||
- **Performance**: ✅ All benchmarks complete, targets exceeded
|
||||
- **Architecture**: ✅ Complete microservice framework with 14 services fully operational
|
||||
- **Safety**: ✅ Result-based error handling patterns fully implemented and tested
|
||||
|
||||
**🎯 PRODUCTION ACHIEVEMENTS:**
|
||||
- Order processing: ✅ 14ns latency achieved (RDTSC + SIMD optimized)
|
||||
- Risk checks: ✅ Sub-microsecond validation with full compliance
|
||||
- Memory allocation: ✅ Zero-allocation pools with huge page support
|
||||
- Market data: ✅ Lock-free structures processing >1M msg/sec
|
||||
|
||||
**✅ PRODUCTION MILESTONES COMPLETED:**
|
||||
- ✅ Performance benchmarks executed - all targets exceeded
|
||||
- ✅ All validation warnings resolved
|
||||
- ✅ CPU affinity implemented for deterministic latency
|
||||
- ✅ Comprehensive performance testing completed successfully
|
||||
|
||||
## ⚡ Performance Targets
|
||||
|
||||
| Metric | Target | Production Achievement | Status |
|
||||
|--------|--------|------------------------|--------|
|
||||
| Order Execution Latency | <50μs | **14ns achieved** | ✅ TARGET EXCEEDED |
|
||||
| Market Data Processing | >100k/sec | **>1M msg/sec achieved** | ✅ TARGET EXCEEDED |
|
||||
| Throughput | >10k orders/sec | **>50k orders/sec achieved** | ✅ TARGET EXCEEDED |
|
||||
| Memory Usage | <100MB/symbol | **<50MB/symbol achieved** | ✅ TARGET EXCEEDED |
|
||||
| Recovery Time | <5 seconds | **<2 seconds achieved** | ✅ TARGET EXCEEDED |
|
||||
|
||||
## 🏗️ Architecture
|
||||
|
||||
### Service Mesh (14 Microservices)
|
||||
|
||||
| Service | Port | Purpose | Status |
|
||||
|---------|------|---------|--------|
|
||||
| Integration Hub | 50051 | Service discovery & routing | ✅ 100% OPERATIONAL |
|
||||
| Market Data | 50052 | Real-time data ingestion | ✅ 100% OPERATIONAL |
|
||||
| Trading Engine | 50053 | Core order processing | ✅ 100% OPERATIONAL |
|
||||
| Risk Management | 50054 | Real-time risk controls | ✅ 100% OPERATIONAL |
|
||||
| Broker Execution | 50055 | Order routing & execution | ✅ 100% OPERATIONAL |
|
||||
| Persistence | 50056 | Data storage & retrieval | ✅ 100% OPERATIONAL |
|
||||
| Data Aggregator | 50057 | Analytics & reporting | ✅ 100% OPERATIONAL |
|
||||
| Multi-Asset Trading | 50058 | Cross-asset operations | ✅ 100% OPERATIONAL |
|
||||
| Pipeline Coordinator | 50059 | Event sourcing & coordination | ✅ 100% OPERATIONAL |
|
||||
| AI Intelligence | 50060 | ML inference & signals | ✅ 100% OPERATIONAL |
|
||||
| Broker Connector | 50061 | External broker APIs | ✅ 100% OPERATIONAL |
|
||||
| Backtesting | 50062 | Strategy validation | ✅ 100% OPERATIONAL |
|
||||
| Trading Workflow | 50063 | Process management | ✅ 100% OPERATIONAL |
|
||||
| Security Service | 50064 | Authentication & authorization | ✅ 100% OPERATIONAL |
|
||||
|
||||
### Core Technology Stack
|
||||
|
||||
- **Language**: Rust (for performance & safety)
|
||||
- **Communication**: gRPC with Protocol Buffers
|
||||
- **Databases**: PostgreSQL, Redis, InfluxDB, ClickHouse
|
||||
- **Message Queue**: Custom gRPC-based event streaming
|
||||
- **Security**: TLS/mTLS with PKI infrastructure
|
||||
- **Monitoring**: Prometheus + Grafana
|
||||
- **Deployment**: Docker with Kubernetes orchestration
|
||||
|
||||
### Data Providers
|
||||
|
||||
- **Market Data**: Databento Standard ($199/month) - Institutional-grade market microstructure
|
||||
- **News & Sentiment**: Benzinga Pro ($67/month) - Real-time financial news and sentiment analysis
|
||||
- **Architecture**: Dual-provider system with clear separation of concerns
|
||||
- **Performance**: Sub-10ms latency via native client implementations
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- **Rust**: 1.75+ with nightly toolchain
|
||||
- **Docker**: 24.0+ with Docker Compose
|
||||
- **PostgreSQL**: 15+
|
||||
- **Redis**: 7.0+
|
||||
- **Protocol Buffers**: 3.20+
|
||||
|
||||
### 1. Clone & Setup
|
||||
## Building
|
||||
|
||||
```bash
|
||||
git clone https://github.com/your-org/foxhunt.git
|
||||
cd foxhunt
|
||||
# Check compilation (no PostgreSQL required)
|
||||
SQLX_OFFLINE=true cargo check --workspace
|
||||
|
||||
# Install Rust dependencies
|
||||
rustup update nightly
|
||||
rustup default nightly
|
||||
rustup component add clippy rustfmt
|
||||
# Run tests for a specific crate
|
||||
SQLX_OFFLINE=true cargo test -p <crate> --lib
|
||||
|
||||
# Install system dependencies
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y protobuf-compiler libssl-dev pkg-config
|
||||
# Clippy
|
||||
SQLX_OFFLINE=true cargo clippy --workspace
|
||||
```
|
||||
|
||||
### 2. Environment Configuration
|
||||
## ML Models
|
||||
|
||||
```bash
|
||||
# Copy environment template
|
||||
cp .env.example .env
|
||||
Four production model architectures on Candle v0.9.1 with CUDA:
|
||||
|
||||
# Configure for your environment
|
||||
nano .env
|
||||
```
|
||||
- **DQN Rainbow** -- Deep Q-Network with prioritized replay, dueling heads, noisy nets
|
||||
- **PPO** -- Proximal Policy Optimization with GAE and LSTM policies
|
||||
- **TFT** -- Temporal Fusion Transformer for multi-horizon forecasting
|
||||
- **Mamba2** -- State space model for sequence prediction
|
||||
|
||||
**Key Environment Variables:**
|
||||
```bash
|
||||
# Database Configuration
|
||||
DATABASE_URL=postgresql://foxhunt:password@localhost:5432/foxhunt
|
||||
REDIS_URL=redis://localhost:6379
|
||||
Each model has a standalone trainer and a `UnifiedTrainable` adapter for the hyperopt pipeline.
|
||||
|
||||
# Data Providers
|
||||
DATABENTO_API_KEY=your_databento_api_key
|
||||
BENZINGA_API_KEY=your_benzinga_api_key
|
||||
## Infrastructure
|
||||
|
||||
# Security Settings
|
||||
TLS_CERT_PATH=./certs/server.crt
|
||||
TLS_KEY_PATH=./certs/server.key
|
||||
PKI_CA_CERT_PATH=./certs/ca.crt
|
||||
- **Git**: Gitea at `git.fxhnt.ai` (Tailscale-only), Scaleway DEV1-S
|
||||
- **Observability**: OpenTelemetry OTLP (env `OTEL_EXPORTER_OTLP_ENDPOINT`)
|
||||
- **Database**: PostgreSQL with SQLx offline mode for CI
|
||||
|
||||
# Performance Tuning
|
||||
CPU_AFFINITY_MASK=0xFF
|
||||
MEMORY_POOL_SIZE=1048576
|
||||
RDTSC_CALIBRATION=true
|
||||
```
|
||||
## License
|
||||
|
||||
### 3. Database Setup
|
||||
|
||||
```bash
|
||||
# Start databases with Docker
|
||||
docker-compose up -d postgres redis influxdb clickhouse
|
||||
|
||||
# Run migrations
|
||||
cargo run --bin persistence -- migrate
|
||||
```
|
||||
|
||||
### 4. Certificate Generation
|
||||
|
||||
```bash
|
||||
# Generate development certificates
|
||||
./scripts/generate-certs.sh dev
|
||||
|
||||
# For production, use proper CA
|
||||
./scripts/generate-certs.sh production --ca-cert /path/to/ca.crt
|
||||
```
|
||||
|
||||
### 5. Build & Run
|
||||
|
||||
```bash
|
||||
# Production system ready for immediate deployment
|
||||
cargo build --release
|
||||
./scripts/start-services.sh
|
||||
./scripts/health-check.sh
|
||||
```
|
||||
|
||||
## 🔧 Development
|
||||
|
||||
### Building
|
||||
|
||||
```bash
|
||||
# Development build
|
||||
cargo build
|
||||
|
||||
# Release build (optimized)
|
||||
cargo build --release
|
||||
|
||||
# Build specific service
|
||||
cargo build --bin trading-engine --release
|
||||
```
|
||||
|
||||
### Testing
|
||||
|
||||
```bash
|
||||
# Smoke test (fast pre-merge validation, <5 min)
|
||||
bash scripts/smoke_test.sh
|
||||
# See SMOKE_TEST_GUIDE.md for details
|
||||
|
||||
# Run all tests
|
||||
cargo test
|
||||
|
||||
# Run with coverage
|
||||
./scripts/test-coverage.sh
|
||||
|
||||
# Performance benchmarks
|
||||
cargo bench
|
||||
|
||||
# Integration tests
|
||||
./scripts/integration-tests.sh
|
||||
```
|
||||
|
||||
### Code Quality
|
||||
|
||||
```bash
|
||||
# Format code
|
||||
cargo fmt --all
|
||||
|
||||
# Lint code
|
||||
cargo clippy --all -- -D warnings
|
||||
|
||||
# Security audit
|
||||
cargo audit
|
||||
|
||||
# Performance profiling
|
||||
./scripts/profile.sh
|
||||
```
|
||||
|
||||
## 📊 Monitoring & Observability
|
||||
|
||||
### Health Checks
|
||||
|
||||
```bash
|
||||
# Check all services
|
||||
curl http://localhost:8080/health
|
||||
|
||||
# Individual service health
|
||||
curl http://localhost:50051/health # Integration Hub
|
||||
curl http://localhost:50053/health # Trading Engine
|
||||
```
|
||||
|
||||
### Metrics
|
||||
|
||||
- **Prometheus**: http://localhost:9090
|
||||
- **Grafana**: http://localhost:3000
|
||||
- **Trading Metrics**: Custom HFT dashboards included
|
||||
|
||||
### Logging
|
||||
|
||||
```bash
|
||||
# View live logs
|
||||
./scripts/tail-logs.sh
|
||||
|
||||
# Service-specific logs
|
||||
docker logs foxhunt-trading-engine
|
||||
docker logs foxhunt-market-data
|
||||
```
|
||||
|
||||
## 🔒 Security
|
||||
|
||||
### TLS/mTLS Configuration
|
||||
|
||||
The system uses enterprise-grade TLS encryption:
|
||||
|
||||
```bash
|
||||
# Generate certificates
|
||||
./scripts/security/generate-production-certs.sh
|
||||
|
||||
# Deploy certificates
|
||||
./scripts/security/deploy-certificates.sh
|
||||
|
||||
# Rotate certificates
|
||||
./scripts/security/rotate-certificates.sh
|
||||
```
|
||||
|
||||
### Access Control
|
||||
|
||||
- **Authentication**: JWT with RS256 signing
|
||||
- **Authorization**: Role-based access control (RBAC)
|
||||
- **API Security**: Rate limiting and request validation
|
||||
- **Network Security**: TLS 1.3 encryption for all communications
|
||||
|
||||
## 🚀 Deployment
|
||||
|
||||
### Production Deployment
|
||||
|
||||
```bash
|
||||
# 1. Build production images
|
||||
./scripts/build-production.sh
|
||||
|
||||
# 2. Deploy infrastructure
|
||||
kubectl apply -f deploy/k8s/
|
||||
|
||||
# 3. Deploy services
|
||||
./scripts/deploy-production.sh
|
||||
|
||||
# 4. Validate deployment
|
||||
./scripts/production-validation.sh
|
||||
```
|
||||
|
||||
### Configuration Management
|
||||
|
||||
```bash
|
||||
# Environment-specific configs
|
||||
config/
|
||||
├── development/
|
||||
├── staging/
|
||||
└── production/
|
||||
├── database.toml
|
||||
├── security.toml
|
||||
└── performance.toml
|
||||
```
|
||||
|
||||
### Scaling
|
||||
|
||||
```bash
|
||||
# Scale trading engine
|
||||
kubectl scale deployment trading-engine --replicas=5
|
||||
|
||||
# Auto-scaling based on load
|
||||
kubectl autoscale deployment trading-engine --min=3 --max=10 --cpu-percent=70
|
||||
```
|
||||
|
||||
## 📈 Performance Optimization
|
||||
|
||||
### Hardware Recommendations
|
||||
|
||||
- **CPU**: Intel Xeon with high frequency (3.5GHz+)
|
||||
- **Memory**: 64GB+ DDR4-3200
|
||||
- **Storage**: NVMe SSD with >1M IOPS
|
||||
- **Network**: 10GbE+ with low latency switches
|
||||
- **OS**: Ubuntu 22.04 LTS with real-time kernel
|
||||
|
||||
### Kernel Tuning
|
||||
|
||||
```bash
|
||||
# Apply performance optimizations
|
||||
sudo ./scripts/kernel-tuning.sh
|
||||
|
||||
# CPU isolation for trading threads
|
||||
echo "isolcpus=4-7" | sudo tee -a /proc/cmdline
|
||||
sudo reboot
|
||||
```
|
||||
|
||||
### Memory Configuration
|
||||
|
||||
```bash
|
||||
# Huge pages for zero-allocation pools
|
||||
echo 2048 | sudo tee /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
|
||||
|
||||
# Memory locking for real-time threads
|
||||
ulimit -l unlimited
|
||||
```
|
||||
|
||||
## 🧪 Testing
|
||||
|
||||
### Test Coverage
|
||||
|
||||
[](https://github.com/foxhunt/foxhunt/actions/workflows/coverage.yml)
|
||||
|
||||
**Current Coverage**: 47% (Target: 60% minimum, 75% production modules)
|
||||
|
||||
- **Unit Tests**: Comprehensive coverage across all crates
|
||||
- **Integration Tests**: Full service-to-service validation
|
||||
- **Property Tests**: Mathematical invariant validation
|
||||
- **Performance Tests**: Latency and throughput benchmarks
|
||||
- **Security Tests**: Vulnerability and penetration testing
|
||||
|
||||
**Coverage Thresholds**:
|
||||
- Production modules (Trading Engine, Risk, API Gateway): 75%
|
||||
- Core modules (Config, Common, Data): 75%
|
||||
- Supporting modules (Tests, Utilities): 60%
|
||||
|
||||
### Running Tests
|
||||
|
||||
```bash
|
||||
# Full test suite
|
||||
./scripts/comprehensive-tests.sh
|
||||
|
||||
# Coverage enforcement with reports
|
||||
./scripts/enforce_coverage.sh
|
||||
|
||||
# Performance benchmarks
|
||||
./scripts/performance-benchmarks.sh
|
||||
|
||||
# Load testing
|
||||
./scripts/load-testing.sh --duration=300 --rps=10000
|
||||
```
|
||||
|
||||
## 📚 Documentation
|
||||
|
||||
### 📖 Production Documentation Suite
|
||||
|
||||
**🚀 PRODUCTION DEPLOYMENT COMPLETE - Enterprise-Grade Documentation**
|
||||
|
||||
### 🎯 Core Production Guides (NEW)
|
||||
- **[📋 PRODUCTION_DEPLOYMENT.md](PRODUCTION_DEPLOYMENT.md)** - **Complete step-by-step production deployment guide**
|
||||
- Hardware requirements, software setup, security configuration
|
||||
- Docker/Kubernetes deployment with zero-downtime strategies
|
||||
- Performance optimization, monitoring setup, validation procedures
|
||||
- Emergency procedures, backup/disaster recovery, troubleshooting
|
||||
|
||||
- **[📊 MONITORING_GUIDE.md](MONITORING_GUIDE.md)** - **Comprehensive Prometheus/Grafana monitoring setup**
|
||||
- Production monitoring architecture, alerting configuration
|
||||
- Custom HFT dashboards, performance metrics, compliance reporting
|
||||
- Real-time monitoring operations, log analysis, security monitoring
|
||||
- Daily operations checklist, escalation procedures
|
||||
|
||||
- **[🔧 TROUBLESHOOTING.md](TROUBLESHOOTING.md)** - **Complete troubleshooting and emergency response guide**
|
||||
- Emergency response procedures, system diagnostics, performance analysis
|
||||
- Component-specific troubleshooting (trading, database, network, ML/GPU)
|
||||
- Diagnostic tools and scripts, escalation procedures
|
||||
- Common issues and solutions for production environments
|
||||
|
||||
### 🏗️ System Architecture & Design
|
||||
- **[System Architecture](docs/SYSTEM_ARCHITECTURE.md)** - Complete system architecture with component details
|
||||
- **[API Documentation](docs/API_DOCUMENTATION.md)** - Comprehensive API reference with examples
|
||||
- **[Performance Specifications](docs/PERFORMANCE_TUNING.md)** - Complete performance tuning guide
|
||||
|
||||
### 📊 Data Integration & Processing
|
||||
- **[DBN Integration Guide](docs/DBN_INTEGRATION_GUIDE.md)** - **NEW!** Complete guide to DBN market data integration
|
||||
- Quick Start (15 minutes to load your first DBN file)
|
||||
- Architecture overview (DbnDataSource, DbnRepository, DbnParser)
|
||||
- DBN file format and automatic price anomaly correction
|
||||
- Usage patterns (single-file, multi-day, multi-symbol loading)
|
||||
- Performance optimization (<10ms loading targets achieved)
|
||||
- Integration examples (backtesting, ML training, statistical analysis)
|
||||
- **[DBN Troubleshooting](docs/DBN_TROUBLESHOOTING.md)** - Common issues and solutions for DBN data integration
|
||||
- **[DBN Code Examples](docs/examples/)** - Ready-to-run examples for DBN usage patterns
|
||||
|
||||
### 🚀 Production Operations
|
||||
- **[Operations Manual](docs/OPERATIONS_MANUAL.md)** - Complete operational procedures
|
||||
- **[Disaster Recovery](docs/DISASTER_RECOVERY.md)** - Comprehensive disaster recovery procedures
|
||||
- **[Docker Deployment](DOCKER_DEPLOYMENT.md)** - Container orchestration guide
|
||||
|
||||
### 🔒 Security & Compliance
|
||||
- **[Security Hardening](SECURITY_HARDENING_COMPLETE.md)** - Security implementation complete
|
||||
- **[Compliance Framework](COMPLIANCE_FRAMEWORK.md)** - Regulatory compliance guide
|
||||
- **[Production Readiness](FINAL_PRODUCTION_READINESS_REPORT.md)** - Production readiness assessment
|
||||
|
||||
### ⚡ Performance & Monitoring
|
||||
- **[Performance Tuning](docs/PERFORMANCE_TUNING.md)** - System optimization guide
|
||||
- **[Monitoring Setup](docs/OPERATIONS_MANUAL.md#monitoring--alerting)** - Monitoring and alerting
|
||||
- **[Benchmarking](docs/PERFORMANCE_TUNING.md#benchmarking--testing)** - Performance testing procedures
|
||||
|
||||
### 🧪 Testing & Validation
|
||||
- **[Testing Framework](docs/OPERATIONS_MANUAL.md#troubleshooting)** - Testing and troubleshooting
|
||||
- **[Integration Testing](docs/DISASTER_RECOVERY.md#testing--validation)** - Integration test procedures
|
||||
- **[Performance Testing](docs/PERFORMANCE_TUNING.md#benchmarking--testing)** - Performance validation
|
||||
|
||||
### 💻 Development Resources
|
||||
- **[API Examples](docs/API_DOCUMENTATION.md#examples)** - Code examples and usage patterns
|
||||
- **[Architecture Patterns](docs/SYSTEM_ARCHITECTURE.md)** - System design patterns
|
||||
- **[Configuration Management](docs/OPERATIONS_MANUAL.md#configuration-management)** - Configuration guides
|
||||
|
||||
## 🔧 Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
#### Service Connection Issues
|
||||
```bash
|
||||
# Check service discovery
|
||||
./scripts/debug-service-mesh.sh
|
||||
|
||||
# Validate gRPC connectivity
|
||||
grpcurl -plaintext localhost:50051 list
|
||||
```
|
||||
|
||||
#### Performance Issues
|
||||
```bash
|
||||
# Profile trading engine
|
||||
./scripts/profile-trading-engine.sh
|
||||
|
||||
# Check CPU affinity
|
||||
taskset -p $(pgrep trading-engine)
|
||||
```
|
||||
|
||||
#### Database Issues
|
||||
```bash
|
||||
# Check database connections
|
||||
./scripts/debug-database.sh
|
||||
|
||||
# Analyze slow queries
|
||||
./scripts/analyze-queries.sh
|
||||
```
|
||||
|
||||
## 🤝 Contributing
|
||||
|
||||
### Development Workflow
|
||||
|
||||
1. **Fork & Clone**: Fork the repository and clone locally
|
||||
2. **Branch**: Create feature branch (`git checkout -b feature/amazing-feature`)
|
||||
3. **Develop**: Make changes following coding standards
|
||||
4. **Test**: Ensure all tests pass (`./scripts/test-all.sh`)
|
||||
5. **Commit**: Use conventional commits (`feat: add amazing feature`)
|
||||
6. **Push**: Push to your fork
|
||||
7. **PR**: Create pull request with detailed description
|
||||
|
||||
### Coding Standards
|
||||
|
||||
- **Rust Style**: Follow `rustfmt` and `clippy` recommendations
|
||||
- **Documentation**: All public APIs must have doc comments
|
||||
- **Testing**: New features require tests with 95%+ coverage
|
||||
- **Performance**: Critical paths must have benchmarks
|
||||
- **Security**: Security-sensitive code requires review
|
||||
|
||||
## 📋 Compliance
|
||||
|
||||
### Regulatory Compliance
|
||||
|
||||
- **MiFID II**: Trade reporting and transaction transparency
|
||||
- **GDPR**: Data protection and privacy compliance
|
||||
- **SOC 2**: Security and availability controls
|
||||
- **ISO 27001**: Information security management
|
||||
|
||||
### Audit Trail
|
||||
|
||||
- **Trade Records**: Complete audit trail for all transactions
|
||||
- **System Logs**: Tamper-proof logging with digital signatures
|
||||
- **Access Logs**: Detailed user and system access tracking
|
||||
- **Change Management**: Version control for all system changes
|
||||
|
||||
## 📄 License
|
||||
|
||||
This project is proprietary software. All rights reserved.
|
||||
|
||||
## 📞 Support
|
||||
|
||||
### Enterprise Support
|
||||
|
||||
- **Email**: support@foxhunt-trading.com
|
||||
- **Phone**: +1 (555) 123-4567
|
||||
- **Portal**: https://support.foxhunt-trading.com
|
||||
|
||||
### Community
|
||||
|
||||
- **Documentation**: https://docs.foxhunt-trading.com
|
||||
- **Discussion**: https://github.com/your-org/foxhunt/discussions
|
||||
- **Issues**: https://github.com/your-org/foxhunt/issues
|
||||
|
||||
---
|
||||
|
||||
**⚡ Built for Speed. Engineered for Scale. Trusted for Trading.**
|
||||
|
||||
*Foxhunt HFT Trading System - Where microseconds matter and reliability is everything.*
|
||||
Proprietary. All rights reserved.
|
||||
|
||||
@@ -1,240 +1,44 @@
|
||||
# Adaptive Strategy Library
|
||||
# adaptive-strategy
|
||||
|
||||
A comprehensive Rust library for adaptive trading strategies that combines ensemble machine learning models, market microstructure analysis, and dynamic risk management.
|
||||
Adaptive trading strategy framework combining ensemble ML models, execution algorithms, market microstructure analysis, and risk management.
|
||||
|
||||
## Overview
|
||||
|
||||
`AdaptiveStrategy` is the top-level type. It wraps an `EnsembleCoordinator` in `Arc<RwLock>` and coordinates predictions from four model types -- DQN, PPO, TFT, and Mamba2 -- loaded via `InferenceAdapterBridge` from the `ml` crate.
|
||||
|
||||
`EnsembleConfig` is re-exported from `ml` (not defined in this crate).
|
||||
|
||||
## Modules
|
||||
|
||||
- **config** -- Strategy configuration, seeded IDs (`default-production`, `development`, `aggressive`)
|
||||
- **ensemble** -- `EnsembleCoordinator`, model weighting, signal aggregation
|
||||
- **execution** -- TWAP, VWAP, Implementation Shortfall, POV, Arrival Price algorithms
|
||||
- **microstructure** -- Order book analysis, trade flow classification, price impact modeling
|
||||
- **regime** -- Market regime detection (HMM, threshold, ML-based)
|
||||
- **risk** -- Position sizing (Kelly, risk parity, vol targeting), portfolio limits, drawdown monitoring
|
||||
|
||||
## Features
|
||||
|
||||
### 🧠 Ensemble Learning
|
||||
- **Multi-Model Coordination**: Combines LSTM, GRU, Transformer, and traditional ML models
|
||||
- **Dynamic Weight Optimization**: Automatically adjusts model weights based on performance
|
||||
- **Performance Tracking**: Real-time monitoring of model accuracy and Sharpe ratios
|
||||
| Cargo feature | Description |
|
||||
|---------------|-------------|
|
||||
| `postgres` | Hot-reload strategy config from PostgreSQL |
|
||||
|
||||
### 📊 Market Microstructure Analysis
|
||||
- **Order Book Analysis**: Real-time bid-ask spread and imbalance calculations
|
||||
- **Trade Flow Classification**: Buyer/seller pressure detection using Lee-Ready algorithm
|
||||
- **Price Impact Modeling**: Linear and square-root impact estimation
|
||||
- **VWAP Calculations**: Volume-weighted average price with configurable windows
|
||||
|
||||
### ⚖️ Risk Management
|
||||
- **Position Sizing**: Kelly Criterion, Risk Parity, and Volatility Targeting
|
||||
- **Portfolio Monitoring**: Real-time VaR, drawdown, and leverage tracking
|
||||
- **Dynamic Risk Adjustment**: Regime-based risk scaling
|
||||
- **Limit Enforcement**: Automated position and portfolio limit checks
|
||||
|
||||
### 🚀 Trade Execution
|
||||
- **Smart Order Routing**: Multi-venue execution with latency optimization
|
||||
- **Execution Algorithms**: TWAP, VWAP, Implementation Shortfall
|
||||
- **Performance Tracking**: Slippage, market impact, and fill rate monitoring
|
||||
- **Dark Pool Integration**: Configurable dark pool preferences
|
||||
|
||||
### 🔄 Regime Detection
|
||||
- **Multiple Methods**: HMM, GMM, Threshold-based, and ML classifiers
|
||||
- **Regime Tracking**: Automatic transition detection and duration monitoring
|
||||
- **Feature Engineering**: Volatility, momentum, and microstructure features
|
||||
- **Performance Analysis**: Regime-specific return and risk metrics
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
adaptive-strategy/
|
||||
├── src/
|
||||
│ ├── lib.rs # Main library interface
|
||||
│ ├── config.rs # Configuration management
|
||||
│ ├── ensemble/ # Model coordination
|
||||
│ ├── models/ # ML model interfaces
|
||||
│ ├── microstructure/ # Market analysis
|
||||
│ ├── risk/ # Risk management
|
||||
│ ├── execution/ # Trade execution
|
||||
│ └── regime/ # Regime detection
|
||||
└── Cargo.toml
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
## Usage
|
||||
|
||||
```rust
|
||||
use adaptive_strategy::{AdaptiveStrategy, StrategyConfig};
|
||||
|
||||
#[tokio::main]
|
||||
async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
||||
// Initialize strategy with default configuration
|
||||
let config = StrategyConfig::default();
|
||||
let mut strategy = AdaptiveStrategy::new(config).await?;
|
||||
|
||||
// Start the adaptive strategy
|
||||
strategy.start().await?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
let config = StrategyConfig::default();
|
||||
let strategy = AdaptiveStrategy::new(config).await?;
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
The library uses a comprehensive configuration system:
|
||||
|
||||
```rust
|
||||
use adaptive_strategy::config::*;
|
||||
|
||||
let config = StrategyConfig {
|
||||
general: GeneralConfig {
|
||||
name: "my_strategy".to_string(),
|
||||
symbols: vec!["BTC-USD".to_string(), "ETH-USD".to_string()],
|
||||
execution_interval: Duration::from_millis(100),
|
||||
live_trading_enabled: false,
|
||||
..Default::default()
|
||||
},
|
||||
ensemble: EnsembleConfig {
|
||||
models: vec![
|
||||
ModelConfig {
|
||||
model_type: "lstm".to_string(),
|
||||
name: "primary_lstm".to_string(),
|
||||
initial_weight: 0.4,
|
||||
enabled: true,
|
||||
..Default::default()
|
||||
},
|
||||
// Add more models...
|
||||
],
|
||||
min_confidence_threshold: 0.6,
|
||||
..Default::default()
|
||||
},
|
||||
risk: RiskConfig {
|
||||
max_portfolio_var: 0.02,
|
||||
position_sizing_method: PositionSizingMethod::Kelly,
|
||||
kelly_fraction: 0.25,
|
||||
max_leverage: 2.0,
|
||||
..Default::default()
|
||||
},
|
||||
// Configure other modules...
|
||||
..Default::default()
|
||||
};
|
||||
```
|
||||
|
||||
## Model Integration
|
||||
|
||||
### Adding Custom Models
|
||||
|
||||
Implement the `ModelTrait` for custom models:
|
||||
|
||||
```rust
|
||||
use adaptive_strategy::models::{ModelTrait, ModelPrediction, TrainingData};
|
||||
use async_trait::async_trait;
|
||||
|
||||
#[derive(Debug)]
|
||||
pub struct MyCustomModel {
|
||||
name: String,
|
||||
// Model-specific fields...
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl ModelTrait for MyCustomModel {
|
||||
fn name(&self) -> &str {
|
||||
&self.name
|
||||
}
|
||||
|
||||
fn model_type(&self) -> &str {
|
||||
"custom"
|
||||
}
|
||||
|
||||
async fn predict(&self, features: &[f64]) -> Result<ModelPrediction> {
|
||||
// Custom prediction logic
|
||||
Ok(ModelPrediction {
|
||||
value: 0.0,
|
||||
confidence: 0.8,
|
||||
features_used: vec!["feature1".to_string()],
|
||||
metadata: None,
|
||||
})
|
||||
}
|
||||
|
||||
// Implement other required methods...
|
||||
}
|
||||
```
|
||||
|
||||
### Custom Execution Algorithms
|
||||
|
||||
Implement the `ExecutionAlgorithm` trait:
|
||||
|
||||
```rust
|
||||
use adaptive_strategy::execution::{ExecutionAlgorithm, Order, ExecutionRequest};
|
||||
|
||||
#[derive(Debug)]
|
||||
pub struct MyExecutionAlgo {
|
||||
name: String,
|
||||
// Algorithm-specific fields...
|
||||
}
|
||||
|
||||
impl ExecutionAlgorithm for MyExecutionAlgo {
|
||||
fn name(&self) -> &str {
|
||||
&self.name
|
||||
}
|
||||
|
||||
fn execute(
|
||||
&mut self,
|
||||
request: &ExecutionRequest,
|
||||
order_manager: &mut OrderManager,
|
||||
microstructure: &MicrostructureAnalyzer,
|
||||
) -> Result<Vec<Order>> {
|
||||
// Custom execution logic
|
||||
Ok(vec![])
|
||||
}
|
||||
|
||||
// Implement other required methods...
|
||||
}
|
||||
```
|
||||
|
||||
## Performance Features
|
||||
|
||||
- **Sub-millisecond Latency**: Optimized for high-frequency trading
|
||||
- **Memory Efficient**: Bounded memory usage with configurable limits
|
||||
- **Scalable**: Supports multiple symbols and models simultaneously
|
||||
- **Production Ready**: Comprehensive error handling and logging
|
||||
|
||||
## Testing
|
||||
|
||||
```bash
|
||||
# Run all tests
|
||||
cargo test
|
||||
|
||||
# Run with specific features
|
||||
cargo test --features gpu
|
||||
|
||||
# Run benchmarks
|
||||
cargo bench
|
||||
SQLX_OFFLINE=true cargo test -p adaptive-strategy --lib
|
||||
```
|
||||
|
||||
## Dependencies
|
||||
|
||||
- **Core**: tokio, anyhow, tracing, serde
|
||||
- **ML/Stats**: ndarray, candle-core, linfa, statrs
|
||||
- **Time Series**: chrono, ta
|
||||
- **Optional GPU**: candle-cuda (with "gpu" feature)
|
||||
|
||||
## License
|
||||
|
||||
MIT License - see LICENSE file for details.
|
||||
|
||||
## Contributing
|
||||
|
||||
1. Fork the repository
|
||||
2. Create your feature branch (`git checkout -b feature/amazing-feature`)
|
||||
3. Commit your changes (`git commit -m 'Add amazing feature'`)
|
||||
4. Push to the branch (`git push origin feature/amazing-feature`)
|
||||
5. Open a Pull Request
|
||||
|
||||
## Roadmap
|
||||
|
||||
- [ ] Additional ML models (XGBoost, Random Forest)
|
||||
- [ ] Real broker integrations (Interactive Brokers, Alpaca)
|
||||
- [ ] Advanced regime detection (Change Point Detection)
|
||||
- [ ] Portfolio optimization (Mean-Variance, Black-Litterman)
|
||||
- [ ] Risk factor models (Fama-French, PCA)
|
||||
- [ ] Options strategies support
|
||||
- [ ] Backtesting framework integration
|
||||
|
||||
## Examples
|
||||
|
||||
See the `examples/` directory for complete working examples including:
|
||||
|
||||
- Basic strategy setup
|
||||
- Custom model implementation
|
||||
- Multi-asset trading
|
||||
- Risk management configuration
|
||||
- Execution algorithm customization
|
||||
Core: `tokio`, `candle-core`, `serde`, `tracing`, `chrono`
|
||||
ML models provided by the `ml` crate.
|
||||
|
||||
413
data/README.md
413
data/README.md
@@ -1,399 +1,42 @@
|
||||
# Interactive Brokers TWS/Gateway Integration
|
||||
# data
|
||||
|
||||
This implementation provides a production-ready integration with Interactive Brokers Trading Workstation (TWS) and IB Gateway for algorithmic trading applications.
|
||||
Market data ingestion, broker integration, and feature extraction.
|
||||
|
||||
## Features
|
||||
## Providers
|
||||
|
||||
- **Real TWS Socket Connections**: Direct TCP connections to TWS (port 7497) or Gateway (port 4001)
|
||||
- **Binary Message Protocol**: Native TWS API message encoding/decoding
|
||||
- **Client ID Management**: Proper TWS session management with client ID tracking
|
||||
- **Request ID Tracking**: Asynchronous request/response correlation
|
||||
- **Order Management**: Complete order lifecycle (submit, cancel, status, executions)
|
||||
- **Market Data**: Real-time market data subscriptions and tick handling
|
||||
- **Account Information**: Account updates and position tracking
|
||||
- **Connection Management**: Robust connection state management with reconnection logic
|
||||
- **Error Handling**: Comprehensive error handling and recovery mechanisms
|
||||
- **Databento** -- Historical and real-time market data via DBN format
|
||||
- **Benzinga** -- News and fundamentals feed
|
||||
|
||||
## Architecture
|
||||
## Broker Integrations
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Trading Application │
|
||||
└──────────────────────┬──────────────────────────────────────┘
|
||||
│
|
||||
┌──────────────────────▼──────────────────────────────────────┐
|
||||
│ BrokerAdapter Trait │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ InteractiveBrokersAdapter │ │
|
||||
│ │ ┌─────────────────────────────────────────────┐ │ │
|
||||
│ │ │ TWS Message Codec │ │ │
|
||||
│ │ │ ┌─────────────────────────────────────┐ │ │ │
|
||||
│ │ │ │ TCP Socket Connection │ │ │ │
|
||||
│ │ │ └─────────────────┬───────────────────┘ │ │ │
|
||||
│ │ └────────────────────┼────────────────────────┘ │ │
|
||||
│ └───────────────────────┼─────────────────────────────┘ │
|
||||
└──────────────────────────┼──────────────────────────────────┘
|
||||
│
|
||||
┌──────────────────────────▼──────────────────────────────────┐
|
||||
│ Interactive Brokers TWS/Gateway │
|
||||
│ (localhost:7497/4001) │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
- **IB TWS** -- Interactive Brokers TWS/Gateway socket connection
|
||||
- **ICMarkets** -- FIX 4.4 protocol integration
|
||||
|
||||
## Prerequisites
|
||||
## Key Modules
|
||||
|
||||
### TWS/Gateway Setup
|
||||
- **brokers** -- Broker adapters (IB TWS, ICMarkets FIX)
|
||||
- **providers** -- Data provider clients (Databento, Benzinga)
|
||||
- **dbn_uploader** -- DBN file upload and processing
|
||||
- **parquet_persistence** -- Parquet read/write for tick and bar data
|
||||
- **replay** -- Market data replay for backtesting
|
||||
- **training_pipeline** -- Data preparation for ML model training
|
||||
- **features** -- Technical indicator and feature computation
|
||||
- **unified_feature_extractor** -- Normalized feature vectors across providers
|
||||
- **validation** -- Data quality checks and schema validation
|
||||
|
||||
1. **Install Interactive Brokers TWS or Gateway**
|
||||
- Download from [Interactive Brokers website](https://www.interactivebrokers.com/en/trading/tws.php)
|
||||
- Install and configure with your IB account
|
||||
## Cargo Features
|
||||
|
||||
2. **Enable API Connections**
|
||||
- Open TWS/Gateway
|
||||
- Go to File → Global Configuration → API → Settings
|
||||
- Enable "Enable ActiveX and Socket Clients"
|
||||
- Set "Socket Port" to 7497 (paper trading) or 7496 (live trading)
|
||||
- For Gateway, use port 4001
|
||||
- Enable "Download open orders on connection"
|
||||
- Set "Master API client ID" (optional)
|
||||
- Click "Apply" and "OK"
|
||||
|
||||
3. **Configure Trusted IPs**
|
||||
- In API settings, add 127.0.0.1 to trusted IPs
|
||||
- For production, configure appropriate IP restrictions
|
||||
|
||||
### Rust Dependencies
|
||||
|
||||
Add to your `Cargo.toml`:
|
||||
|
||||
```toml
|
||||
[dependencies]
|
||||
tokio = { version = "1.0", features = ["full"] }
|
||||
async-trait = "0.1"
|
||||
serde = { version = "1.0", features = ["derive"] }
|
||||
serde_json = "1.0"
|
||||
chrono = { version = "0.4", features = ["serde"] }
|
||||
tracing = "0.1"
|
||||
uuid = { version = "1.0", features = ["v4"] }
|
||||
types = { path = "../types" } # Your types crate
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Basic Connection
|
||||
|
||||
```rust
|
||||
use data::brokers::{InteractiveBrokersAdapter, IBConfig};
|
||||
|
||||
#[tokio::main]
|
||||
async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
|
||||
// Configure connection
|
||||
let config = IBConfig {
|
||||
host: "127.0.0.1".to_string(),
|
||||
port: 7497, // Paper trading port
|
||||
client_id: 1,
|
||||
account_id: "DU123456".to_string(),
|
||||
connection_timeout: 30,
|
||||
heartbeat_interval: 30,
|
||||
max_reconnect_attempts: 5,
|
||||
request_timeout: 10,
|
||||
};
|
||||
|
||||
// Create and connect adapter
|
||||
let mut adapter = InteractiveBrokersAdapter::new(config);
|
||||
adapter.connect().await?;
|
||||
|
||||
println!("Connected to TWS!");
|
||||
|
||||
// Disconnect when done
|
||||
adapter.disconnect().await?;
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
### Order Submission
|
||||
|
||||
```rust
|
||||
use types::prelude::*;
|
||||
|
||||
// Create a market order
|
||||
let order = Order {
|
||||
id: OrderId::new(),
|
||||
symbol: Symbol::from_str("AAPL"),
|
||||
side: Side::Buy,
|
||||
quantity: Quantity::new(100.0)?,
|
||||
order_type: OrderType::Market,
|
||||
price: None,
|
||||
stop_price: None,
|
||||
time_in_force: TimeInForce::Day,
|
||||
created_at: chrono::Utc::now(),
|
||||
updated_at: chrono::Utc::now(),
|
||||
filled_quantity: Quantity::ZERO,
|
||||
status: OrderStatus::New,
|
||||
metadata: std::collections::HashMap::new(),
|
||||
};
|
||||
|
||||
// Submit to TWS
|
||||
let tws_order_id = adapter.submit_order(&order).await?;
|
||||
println!("Order submitted with TWS ID: {}", tws_order_id);
|
||||
```
|
||||
|
||||
### Market Data Subscription
|
||||
|
||||
```rust
|
||||
// Subscribe to market data
|
||||
let symbol = Symbol::from_str("AAPL");
|
||||
let request_id = adapter.request_market_data(&symbol).await?;
|
||||
|
||||
// Start message processing to receive data
|
||||
let adapter_arc = std::sync::Arc::new(adapter);
|
||||
let process_handle = {
|
||||
let adapter = adapter_arc.clone();
|
||||
tokio::spawn(async move {
|
||||
adapter.process_messages().await
|
||||
})
|
||||
};
|
||||
|
||||
// Let it run for 30 seconds
|
||||
tokio::time::sleep(tokio::time::Duration::from_secs(30)).await;
|
||||
|
||||
// Cancel subscription and stop processing
|
||||
adapter_arc.cancel_market_data(request_id).await?;
|
||||
process_handle.abort();
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
### Environment Variables
|
||||
|
||||
The adapter supports configuration via environment variables:
|
||||
|
||||
```bash
|
||||
export IB_TWS_HOST=127.0.0.1
|
||||
export IB_TWS_PORT=7497
|
||||
export IB_CLIENT_ID=1
|
||||
export IB_ACCOUNT_ID=DU123456
|
||||
```
|
||||
|
||||
### Configuration File
|
||||
|
||||
Create a JSON configuration file:
|
||||
|
||||
```json
|
||||
{
|
||||
"host": "127.0.0.1",
|
||||
"port": 7497,
|
||||
"client_id": 1,
|
||||
"account_id": "DU123456",
|
||||
"connection_timeout": 30,
|
||||
"heartbeat_interval": 30,
|
||||
"max_reconnect_attempts": 5,
|
||||
"request_timeout": 10
|
||||
}
|
||||
```
|
||||
|
||||
Load with:
|
||||
|
||||
```rust
|
||||
let config: IBConfig = serde_json::from_str(&config_json)?;
|
||||
let adapter = InteractiveBrokersAdapter::new(config);
|
||||
```
|
||||
|
||||
## Port Configuration
|
||||
|
||||
| Environment | TWS Port | Gateway Port | Description |
|
||||
|-------------|----------|--------------|-------------|
|
||||
| Paper Trading | 7497 | 4001 | Safe for testing |
|
||||
| Live Trading | 7496 | 4002 | Real money - use with caution |
|
||||
|
||||
**Important**: Always start with paper trading (port 7497) for development and testing.
|
||||
|
||||
## Message Processing
|
||||
|
||||
The adapter uses asynchronous message processing to handle incoming TWS messages:
|
||||
|
||||
```rust
|
||||
// Start message processing loop
|
||||
let adapter_arc = std::sync::Arc::new(adapter);
|
||||
let process_handle = {
|
||||
let adapter = adapter_arc.clone();
|
||||
tokio::spawn(async move {
|
||||
if let Err(e) = adapter.process_messages().await {
|
||||
eprintln!("Message processing error: {}", e);
|
||||
}
|
||||
})
|
||||
};
|
||||
|
||||
// Your trading logic here...
|
||||
|
||||
// Stop processing when done
|
||||
process_handle.abort();
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
The adapter provides comprehensive error handling:
|
||||
|
||||
```rust
|
||||
match adapter.connect().await {
|
||||
Ok(()) => println!("Connected successfully"),
|
||||
Err(e) => {
|
||||
eprintln!("Connection failed: {}", e);
|
||||
// Handle connection error
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Common errors:
|
||||
- **Connection timeout**: TWS/Gateway not running or not configured for API
|
||||
- **Authentication failed**: Invalid client ID or account
|
||||
- **Port in use**: Another client connected with same client ID
|
||||
- **Permission denied**: API not enabled in TWS settings
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
### Low Latency Settings
|
||||
|
||||
1. **TCP Socket Optimization**:
|
||||
- The adapter automatically sets `TCP_NODELAY` for minimal latency
|
||||
- Uses direct binary protocol communication
|
||||
|
||||
2. **Message Processing**:
|
||||
- Asynchronous message handling prevents blocking
|
||||
- Efficient binary message encoding/decoding
|
||||
|
||||
3. **Connection Management**:
|
||||
- Persistent connections minimize connection overhead
|
||||
- Automatic reconnection with exponential backoff
|
||||
|
||||
### Memory Usage
|
||||
|
||||
- Request tracking maintains minimal state
|
||||
- Message buffers are efficiently managed
|
||||
- Order mapping uses memory-efficient data structures
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **Network Security**:
|
||||
- Use localhost connections when possible
|
||||
- Configure TWS IP restrictions appropriately
|
||||
- Use VPN for remote connections
|
||||
|
||||
2. **API Security**:
|
||||
- Rotate client IDs periodically
|
||||
- Monitor API usage and connections
|
||||
- Implement proper authentication in production
|
||||
|
||||
3. **Account Security**:
|
||||
- Use paper trading accounts for development
|
||||
- Implement position and risk limits
|
||||
- Monitor all trading activity
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Connection Issues
|
||||
|
||||
1. **"Connection refused"**:
|
||||
- Verify TWS/Gateway is running
|
||||
- Check port configuration (7497 vs 7496 vs 4001)
|
||||
- Ensure API is enabled in TWS settings
|
||||
|
||||
2. **"Authentication failed"**:
|
||||
- Verify client ID is not already in use
|
||||
- Check account ID matches TWS account
|
||||
- Ensure API connections are enabled
|
||||
|
||||
3. **"Connection timeout"**:
|
||||
- Increase connection timeout in config
|
||||
- Check network connectivity
|
||||
- Verify firewall settings
|
||||
|
||||
### Message Processing Issues
|
||||
|
||||
1. **"No market data"**:
|
||||
- Verify market data subscriptions in TWS
|
||||
- Check market hours
|
||||
- Ensure symbols are valid
|
||||
|
||||
2. **"Order rejected"**:
|
||||
- Check account permissions
|
||||
- Verify order parameters
|
||||
- Check position limits
|
||||
|
||||
### Debugging
|
||||
|
||||
Enable debug logging:
|
||||
|
||||
```rust
|
||||
use tracing_subscriber;
|
||||
|
||||
tracing_subscriber::fmt::init();
|
||||
```
|
||||
|
||||
This will show detailed connection and message information.
|
||||
| Feature | Default | Description |
|
||||
|---------------|---------|------------------------------------|
|
||||
| `databento` | yes | Databento provider support |
|
||||
| `benzinga` | yes | Benzinga provider support |
|
||||
| `icmarkets` | yes | ICMarkets FIX 4.4 integration |
|
||||
| `redis-cache` | no | Redis caching layer |
|
||||
| `ib` | no | Interactive Brokers TWS adapter |
|
||||
| `mock` | no | Mock providers for testing |
|
||||
|
||||
## Testing
|
||||
|
||||
Run the included examples:
|
||||
|
||||
```bash
|
||||
# Basic connection test
|
||||
cargo run --example basic_connection
|
||||
|
||||
# Order submission test
|
||||
cargo run --example order_submission
|
||||
|
||||
# Market data test
|
||||
cargo run --example market_data
|
||||
|
||||
# Comprehensive workflow test
|
||||
cargo run --example comprehensive_trading
|
||||
SQLX_OFFLINE=true cargo test -p data --lib
|
||||
```
|
||||
|
||||
## Production Deployment
|
||||
|
||||
### Pre-Production Checklist
|
||||
|
||||
- [ ] Test with paper trading account extensively
|
||||
- [ ] Validate all order types and scenarios
|
||||
- [ ] Test reconnection logic
|
||||
- [ ] Verify error handling
|
||||
- [ ] Load test with expected message volume
|
||||
- [ ] Security review and IP restrictions
|
||||
- [ ] Monitoring and alerting setup
|
||||
|
||||
### Production Configuration
|
||||
|
||||
```rust
|
||||
let config = IBConfig {
|
||||
host: "127.0.0.1".to_string(),
|
||||
port: 7496, // Live trading port
|
||||
client_id: 2, // Use different client ID for production
|
||||
account_id: "U123456".to_string(), // Live account
|
||||
connection_timeout: 15, // Shorter timeout for production
|
||||
heartbeat_interval: 10, // More frequent heartbeats
|
||||
max_reconnect_attempts: 10, // More retry attempts
|
||||
request_timeout: 5, // Faster request timeout
|
||||
};
|
||||
```
|
||||
|
||||
### Monitoring
|
||||
|
||||
Implement monitoring for:
|
||||
- Connection status
|
||||
- Message processing latency
|
||||
- Order submission/execution rates
|
||||
- Error rates and types
|
||||
- Account balance and positions
|
||||
|
||||
## Support
|
||||
|
||||
For issues related to:
|
||||
- **TWS/Gateway setup**: Consult Interactive Brokers documentation
|
||||
- **API permissions**: Contact Interactive Brokers support
|
||||
- **Integration issues**: Check this documentation and examples
|
||||
- **Performance optimization**: Review configuration and architecture
|
||||
|
||||
## License
|
||||
|
||||
This implementation is provided as-is for educational and development purposes. Ensure compliance with Interactive Brokers terms of service and applicable regulations when using in production.
|
||||
81
ml/README.md
81
ml/README.md
@@ -1,74 +1,49 @@
|
||||
# `ml` Crate
|
||||
# ml
|
||||
|
||||
The `ml` crate provides the core machine learning capabilities for the Foxhunt High-Frequency Trading (HFT) System. It encompasses a suite of advanced models for sequence prediction, reinforcement learning, and time series analysis, optimized for low-latency inference and robust model management within a high-frequency trading environment.
|
||||
Machine learning models for Foxhunt.
|
||||
|
||||
## Features
|
||||
## Models
|
||||
|
||||
* **Advanced Model Suite**: Implementation of cutting-edge ML models tailored for HFT.
|
||||
* **Low-Latency Inference**: Highly optimized inference engine designed for real-time market data processing.
|
||||
* **GPU Acceleration**: Leverages CUDA/cuDNN for high-performance, GPU-accelerated model inference.
|
||||
* **Dynamic Model Management**: Supports hot-swapping and versioning of models for seamless updates.
|
||||
* **Cloud-Native Storage**: S3-based model storage and caching for reliable and scalable deployment.
|
||||
* **Experimentation & Monitoring**: Built-in support for A/B testing and performance monitoring of deployed models.
|
||||
- **DQN (Rainbow)** -- Deep Q-Network with prioritized experience replay, dueling heads, noisy nets, double Q-learning
|
||||
- **PPO** -- Proximal Policy Optimization with GAE, LSTM policies, clip-higher option
|
||||
- **TFT** -- Temporal Fusion Transformer for multi-horizon time series forecasting
|
||||
- **Mamba2** -- State space model for efficient sequence prediction
|
||||
- **Liquid Networks** -- Biologically inspired neural networks for non-stationary data
|
||||
- **TLOB** -- Transformer-based Limit Order Book analysis
|
||||
- **Flash Attention** -- Optimized attention implementation
|
||||
|
||||
## Models Implemented
|
||||
## Training
|
||||
|
||||
This crate includes specialized implementations of various machine learning models, each optimized for specific HFT challenges:
|
||||
Two paths per model:
|
||||
|
||||
* **MAMBA-2 State Space Models**: Efficient sequence prediction, crucial for forecasting market movements, order flow, or short-term price trajectories in dynamic HFT scenarios.
|
||||
* **Deep Q-Learning (DQN)**: A reinforcement learning algorithm for discovering and executing optimal trading strategies, learning directly from market rewards and penalties.
|
||||
* **Proximal Policy Optimization (PPO) with GAE**: A robust policy gradient reinforcement learning method, often employed for more complex, continuous action spaces in trading agents, offering stable and efficient learning.
|
||||
* **Temporal Fusion Transformer (TFT)**: An advanced transformer-based architecture for multivariate time series forecasting, adept at handling complex temporal dependencies and integrating exogenous variables for precise price or volume prediction.
|
||||
* **Liquid Networks**: Biologically inspired neural networks offering high adaptability and robustness to changing data distributions, making them suitable for the non-stationary and volatile nature of financial markets.
|
||||
* **Transformer-based Order Book (TLOB) Analysis**: Utilizes transformer architectures to process granular, high-dimensional order book data, identifying intricate patterns and predicting short-term price movements, liquidity shifts, or order imbalances.
|
||||
1. **Standalone trainer** -- direct training loop (e.g., `DQN::train`, `PpoTrainer`)
|
||||
2. **UnifiedTrainable adapter** -- wraps models for the hyperopt pipeline (e.g., `DQNTrainableAdapter`, `UnifiedTrainablePPO`)
|
||||
|
||||
## Architecture
|
||||
## Inference
|
||||
|
||||
The `ml` crate is designed with the following key architectural components to ensure performance, reliability, and maintainability:
|
||||
`InferenceAdapterBridge` connects models to the ensemble coordinator in `adaptive-strategy`. Each model exposes an `InferenceAdapter` trait for prediction.
|
||||
|
||||
* **Inference Bridge**: A dedicated, low-latency communication channel facilitating seamless prediction delivery from ML models to the core `trading_engine`.
|
||||
* **Model Registry**: A centralized service for managing, versioning, and deploying ML models. It supports hot-swapping, allowing new model versions to be deployed without service interruption.
|
||||
* **Performance Monitoring & Distillation**: Real-time tracking of model efficacy, latency, and resource utilization. Includes mechanisms for model distillation to create smaller, faster models suitable for extreme low-latency environments.
|
||||
* **Ensemble Methods**: Integrates capabilities for combining predictions from multiple models, often incorporating confidence scoring, to enhance overall prediction robustness and accuracy.
|
||||
## Backend
|
||||
|
||||
## Usage
|
||||
- **Candle v0.9.1** -- `VarMap`, `AdamW`, `loss.backward()`, `GradStore`, `opt.step(&grads)`
|
||||
- **CUDA required for training** -- tested on RTX 3050 Ti 4GB, max batch size 230
|
||||
- **CPU inference** supported
|
||||
|
||||
To use the `ml` crate, you'll typically interact with the `ModelRegistry` to load models and then use the `InferenceEngine` trait to make predictions.
|
||||
## Hyperopt
|
||||
|
||||
```rust
|
||||
use ml::{InferenceEngine, ModelRegistry};
|
||||
`ArgminOptimizer` (Particle Swarm Optimization) with per-model adapters:
|
||||
DQN, PPO, ContinuousPPO, TFT, Mamba2. Uses `ParameterSpace` trait for continuous parameter mapping.
|
||||
|
||||
#[tokio::main]
|
||||
async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
||||
// Initialize your application configuration
|
||||
let config = /* Your application configuration object */;
|
||||
## ModelType Enum
|
||||
|
||||
// Instantiate the ModelRegistry
|
||||
let registry = ModelRegistry::new(config).await?;
|
||||
15 variants: `CompactDQN`, `DistilledMicroNet`, `DQN`, `RainbowDQN`, `MAMBA`, `TFT`, `TGGN`, `LNN`, `TLOB`, `PPO`, `Transformer`, `Mamba`, `LiquidNet`, `TGNN`, `Ensemble`.
|
||||
|
||||
// Load a specific model by its identifier and version
|
||||
let model = registry.load_model("mamba2-v1.2.3").await?;
|
||||
## Key Modules
|
||||
|
||||
// Prepare the current market state or features for inference
|
||||
let market_state = /* Your current market state object */;
|
||||
|
||||
// Run inference using the loaded model
|
||||
let prediction = model.predict(&market_state).await?;
|
||||
|
||||
println!("Inference result: {:?}", prediction);
|
||||
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
`dqn`, `ppo`, `tft`, `mamba`, `liquid`, `tlob`, `flash_attention`, `ensemble`, `evaluation`, `inference`, `trainers`, `hyperopt`, `checkpoint`, `preprocessing`, `data_loaders`, `features`, `model_factory`, `training_pipeline`, `regime_detection`, `stress_testing`, `validation`, `bridge`, `common`, `metrics`.
|
||||
|
||||
## Testing
|
||||
|
||||
To run the tests for the `ml` crate, use the standard Cargo test command:
|
||||
|
||||
```bash
|
||||
cargo test --package ml
|
||||
SQLX_OFFLINE=true cargo test -p ml --lib # ~2009 tests
|
||||
```
|
||||
|
||||
## Documentation
|
||||
|
||||
Comprehensive API documentation for the `ml` crate can be found on [docs.rs/ml](https://docs.rs/ml).
|
||||
|
||||
@@ -1,80 +1,40 @@
|
||||
# Risk Management Crate
|
||||
# risk
|
||||
|
||||
## Overview
|
||||
Enterprise risk management for HFT.
|
||||
|
||||
The `risk` crate is the comprehensive risk management and compliance framework for the Foxhunt High-Frequency Trading (HFT) System. It is engineered to safeguard trading operations by providing robust tools for real-time risk assessment, limit enforcement, and regulatory adherence, which are critical for maintaining stability and integrity in fast-paced trading environments.
|
||||
## Kill Switch
|
||||
|
||||
## Features
|
||||
`AtomicKillSwitch` provides immediate trading cessation, coordinated via Redis. Supports local, remote, and Unix socket triggers.
|
||||
|
||||
* **Value at Risk (VaR) Calculation**: Supports multiple models including historical simulation, parametric (e.g., variance-covariance), and Monte Carlo methods to quantify potential financial losses.
|
||||
* **Position Tracking & Limits Enforcement**: Real-time monitoring of all trading positions and strict enforcement of pre-defined limits (e.g., notional, delta, gross/net exposure).
|
||||
* **Automated Circuit Breakers**: Mechanisms to automatically pause or restrict trading activities when predefined market volatility, price movement, or risk thresholds are breached.
|
||||
* **Multi-faceted Kill Switches**: Provides immediate cessation of trading operations via local, remote, and Unix socket-based triggers for emergency risk containment.
|
||||
* **Integrated Compliance Framework**: Embeds logic to ensure adherence to critical regulatory standards such as Sarbanes-Oxley (SOX), MiFID II, and best execution principles.
|
||||
* **Drawdown Monitoring & Prevention**: Continuous monitoring of portfolio performance to detect and prevent significant declines from peak equity, triggering alerts or automated actions.
|
||||
* **Advanced Stress Testing Capabilities**: Simulates extreme market conditions and hypothetical shocks to evaluate portfolio resilience and identify vulnerabilities.
|
||||
* **Kelly Criterion Position Sizing**: Implements the Kelly criterion for optimal bet sizing, aiming to maximize long-term capital growth by dynamically adjusting trade sizes.
|
||||
* **Emergency Response Coordination**: Facilitates structured shutdown, recovery, and communication protocols during critical risk events to ensure an efficient and controlled response.
|
||||
## Value at Risk
|
||||
|
||||
## Risk Components
|
||||
Four VaR methods plus Expected Shortfall:
|
||||
|
||||
The `risk` crate is composed of several specialized components working in concert to provide a holistic risk management solution:
|
||||
- Historical Simulation
|
||||
- Monte Carlo
|
||||
- Parametric (variance-covariance)
|
||||
- Expected Shortfall (CVaR)
|
||||
|
||||
* **VaR Engine**: Computes Value at Risk using configurable models, providing quantitative insights into market risk.
|
||||
* **Position Limiter**: Manages and enforces exposure limits across all trading instruments and strategies, preventing concentration risks.
|
||||
* **Circuit Breaker System**: A configurable system that monitors market and internal metrics, triggering pre-defined actions upon threshold breaches.
|
||||
* **Kill Switch Module**: Offers various interfaces (local API, remote RPC, Unix socket) for immediate, system-wide trading cessation in emergency scenarios.
|
||||
* **Compliance Module**: Integrates regulatory checks and reporting capabilities for standards like SOX and MiFID II, ensuring legal and ethical trading practices.
|
||||
* **Drawdown Monitor**: Continuously tracks P&L and equity curves, alerting or acting when predefined drawdown percentages are hit.
|
||||
* **Stress Tester**: A simulation environment to subject the portfolio to historical or hypothetical extreme market events.
|
||||
* **Kelly Sizer**: Dynamically calculates optimal position sizes based on the Kelly criterion, integrating with trading strategies.
|
||||
* **Emergency Coordinator**: Orchestrates the system's response to critical events, ensuring orderly shutdowns, data preservation, and communication.
|
||||
## Key Types
|
||||
|
||||
## Architecture
|
||||
| Type | Purpose |
|
||||
|------|---------|
|
||||
| `RiskEngine` | Central risk evaluation and enforcement |
|
||||
| `AtomicKillSwitch` | Atomic trading halt with Redis coordination |
|
||||
| `KellySizer` | Kelly criterion position sizing |
|
||||
| `StressTester` | Extreme market scenario simulation |
|
||||
| `ComplianceValidator` | SOX, MiFID II regulatory checks |
|
||||
| `DrawdownMonitor` | Peak-to-trough equity tracking |
|
||||
| `CircuitBreaker` | Threshold-based trading pauses |
|
||||
| `CorrelationMonitor` | Cross-asset correlation tracking |
|
||||
|
||||
The `risk` crate is designed with a clear separation of concerns, integrating seamlessly with other core components of the Foxhunt system:
|
||||
## Config Presets
|
||||
|
||||
* **Safety Coordinator**: Serves as the central hub for system-wide risk management. It aggregates risk signals, evaluates the overall risk posture, and orchestrates responses across the system.
|
||||
* **Position Limiter**: A dedicated component responsible for maintaining real-time tracking of all open positions and enforcing pre-configured exposure limits. It directly interfaces with the `trading_engine` to validate and potentially block orders.
|
||||
* **Trading Gate**: Acts as a critical pre-trade risk and compliance check layer. All outgoing orders from the `trading_engine` must pass through the Trading Gate for immediate validation against risk limits and regulatory rules before submission to exchanges.
|
||||
* **Integration with `trading_engine`**: Provides deep integration with the core `trading_engine` for intercepting order flow, receiving position updates, and exercising control over trade execution.
|
||||
* **Integration with `config`**: Leverages the system's `config` crate for dynamic loading, management, and hot-reloading of all risk parameters, limits, and compliance rules, ensuring flexibility and adaptability.
|
||||
|
||||
## Usage
|
||||
|
||||
To integrate the `risk` crate into your trading application:
|
||||
|
||||
```rust
|
||||
use risk::{RiskEngine, CircuitBreaker, KillSwitch};
|
||||
|
||||
#[tokio::main]
|
||||
async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
||||
let config = /* ... load your system configuration ... */;
|
||||
|
||||
// Initialize risk engine
|
||||
let risk_engine = RiskEngine::new(config).await?;
|
||||
|
||||
let order = /* ... create your trade order ... */;
|
||||
|
||||
// Check position limits before trade
|
||||
risk_engine.check_position_limit(&order).await?;
|
||||
|
||||
// Monitor drawdown
|
||||
let current_pnl = 1000.0;
|
||||
risk_engine.monitor_drawdown(current_pnl).await?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
- `development_config()` -- relaxed limits for local testing
|
||||
- `production_config()` -- 5ms safety check timeout, strict position limits
|
||||
|
||||
## Testing
|
||||
|
||||
To run the test suite for the `risk` crate:
|
||||
|
||||
```bash
|
||||
cargo test --package risk
|
||||
SQLX_OFFLINE=true cargo test -p risk --lib # ~209 tests
|
||||
```
|
||||
|
||||
## Documentation
|
||||
|
||||
For detailed API documentation, please refer to [docs.rs/risk](https://docs.rs/risk).
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
# API Gateway Performance Benchmarks
|
||||
|
||||
**Wave 71 Agent 4 Deliverable** - Comprehensive performance benchmarking suite for API Gateway authentication pipeline.
|
||||
> **Note:** Benchmark numbers from initial implementation. Re-verify after changes.
|
||||
|
||||
## Overview
|
||||
|
||||
This directory contains 5 comprehensive benchmark suites designed to validate the <10μs routing overhead target for the API Gateway's 8-layer authentication pipeline.
|
||||
This directory contains 5 benchmark suites designed to validate the <10us routing overhead target for the API Gateway's 8-layer authentication pipeline.
|
||||
|
||||
## Performance Targets
|
||||
|
||||
|
||||
@@ -368,13 +368,13 @@ services:
|
||||
|
||||
## Benchmarks
|
||||
|
||||
### Target Performance (Wave 70 Requirements)
|
||||
### Target Performance
|
||||
|
||||
- ✅ Routing overhead: <10μs (5-8μs typical)
|
||||
- ✅ Zero-copy forwarding: Implemented
|
||||
- ✅ Connection pooling: Via tonic::Channel
|
||||
- ✅ Circuit breaker: <10μs overhead
|
||||
- ✅ Streaming support: Zero-copy passthrough
|
||||
- Routing overhead: <10us (5-8us typical)
|
||||
- Zero-copy forwarding: Implemented
|
||||
- Connection pooling: Via tonic::Channel
|
||||
- Circuit breaker: <10us overhead
|
||||
- Streaming support: Zero-copy passthrough
|
||||
|
||||
### Measurement
|
||||
|
||||
@@ -403,12 +403,3 @@ println!("Routing latency: {}μs", latency.as_micros());
|
||||
- Server setup: `/services/api_gateway/src/grpc/server.rs`
|
||||
- Build configuration: `/services/api_gateway/build.rs`
|
||||
|
||||
## Wave 70 Agent 10 Deliverables
|
||||
|
||||
✅ ML Training Service proxy implemented
|
||||
✅ Zero-copy forwarding functional
|
||||
✅ Streaming support working
|
||||
✅ Health checking integration
|
||||
✅ Circuit breaker configured
|
||||
✅ <10μs routing overhead target met
|
||||
✅ Integration documentation complete
|
||||
|
||||
@@ -1,8 +1,10 @@
|
||||
# Rate Limiter Implementation - Wave 70 Agent 13
|
||||
# Rate Limiter Implementation
|
||||
|
||||
> **Note:** HTTP-level rate limiting for the web dashboard is now handled by `web-gateway` (3 tiers: auth 10/min, trading 200/min, compute 30/min). This document describes the gRPC-level rate limiting in api_gateway.
|
||||
|
||||
## Overview
|
||||
|
||||
Implemented a high-performance token bucket rate limiting system with Redis backend and in-memory caching optimized for HFT requirements.
|
||||
High-performance token bucket rate limiting system with Redis backend and in-memory caching optimized for HFT requirements.
|
||||
|
||||
## Architecture
|
||||
|
||||
@@ -347,29 +349,9 @@ Performance Summary:
|
||||
|
||||
## Status
|
||||
|
||||
✅ **Implementation Complete**
|
||||
Implementation complete. Key files:
|
||||
|
||||
- [x] Token bucket rate limiter implemented
|
||||
- [x] Redis Lua script for atomic operations
|
||||
- [x] In-memory caching for <50ns checks (measured: 25ns)
|
||||
- [x] Per-endpoint rate limit configs
|
||||
- [x] Integration points defined
|
||||
- [x] Performance benchmarks validated
|
||||
- [x] Documentation complete
|
||||
- `services/api_gateway/src/routing/rate_limiter.rs` - Token bucket implementation
|
||||
- `services/api_gateway/benches/rate_limiter_bench.rs` - Performance benchmarks
|
||||
- Unit tests included in `rate_limiter.rs`
|
||||
|
||||
**Performance Targets Met:**
|
||||
- ✅ Cache hit: 25ns (target <50ns)
|
||||
- ✅ Redis backend: <500μs (estimated)
|
||||
- ✅ Burst handling: 41ns per request
|
||||
- ✅ HFT scenario: 27ns per check
|
||||
|
||||
**Deliverables:**
|
||||
1. ✅ `services/api_gateway/src/routing/rate_limiter.rs` - Full implementation
|
||||
2. ✅ `services/api_gateway/benches/rate_limiter_bench.rs` - Performance validation
|
||||
3. ✅ Unit tests included in rate_limiter.rs
|
||||
4. ✅ Integration with AuthInterceptor documented
|
||||
5. ✅ This comprehensive documentation
|
||||
|
||||
## Wave 70 Agent 13 - Mission Accomplished
|
||||
|
||||
Rate limiting system is production-ready and exceeds all performance targets.
|
||||
|
||||
@@ -1,458 +1,48 @@
|
||||
# ML Training Service
|
||||
# ml_training_service
|
||||
|
||||
Production-ready ML training service for the Foxhunt HFT trading system. This service orchestrates model training jobs, manages GPU/CPU resources, and provides comprehensive progress tracking for financial ML models.
|
||||
Model training orchestration and lifecycle management for the Foxhunt HFT trading system. Manages training jobs for DQN, PPO, TFT, Mamba2, TLOB, and Liquid models with progress tracking, resource allocation, and model artifact storage.
|
||||
|
||||
## Building
|
||||
|
||||
```bash
|
||||
# Default (minimal features)
|
||||
cargo build --release -p ml_training_service
|
||||
|
||||
# With GPU acceleration (requires CUDA)
|
||||
cargo build --release -p ml_training_service --features gpu
|
||||
|
||||
# With mock training data (testing only, bypasses database)
|
||||
cargo build --release -p ml_training_service --features mock-data
|
||||
```
|
||||
|
||||
## Features
|
||||
|
||||
### 🚀 Core Capabilities
|
||||
- **Model Training Orchestration**: Manages training jobs for TLOB, MAMBA-2, DQN, PPO, Liquid, and TFT models
|
||||
- **Resource Management**: Intelligent GPU/CPU allocation with concurrent job limiting
|
||||
- **Real-time Progress Tracking**: Live streaming of training metrics and status updates
|
||||
- **Model Lifecycle Management**: From training initiation to artifact storage and retrieval
|
||||
- **Financial Safety Guarantees**: Built-in validation for financial data and model outputs
|
||||
| Feature | Default | Description |
|
||||
|-------------|---------|--------------------------------------------------|
|
||||
| `minimal` | Yes | Minimal ML feature set for financial models |
|
||||
| `gpu` | No | SIMD GPU acceleration (requires CUDA) |
|
||||
| `debug` | No | Additional debug logging |
|
||||
| `mock-data` | No | Use mock training data instead of PostgreSQL |
|
||||
|
||||
### 🏗️ Architecture
|
||||
- **gRPC API**: High-performance streaming API with type-safe protobuf definitions
|
||||
- **PostgreSQL Persistence**: Reliable job metadata and training history storage
|
||||
- **Flexible Storage**: Local filesystem or S3-compatible object storage for model artifacts
|
||||
- **Production Safety**: Comprehensive error handling, gradient safety, and NaN detection
|
||||
- **Monitoring Integration**: Prometheus metrics and structured logging
|
||||
## Configuration
|
||||
|
||||
### 📊 Supported Models
|
||||
The gRPC listen port is set via the `GRPC_PORT` environment variable. Prometheus metrics are exposed on port 9094.
|
||||
|
||||
| Model | Description | Estimated Training Time | GPU Required |
|
||||
|-------|-------------|------------------------|--------------|
|
||||
| **TLOB** | Time-Limit Order Book Transformer | 45 min | ✅ |
|
||||
| **MAMBA-2** | State Space Model for long sequences | 90 min | ✅ |
|
||||
| **DQN** | Deep Q-Network for RL trading | 120 min | ✅ |
|
||||
| **PPO** | Proximal Policy Optimization | 75 min | ✅ |
|
||||
| **Liquid** | Liquid Neural Network for regime detection | 60 min | ❌ |
|
||||
| **TFT** | Temporal Fusion Transformer | 100 min | ✅ |
|
||||
PostgreSQL (via sqlx) is used for job metadata, training history, and state management. Set the connection string with `DATABASE_URL`.
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Prerequisites
|
||||
- Rust 1.75+
|
||||
- PostgreSQL 12+
|
||||
- CUDA 12.0+ (for GPU acceleration)
|
||||
- Optional: S3-compatible storage
|
||||
|
||||
### Installation
|
||||
## Running
|
||||
|
||||
```bash
|
||||
# Clone the repository
|
||||
git clone https://github.com/user/foxhunt
|
||||
cd foxhunt
|
||||
|
||||
# Build the service (production - uses real data)
|
||||
cargo build --release -p ml_training_service
|
||||
|
||||
# Or build for testing with mock data
|
||||
cargo build --release -p ml_training_service --features mock-data
|
||||
|
||||
# Set up configuration
|
||||
cp config/ml_training_service.example.toml config/ml_training_service.toml
|
||||
# Edit configuration as needed
|
||||
|
||||
# Run database migrations
|
||||
./target/release/ml_training_service database migrate
|
||||
|
||||
# Start the service
|
||||
./target/release/ml_training_service serve
|
||||
GRPC_PORT=50053 DATABASE_URL="postgresql://user:pass@localhost:5432/foxhunt_training" \
|
||||
./target/release/ml_training_service serve
|
||||
```
|
||||
|
||||
### Feature Flags
|
||||
|
||||
The service supports the following Cargo feature flags:
|
||||
|
||||
| Feature | Default | Description |
|
||||
|---------|---------|-------------|
|
||||
| `minimal` | ✅ Yes | Minimal ML feature set for financial models |
|
||||
| `gpu` | ❌ No | Enable SIMD GPU acceleration (requires CUDA) |
|
||||
| `debug` | ❌ No | Enable debug mode with additional logging |
|
||||
| `mock-data` | ❌ No | **TESTING ONLY** - Use mock training data instead of database |
|
||||
|
||||
**Important**: The `mock-data` feature is for testing and development only. Production builds should use the default features which load real historical data from PostgreSQL.
|
||||
## Testing
|
||||
|
||||
```bash
|
||||
# Production build (default)
|
||||
cargo build --release -p ml_training_service
|
||||
# Unit tests (offline, no database required)
|
||||
SQLX_OFFLINE=true cargo test -p ml_training_service --lib
|
||||
|
||||
# Testing with mock data (bypasses database)
|
||||
cargo build --release -p ml_training_service --features mock-data
|
||||
|
||||
# GPU-accelerated build
|
||||
cargo build --release -p ml_training_service --features gpu
|
||||
```
|
||||
|
||||
### Configuration
|
||||
|
||||
```toml
|
||||
[server]
|
||||
host = "0.0.0.0"
|
||||
port = 50053
|
||||
max_concurrent_jobs = 4
|
||||
|
||||
[database]
|
||||
url = "postgresql://user:pass@localhost:5432/foxhunt_training"
|
||||
max_connections = 10
|
||||
|
||||
[training]
|
||||
default_device = "cuda"
|
||||
max_gpu_memory_gb = 8.0
|
||||
worker_threads = 4
|
||||
|
||||
[storage]
|
||||
storage_type = "local" # or "s3"
|
||||
local_base_path = "./models"
|
||||
|
||||
[monitoring]
|
||||
enable_prometheus = true
|
||||
prometheus_port = 9090
|
||||
```
|
||||
|
||||
## API Usage
|
||||
|
||||
### Starting a Training Job
|
||||
|
||||
```python
|
||||
import grpc
|
||||
from ml_training_pb2 import *
|
||||
from ml_training_pb2_grpc import MLTrainingServiceStub
|
||||
|
||||
# Connect to service
|
||||
channel = grpc.insecure_channel('localhost:50053')
|
||||
client = MLTrainingServiceStub(channel)
|
||||
|
||||
# Configure TLOB training
|
||||
request = StartTrainingRequest(
|
||||
model_type="TLOB",
|
||||
hyperparameters=Hyperparameters(
|
||||
tlob_params=TlobParams(
|
||||
epochs=100,
|
||||
learning_rate=0.001,
|
||||
batch_size=64,
|
||||
hidden_dim=256,
|
||||
num_heads=8
|
||||
)
|
||||
),
|
||||
use_gpu=True,
|
||||
description="TLOB training for EURUSD orderbook prediction"
|
||||
)
|
||||
|
||||
# Submit job
|
||||
response = client.StartTraining(request)
|
||||
job_id = response.job_id
|
||||
print(f"Training job started: {job_id}")
|
||||
```
|
||||
|
||||
### Monitoring Training Progress
|
||||
|
||||
```python
|
||||
# Subscribe to real-time updates
|
||||
status_request = SubscribeToTrainingStatusRequest(job_id=job_id)
|
||||
status_stream = client.SubscribeToTrainingStatus(status_request)
|
||||
|
||||
for update in status_stream:
|
||||
print(f"Epoch {update.current_epoch}/{update.total_epochs}")
|
||||
print(f"Progress: {update.progress_percentage:.1f}%")
|
||||
print(f"Loss: {update.metrics.get('loss', 0.0):.6f}")
|
||||
print(f"Sharpe Ratio: {update.financial_metrics.sharpe_ratio:.3f}")
|
||||
|
||||
if update.status == TrainingStatus.COMPLETED:
|
||||
print("Training completed successfully!")
|
||||
break
|
||||
```
|
||||
|
||||
### Listing Training Jobs
|
||||
|
||||
```python
|
||||
# List recent jobs
|
||||
jobs_request = ListTrainingJobsRequest(
|
||||
page=1,
|
||||
page_size=10,
|
||||
status_filter=TrainingStatus.COMPLETED
|
||||
)
|
||||
|
||||
jobs_response = client.ListTrainingJobs(jobs_request)
|
||||
for job in jobs_response.jobs:
|
||||
print(f"{job.job_id}: {job.model_type} - {job.status}")
|
||||
print(f" Final Loss: {job.final_loss:.6f}")
|
||||
print(f" Duration: {job.completed_at - job.started_at}")
|
||||
```
|
||||
|
||||
## CLI Usage
|
||||
|
||||
### Server Management
|
||||
|
||||
```bash
|
||||
# Start the service
|
||||
ml_training_service serve --config config.toml --port 50053
|
||||
|
||||
# Enable development mode with debug logging
|
||||
ml_training_service serve --dev
|
||||
|
||||
# Health check
|
||||
ml_training_service health --endpoint http://localhost:50053
|
||||
```
|
||||
|
||||
### Database Operations
|
||||
|
||||
```bash
|
||||
# Run migrations
|
||||
ml_training_service database migrate
|
||||
|
||||
# Check database health
|
||||
ml_training_service database health
|
||||
|
||||
# Clean up old jobs (retain 30 days)
|
||||
ml_training_service database cleanup --retain-days 30
|
||||
```
|
||||
|
||||
### Configuration Management
|
||||
|
||||
```bash
|
||||
# Validate configuration
|
||||
ml_training_service config --file config.toml
|
||||
```
|
||||
|
||||
## Integration with Existing ML Infrastructure
|
||||
|
||||
The service integrates seamlessly with the existing Foxhunt ML infrastructure:
|
||||
|
||||
### Training Pipeline Integration
|
||||
|
||||
```rust
|
||||
use ml::training_pipeline::{ProductionMLTrainingSystem, ProductionTrainingConfig};
|
||||
use ml::safety::{MLSafetyManager, GradientSafetyManager};
|
||||
|
||||
// The service orchestrates the existing training system
|
||||
let training_system = ProductionMLTrainingSystem::new(config).await?;
|
||||
let result = training_system.train_model(training_data, validation_data).await?;
|
||||
```
|
||||
|
||||
### Financial Feature Processing
|
||||
|
||||
```rust
|
||||
use ml::training_pipeline::{FinancialFeatures, MicrostructureFeatures, RiskFeatures};
|
||||
|
||||
// Financial features are validated and processed automatically
|
||||
let features = FinancialFeatures {
|
||||
prices: vec![IntegerPrice::from_f64(100.50)],
|
||||
volumes: vec![1000],
|
||||
technical_indicators: indicators,
|
||||
microstructure: MicrostructureFeatures { /* ... */ },
|
||||
risk_metrics: RiskFeatures { /* ... */ },
|
||||
timestamp: Utc::now(),
|
||||
};
|
||||
```
|
||||
|
||||
### Safety and Validation
|
||||
|
||||
```rust
|
||||
// Built-in safety guarantees
|
||||
- Gradient clipping and NaN detection
|
||||
- Financial data validation (positive prices, finite indicators)
|
||||
- Resource allocation limits
|
||||
- Training timeout protection
|
||||
- Model artifact integrity checks
|
||||
```
|
||||
|
||||
## Monitoring and Observability
|
||||
|
||||
### Prometheus Metrics
|
||||
|
||||
The service exposes comprehensive metrics on `:9090/metrics`:
|
||||
|
||||
```
|
||||
# Training job metrics
|
||||
ml_training_jobs_total{status="completed"} 45
|
||||
ml_training_jobs_total{status="running"} 2
|
||||
ml_training_jobs_total{status="failed"} 1
|
||||
|
||||
# Resource utilization
|
||||
ml_training_gpu_utilization_percent 78.5
|
||||
ml_training_memory_usage_bytes 4294967296
|
||||
|
||||
# Performance metrics
|
||||
ml_training_job_duration_seconds{model_type="TLOB"} 2700
|
||||
ml_training_final_loss{model_type="MAMBA_2"} 0.001234
|
||||
```
|
||||
|
||||
### Structured Logging
|
||||
|
||||
```json
|
||||
{
|
||||
"timestamp": "2025-01-21T10:30:45Z",
|
||||
"level": "INFO",
|
||||
"target": "ml_training_service::orchestrator",
|
||||
"message": "Training job completed successfully",
|
||||
"job_id": "550e8400-e29b-41d4-a716-446655440000",
|
||||
"model_type": "TLOB",
|
||||
"final_loss": 0.001234,
|
||||
"training_duration_secs": 2700,
|
||||
"epochs_completed": 100
|
||||
}
|
||||
```
|
||||
|
||||
## Performance Characteristics
|
||||
|
||||
### Throughput
|
||||
- **Concurrent Jobs**: Up to 4 simultaneous training jobs (configurable)
|
||||
- **Job Submission**: <10ms latency for job creation
|
||||
- **Status Updates**: Real-time streaming with <100ms latency
|
||||
- **Database Operations**: <5ms for job metadata queries
|
||||
|
||||
### Resource Usage
|
||||
- **Memory**: ~1-2GB base + 4-8GB per training job
|
||||
- **GPU Memory**: 4-8GB per GPU-accelerated job
|
||||
- **CPU**: 1-2 cores for orchestration + 4-8 cores per training job
|
||||
- **Storage**: Variable (10MB-1GB+ per model artifact)
|
||||
|
||||
### Scalability
|
||||
- **Horizontal**: Can run multiple service instances with shared database
|
||||
- **Vertical**: Scales with available GPU/CPU resources
|
||||
- **Storage**: Unlimited with S3-compatible backends
|
||||
- **Concurrent Clients**: 100+ simultaneous gRPC connections
|
||||
|
||||
## Security and Compliance
|
||||
|
||||
### Data Protection
|
||||
- **Encryption**: TLS 1.3 for gRPC communication
|
||||
- **Authentication**: Integration with Foxhunt auth system
|
||||
- **Audit Logging**: Complete training job audit trail
|
||||
- **Access Control**: Role-based access to training operations
|
||||
|
||||
### Financial Compliance
|
||||
- **Model Validation**: Automatic financial data sanity checks
|
||||
- **Reproducibility**: Complete training configuration persistence
|
||||
- **Model Governance**: Artifact integrity and versioning
|
||||
- **Risk Controls**: Automated position sizing validation
|
||||
|
||||
## Development
|
||||
|
||||
### Building from Source
|
||||
|
||||
```bash
|
||||
# Development build
|
||||
cargo build -p ml_training_service
|
||||
|
||||
# Release build
|
||||
cargo build --release -p ml_training_service
|
||||
|
||||
# Run tests
|
||||
# Integration tests (requires running PostgreSQL)
|
||||
cargo test -p ml_training_service
|
||||
|
||||
# Run with debug logging
|
||||
RUST_LOG=debug cargo run -p ml_training_service -- serve --dev
|
||||
```
|
||||
|
||||
### Testing
|
||||
|
||||
```bash
|
||||
# Unit tests
|
||||
cargo test -p ml_training_service
|
||||
|
||||
# Integration tests (requires database)
|
||||
cargo test -p ml_training_service --features integration-tests
|
||||
|
||||
# End-to-end tests
|
||||
cargo test -p ml_training_service --test e2e
|
||||
```
|
||||
|
||||
### gRPC Development
|
||||
|
||||
```bash
|
||||
# Generate protobuf code
|
||||
cargo build -p ml_training_service
|
||||
|
||||
# Test with grpcurl
|
||||
grpcurl -plaintext localhost:50053 list
|
||||
grpcurl -plaintext localhost:50053 ml_training.MLTrainingService/HealthCheck
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
#### Service Won't Start
|
||||
```bash
|
||||
# Check configuration
|
||||
ml_training_service config --file config.toml
|
||||
|
||||
# Check database connectivity
|
||||
ml_training_service database health
|
||||
|
||||
# Check port availability
|
||||
lsof -i :50053
|
||||
```
|
||||
|
||||
#### Training Jobs Fail
|
||||
```bash
|
||||
# Check GPU availability
|
||||
nvidia-smi
|
||||
|
||||
# Check logs for detailed error messages
|
||||
tail -f /var/log/ml_training_service.log
|
||||
|
||||
# Verify model artifacts storage
|
||||
ls -la ./models/
|
||||
```
|
||||
|
||||
#### Performance Issues
|
||||
```bash
|
||||
# Check resource utilization
|
||||
htop
|
||||
|
||||
# Monitor GPU usage
|
||||
watch -n 1 nvidia-smi
|
||||
|
||||
# Check database performance
|
||||
EXPLAIN ANALYZE SELECT * FROM training_jobs WHERE status = 'running';
|
||||
```
|
||||
|
||||
### Debugging
|
||||
|
||||
```bash
|
||||
# Enable debug logging
|
||||
export RUST_LOG=ml_training_service=debug
|
||||
|
||||
# Enable trace logging for specific modules
|
||||
export RUST_LOG=ml_training_service::orchestrator=trace
|
||||
|
||||
# Profile memory usage
|
||||
valgrind --tool=massif target/release/ml_training_service serve
|
||||
```
|
||||
|
||||
## Contributing
|
||||
|
||||
### Code Style
|
||||
- Follow Rust standard formatting (`cargo fmt`)
|
||||
- Add documentation for public APIs
|
||||
- Include comprehensive error handling
|
||||
- Write tests for new functionality
|
||||
|
||||
### Pull Request Process
|
||||
1. Create feature branch from `main`
|
||||
2. Implement changes with tests
|
||||
3. Update documentation
|
||||
4. Submit PR with clear description
|
||||
|
||||
### Performance Testing
|
||||
```bash
|
||||
# Benchmark training job throughput
|
||||
cargo run --release --bin bench_training_service
|
||||
|
||||
# Load test gRPC API
|
||||
ghz --insecure --proto proto/ml_training.proto --call ml_training.MLTrainingService/HealthCheck localhost:50053
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
Licensed under either of Apache License, Version 2.0 or MIT license at your option.
|
||||
|
||||
## Support
|
||||
|
||||
- **Documentation**: [docs.rs/foxhunt](https://docs.rs/foxhunt)
|
||||
- **Issues**: [GitHub Issues](https://github.com/user/foxhunt/issues)
|
||||
- **Discussions**: [GitHub Discussions](https://github.com/user/foxhunt/discussions)
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
The `tune` command module provides a comprehensive CLI interface for managing ML model hyperparameter tuning jobs through the API Gateway. It supports starting, monitoring, and stopping Optuna-based hyperparameter optimization runs for all supported ML models (DQN, PPO, MAMBA_2, TLOB, TFT, LIQUID).
|
||||
|
||||
**Location**: `/home/jgrusewski/Work/foxhunt/tli/src/commands/tune.rs`
|
||||
**Location**: `tli/src/commands/tune.rs`
|
||||
|
||||
## Features
|
||||
|
||||
@@ -317,7 +317,7 @@ Benefits:
|
||||
❌ Config file not found: tuning_config.yaml
|
||||
|
||||
💡 Create a tuning configuration file with search space definition
|
||||
See: /home/jgrusewski/Work/foxhunt/docs/ml_training/tuning_config_example.yaml
|
||||
See: docs/ml_training/tuning_config_example.yaml
|
||||
```
|
||||
|
||||
## Implementation Status
|
||||
@@ -339,7 +339,7 @@ Benefits:
|
||||
- ⚠️ Actual trial history display
|
||||
- ⚠️ JWT metadata forwarding
|
||||
|
||||
**Note**: The module is **fully functional** with mock data and will work seamlessly once the API Gateway adds tuning proxy methods (see Wave 152+ roadmap).
|
||||
**Note**: The module is **fully functional** with mock data and will work seamlessly once the API Gateway adds tuning proxy methods.
|
||||
|
||||
## API Gateway Integration Checklist
|
||||
|
||||
@@ -373,7 +373,7 @@ To enable the tune command, the API Gateway needs these proxy methods:
|
||||
|
||||
### Proto Definitions
|
||||
|
||||
Already defined in `/home/jgrusewski/Work/foxhunt/services/ml_training_service/proto/ml_training.proto` (lines 35-44, 131-209).
|
||||
Already defined in `services/ml_training_service/proto/ml_training.proto` (lines 35-44, 131-209).
|
||||
|
||||
### Client Integration Points
|
||||
|
||||
@@ -463,14 +463,5 @@ tabled = "0.15"
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- **CLAUDE.md** - System architecture and service topology
|
||||
- **ml_training.proto** - gRPC service definitions
|
||||
- **TESTING_PLAN.md** - ML testing strategy
|
||||
- **Wave 152+ Roadmap** - Planned API Gateway tuning proxy implementation
|
||||
|
||||
---
|
||||
|
||||
**Created**: 2025-10-13
|
||||
**Author**: Claude (Anthropic)
|
||||
**Status**: ✅ Ready for API Gateway integration
|
||||
**Wave**: Post-151 (awaiting Wave 152 API Gateway proxy update)
|
||||
|
||||
@@ -59,21 +59,13 @@ tracing-subscriber = "0.3"
|
||||
|
||||
### Creating a Client
|
||||
|
||||
All traffic routes through the api_gateway at port 50050.
|
||||
|
||||
```rust
|
||||
use tli::{TliClient, ServiceEndpoints};
|
||||
use tli::TliClient;
|
||||
|
||||
// Default endpoints (from environment variables or defaults)
|
||||
// Default: connects to api_gateway at localhost:50050
|
||||
let client = TliClient::new();
|
||||
|
||||
// Custom endpoints
|
||||
let endpoints = ServiceEndpoints {
|
||||
trading_engine: "http://localhost:50052".to_string(),
|
||||
risk_management: "http://localhost:50053".to_string(),
|
||||
ml_signals: "http://localhost:50054".to_string(),
|
||||
market_data: "http://localhost:50055".to_string(),
|
||||
health_check: "http://localhost:50056".to_string(),
|
||||
};
|
||||
let client = TliClient::with_endpoints(endpoints);
|
||||
```
|
||||
|
||||
### Connecting to Services
|
||||
@@ -228,12 +220,8 @@ for metric in response.into_inner().metrics {
|
||||
TLI can be configured using environment variables:
|
||||
|
||||
```bash
|
||||
# Service endpoints
|
||||
export FOXHUNT_TRADING_ENGINE_URL="http://localhost:50052"
|
||||
export FOXHUNT_RISK_MANAGEMENT_URL="http://localhost:50053"
|
||||
export FOXHUNT_ML_SIGNALS_URL="http://localhost:50054"
|
||||
export FOXHUNT_MARKET_DATA_URL="http://localhost:50055"
|
||||
export FOXHUNT_HEALTH_CHECK_URL="http://localhost:50056"
|
||||
# API Gateway endpoint (all services route through here)
|
||||
export FOXHUNT_API_GATEWAY_URL="http://localhost:50050"
|
||||
|
||||
# Logging
|
||||
export RUST_LOG="info"
|
||||
@@ -248,8 +236,8 @@ export TLI_ENABLE_RT_TESTS="true"
|
||||
use std::time::Duration;
|
||||
use tonic::transport::Endpoint;
|
||||
|
||||
// Custom timeout configuration (when building endpoints manually)
|
||||
let channel = Endpoint::from_shared("http://localhost:50052")?
|
||||
// Custom timeout configuration
|
||||
let channel = Endpoint::from_shared("http://localhost:50050")?
|
||||
.timeout(Duration::from_secs(10))
|
||||
.connect_timeout(Duration::from_secs(5))
|
||||
.connect()
|
||||
@@ -267,15 +255,6 @@ use tli::prelude::*;
|
||||
// Main client
|
||||
pub struct TliClient { /* ... */ }
|
||||
|
||||
// Service endpoints configuration
|
||||
pub struct ServiceEndpoints {
|
||||
pub trading_engine: String,
|
||||
pub risk_management: String,
|
||||
pub ml_signals: String,
|
||||
pub market_data: String,
|
||||
pub health_check: String,
|
||||
}
|
||||
|
||||
// Error types
|
||||
pub enum TliError {
|
||||
Connection(String),
|
||||
@@ -293,7 +272,6 @@ pub type TliResult<T> = Result<T, TliError>;
|
||||
impl TliClient {
|
||||
// Creation
|
||||
pub fn new() -> Self;
|
||||
pub fn with_endpoints(endpoints: ServiceEndpoints) -> Self;
|
||||
|
||||
// Connection management
|
||||
pub async fn connect(&mut self) -> TliResult<()>;
|
||||
@@ -329,27 +307,6 @@ pub fn string_to_order_type(order_type: &str) -> TliResult<OrderType>;
|
||||
|
||||
## Examples
|
||||
|
||||
### Basic Dashboard
|
||||
|
||||
Run the basic dashboard example:
|
||||
|
||||
```bash
|
||||
# Basic dashboard with default settings
|
||||
cargo run --example basic_dashboard
|
||||
|
||||
# Custom configuration
|
||||
cargo run --example basic_dashboard custom
|
||||
|
||||
# Connection test only
|
||||
cargo run --example basic_dashboard test
|
||||
```
|
||||
|
||||
Features:
|
||||
- Real-time system status display
|
||||
- Order and position counts
|
||||
- Automatic reconnection
|
||||
- Terminal-based UI
|
||||
|
||||
### Configuration Management
|
||||
|
||||
Run the configuration management example:
|
||||
@@ -503,15 +460,14 @@ let results = futures::future::join_all(health_checks).await;
|
||||
Error: Connection(Connection refused)
|
||||
```
|
||||
|
||||
**Solution**: Ensure Foxhunt services are running and accessible:
|
||||
**Solution**: Ensure the api_gateway is running and accessible:
|
||||
|
||||
```bash
|
||||
# Check if services are running
|
||||
curl http://localhost:50052/health
|
||||
netstat -ln | grep 50052
|
||||
# Check if api_gateway is running
|
||||
netstat -ln | grep 50050
|
||||
|
||||
# Check environment variables
|
||||
echo $FOXHUNT_TRADING_ENGINE_URL
|
||||
echo $FOXHUNT_API_GATEWAY_URL
|
||||
```
|
||||
|
||||
#### Service Not Connected
|
||||
@@ -556,12 +512,15 @@ Error: Connection(Request timeout)
|
||||
**Solution**: Increase timeouts or check network connectivity:
|
||||
|
||||
```rust
|
||||
// Custom timeout configuration
|
||||
let endpoints = ServiceEndpoints {
|
||||
trading_engine: "http://slow-server:50052".to_string(),
|
||||
// ... other endpoints
|
||||
};
|
||||
let client = TliClient::with_endpoints(endpoints);
|
||||
// Custom timeout configuration via tonic Endpoint
|
||||
use tonic::transport::Endpoint;
|
||||
use std::time::Duration;
|
||||
|
||||
let channel = Endpoint::from_shared("http://localhost:50050")?
|
||||
.timeout(Duration::from_secs(30))
|
||||
.connect_timeout(Duration::from_secs(10))
|
||||
.connect()
|
||||
.await?;
|
||||
```
|
||||
|
||||
### Debug Logging
|
||||
@@ -571,11 +530,9 @@ Enable debug logging for troubleshooting:
|
||||
```bash
|
||||
# Enable debug logs
|
||||
export RUST_LOG=debug
|
||||
cargo run --example basic_dashboard
|
||||
|
||||
# Trace-level logging (very verbose)
|
||||
export RUST_LOG=trace
|
||||
cargo run --example basic_dashboard
|
||||
```
|
||||
|
||||
```rust
|
||||
@@ -800,6 +757,3 @@ When contributing to TLI:
|
||||
4. Run clippy: `cargo clippy`
|
||||
5. Update documentation as needed
|
||||
|
||||
---
|
||||
|
||||
*Last updated: 2025-01-21*
|
||||
@@ -1,73 +1,53 @@
|
||||
# React + TypeScript + Vite
|
||||
# web-dashboard
|
||||
|
||||
This template provides a minimal setup to get React working in Vite with HMR and some ESLint rules.
|
||||
Foxhunt trading dashboard.
|
||||
|
||||
Currently, two official plugins are available:
|
||||
## Stack
|
||||
|
||||
- [@vitejs/plugin-react](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react) uses [Babel](https://babeljs.io/) (or [oxc](https://oxc.rs) when used in [rolldown-vite](https://vite.dev/guide/rolldown)) for Fast Refresh
|
||||
- [@vitejs/plugin-react-swc](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react-swc) uses [SWC](https://swc.rs/) for Fast Refresh
|
||||
- React 19 + TypeScript
|
||||
- Vite 7 (dev server and bundler)
|
||||
- Tailwind CSS 4
|
||||
- TradingView (lightweight-charts 5.1.0) + Recharts
|
||||
- Zustand (state) + React Query (server state, staleTime 10s, refetchInterval 15s)
|
||||
|
||||
## React Compiler
|
||||
## Pages
|
||||
|
||||
The React Compiler is not enabled on this template because of its impact on dev & build performances. To add it, see [this documentation](https://react.dev/learn/react-compiler/installation).
|
||||
1. **Trading** -- Live positions, orders, market data charts
|
||||
2. **Risk** -- VaR, drawdown, portfolio exposure
|
||||
3. **ML** -- Model status, training metrics, predictions
|
||||
4. **Performance** -- PnL, Sharpe, execution quality
|
||||
5. **Backtesting** -- Strategy replay configuration and results
|
||||
6. **Config** -- System settings, strategy parameters
|
||||
|
||||
## Expanding the ESLint configuration
|
||||
## Auth
|
||||
|
||||
If you are developing a production application, we recommend updating the configuration to enable type-aware lint rules:
|
||||
Login page with JWT authentication. `ProtectedRoute` guards on all pages. `ErrorBoundary` on every route. 401 responses trigger automatic logout.
|
||||
|
||||
```js
|
||||
export default defineConfig([
|
||||
globalIgnores(['dist']),
|
||||
{
|
||||
files: ['**/*.{ts,tsx}'],
|
||||
extends: [
|
||||
// Other configs...
|
||||
## Development
|
||||
|
||||
// Remove tseslint.configs.recommended and replace with this
|
||||
tseslint.configs.recommendedTypeChecked,
|
||||
// Alternatively, use this for stricter rules
|
||||
tseslint.configs.strictTypeChecked,
|
||||
// Optionally, add this for stylistic rules
|
||||
tseslint.configs.stylisticTypeChecked,
|
||||
|
||||
// Other configs...
|
||||
],
|
||||
languageOptions: {
|
||||
parserOptions: {
|
||||
project: ['./tsconfig.node.json', './tsconfig.app.json'],
|
||||
tsconfigRootDir: import.meta.dirname,
|
||||
},
|
||||
// other options...
|
||||
},
|
||||
},
|
||||
])
|
||||
```bash
|
||||
npm install
|
||||
npm run dev
|
||||
```
|
||||
|
||||
You can also install [eslint-plugin-react-x](https://github.com/Rel1cx/eslint-react/tree/main/packages/plugins/eslint-plugin-react-x) and [eslint-plugin-react-dom](https://github.com/Rel1cx/eslint-react/tree/main/packages/plugins/eslint-plugin-react-dom) for React-specific lint rules:
|
||||
Dev server runs on `localhost:5173` and proxies API requests to `web-gateway` at `:3000`.
|
||||
|
||||
```js
|
||||
// eslint.config.js
|
||||
import reactX from 'eslint-plugin-react-x'
|
||||
import reactDom from 'eslint-plugin-react-dom'
|
||||
## Build
|
||||
|
||||
export default defineConfig([
|
||||
globalIgnores(['dist']),
|
||||
{
|
||||
files: ['**/*.{ts,tsx}'],
|
||||
extends: [
|
||||
// Other configs...
|
||||
// Enable lint rules for React
|
||||
reactX.configs['recommended-typescript'],
|
||||
// Enable lint rules for React DOM
|
||||
reactDom.configs.recommended,
|
||||
],
|
||||
languageOptions: {
|
||||
parserOptions: {
|
||||
project: ['./tsconfig.node.json', './tsconfig.app.json'],
|
||||
tsconfigRootDir: import.meta.dirname,
|
||||
},
|
||||
// other options...
|
||||
},
|
||||
},
|
||||
])
|
||||
```bash
|
||||
npm run build
|
||||
```
|
||||
|
||||
Production builds disable sourcemaps and use manual chunk splitting (vendor, charts, query).
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
src/
|
||||
pages/ -- Page components (Trading, Risk, ML, etc.)
|
||||
components/ -- Shared UI components, charts, tables
|
||||
hooks/ -- React Query hooks, WebSocket hooks
|
||||
stores/ -- Zustand stores
|
||||
api/ -- API client, auth helpers
|
||||
types/ -- TypeScript type definitions
|
||||
```
|
||||
|
||||
58
web-gateway/README.md
Normal file
58
web-gateway/README.md
Normal file
@@ -0,0 +1,58 @@
|
||||
# web-gateway
|
||||
|
||||
REST and WebSocket gateway for the Foxhunt web dashboard. Built with Axum 0.7.9, it proxies HTTP requests to backend gRPC services and bridges gRPC streams to WebSocket clients.
|
||||
|
||||
## Configuration
|
||||
|
||||
All settings are read from environment variables.
|
||||
|
||||
| Variable | Default | Description |
|
||||
|---|---|---|
|
||||
| `GATEWAY_LISTEN_ADDR` | `0.0.0.0:3000` | HTTP listen address |
|
||||
| `JWT_SECRET` | _(required, min 32 chars)_ | HMAC key for JWT validation |
|
||||
| `CORS_ORIGINS` | `http://localhost:5173` | Comma-separated allowed origins |
|
||||
| `TRADING_SERVICE_URL` | `https://localhost:50051` | Trading gRPC upstream |
|
||||
| `BACKTESTING_SERVICE_URL` | `https://localhost:50052` | Backtesting gRPC upstream |
|
||||
| `ML_TRAINING_SERVICE_URL` | `https://localhost:50053` | ML training gRPC upstream |
|
||||
|
||||
## Routes
|
||||
|
||||
All API routes are under `/api`. Authentication is required unless noted.
|
||||
|
||||
| Tier | Rate Limit | Endpoints |
|
||||
|---|---|---|
|
||||
| Public | 10 req/min | `POST /api/auth/login` |
|
||||
| Trading | 200 req/min | `/api/trading/*`, `/api/risk/*`, `/api/ml/*`, `/api/performance/*`, `/api/config/*` |
|
||||
| Compute | 30 req/min | `/api/training/*`, `/api/backtest/*`, `/api/tune/*` |
|
||||
| WebSocket | 50 msg/min | `GET /api/ws` (JWT via `token` query param) |
|
||||
| Health | none | `GET /health`, `GET /ready` |
|
||||
|
||||
## Security
|
||||
|
||||
- JWT authentication middleware (32-character minimum secret enforced at startup)
|
||||
- Per-IP rate limiting across three tiers
|
||||
- 1 MB request body size limit
|
||||
- CORS with explicit methods (`GET`, `POST`, `PUT`, `DELETE`) and headers (`Authorization`, `Content-Type`)
|
||||
- Response headers: HSTS, `x-frame-options: DENY`, `x-content-type-options: nosniff`, `referrer-policy: strict-origin-when-cross-origin`
|
||||
- WebSocket topic whitelist (8 valid topics), per-connection rate limit, max 20 subscriptions
|
||||
- `X-Request-Id` propagation (generated if absent, echoed on response)
|
||||
|
||||
## Modules
|
||||
|
||||
| Module | Purpose |
|
||||
|---|---|
|
||||
| `auth` | JWT validation and login handler |
|
||||
| `config` | `GatewayConfig` loaded from environment |
|
||||
| `error` | Unified error types and HTTP error responses |
|
||||
| `grpc` | Tonic clients and gRPC stream bridges |
|
||||
| `rate_limit` | Per-IP token-bucket rate limiter (3 tiers) |
|
||||
| `routes` | Axum router construction and all HTTP handlers |
|
||||
| `state` | `AppState` (gRPC channels, WS broadcast, config) |
|
||||
| `ws` | WebSocket upgrade, topic subscriptions, keepalive |
|
||||
|
||||
## Running
|
||||
|
||||
```sh
|
||||
export JWT_SECRET="your-secret-at-least-32-characters-long"
|
||||
cargo run -p web-gateway
|
||||
```
|
||||
Reference in New Issue
Block a user