foxhunt/docs/DOCKER_DEPLOYMENT.md

# 🐳 Foxhunt HFT System - Docker Deployment

## 🎯 One-Click Deployment

We've simplified the Docker deployment from **20+ Dockerfiles and 12 docker-compose files** to a **single, unified solution**.

### Quick Start

```bash
# Deploy everything with one command:
./deploy.sh

# That's it! 🎉
```

## 📋 What Gets Deployed

### Core Services (8)
- **Integration Hub** - Service discovery and coordination (port 50051)
- **Market Data** - Real-time market data with Polygon integration (port 50052)
- **Trading Engine** - Order execution and management (port 50053)
- **Risk Management** - Real-time risk controls (port 50054)
- **Backtesting** - Strategy validation (port 50055)
- **AI Intelligence** - ML/AI predictions (port 50056)
- **Broker Connector** - Broker integrations (port 50057)
- **Persistence** - Data storage service (port 50058)

### Databases (3)
- **PostgreSQL** - Transactional data (port 5432)
- **Redis** - Caching and sessions (port 6379)
- **InfluxDB** - Time-series market data (port 8086)

### Monitoring (2)
- **Prometheus** - Metrics collection (port 9090)
- **Grafana** - Dashboards (port 3000)

## 🚀 Deployment Strategy

### Simplification Achieved
**Before:**
- 20+ individual Dockerfiles
- 12 different docker-compose files
- Complex Kubernetes manifests
- Unclear deployment process

**After:**
- 1 unified docker-compose.dev.yml
- 1 deploy.sh script
- Uses pre-built executables
- Single command deployment

### Architecture Decisions

1. **Use Pre-Built Binaries**
   - Services are compiled with `cargo build --release`
   - Binaries mounted into lightweight `rust:slim` containers
   - No need to rebuild Docker images for code changes

2. **Shared Base Image**
   - All services use `rust:1.89-slim`
   - Reduces total image size
   - Faster deployment

3. **Simple Networking**
   - Single bridge network for all services
   - Services communicate via hostname (e.g., `market-data:50052`)
   - Ports exposed to host for development access

4. **Development-First**
   - All services accessible from host
   - Logs easily viewable
   - Quick restart capability

## 🔧 Usage Commands

### Basic Operations
```bash
# Start all services
./deploy.sh

# View logs for all services
./deploy.sh logs

# View logs for specific service
./deploy.sh logs trading-engine

# Check service status
./deploy.sh status

# Stop all services
./deploy.sh stop

# Clean everything (including data)
./deploy.sh clean
```

### Direct Docker Commands
```bash
# Start specific services
docker-compose -f docker-compose.dev.yml up -d trading-engine market-data

# Restart a service
docker-compose -f docker-compose.dev.yml restart trading-engine

# Scale a service
docker-compose -f docker-compose.dev.yml up -d --scale backtesting=3

# Execute command in container
docker exec -it foxhunt-trading-engine /bin/bash
```

## 🌐 Service Endpoints

### gRPC Services
| Service | Port | Endpoint |
|---------|------|----------|
| Integration Hub | 50051 | `localhost:50051` |
| Market Data | 50052 | `localhost:50052` |
| Trading Engine | 50053 | `localhost:50053` |
| Risk Management | 50054 | `localhost:50054` |
| Backtesting | 50055 | `localhost:50055` |
| AI Intelligence | 50056 | `localhost:50056` |
| Broker Connector | 50057 | `localhost:50057` |
| Persistence | 50058 | `localhost:50058` |

### Web Interfaces
| Service | URL | Credentials |
|---------|-----|-------------|
| Trading Engine API | http://localhost:8080 | - |
| Grafana | http://localhost:3000 | admin/admin |
| Prometheus | http://localhost:9090 | - |
| InfluxDB | http://localhost:8086 | foxhunt/foxhunt-dev |

## 🔍 Testing the Deployment

### 1. Check All Services Running
```bash
./deploy.sh status
```

### 2. Test gRPC Connectivity
```bash
# Install grpcurl if needed
brew install grpcurl  # macOS
# or
go install github.com/fullstorydev/grpcurl/cmd/grpcurl@latest

# Test Integration Hub
grpcurl -plaintext localhost:50051 list

# Test Market Data service
grpcurl -plaintext localhost:50052 list
```

### 3. Test Trading Engine HTTP API
```bash
curl http://localhost:8080/health
```

### 4. Check Database Connectivity
```bash
# PostgreSQL
docker exec -it foxhunt-postgres psql -U foxhunt -d foxhunt -c "SELECT 1"

# Redis
docker exec -it foxhunt-redis redis-cli -a foxhunt-dev ping

# InfluxDB
curl http://localhost:8086/health
```

## 🔐 Environment Variables

Create a `.env` file in the project root:
```env
# API Keys
POLYGON_API_KEY=your-actual-api-key

# Logging
RUST_LOG=info

# Database URLs (for local testing outside Docker)
DATABASE_URL=postgres://foxhunt:foxhunt-dev@localhost:5432/foxhunt
REDIS_URL=redis://:foxhunt-dev@localhost:6379
INFLUXDB_URL=http://localhost:8086
INFLUXDB_TOKEN=foxhunt-dev-token
```

## 🚧 Production Considerations

This deployment is optimized for **development and testing**. For production:

1. **Use Kubernetes**: The existing Helm charts in `/ops/kubernetes/` are production-ready
2. **Separate Docker Images**: Build optimized images for each service
3. **External Secrets**: Use Kubernetes secrets or Vault
4. **Network Policies**: Implement strict network segmentation
5. **Resource Limits**: Set CPU/memory limits for each service
6. **Monitoring**: Full Prometheus/Grafana stack with alerting
7. **High Availability**: Multiple replicas with load balancing

## 📊 Resource Requirements

### Minimum (Development)
- **CPU**: 4 cores
- **RAM**: 8 GB
- **Disk**: 20 GB

### Recommended (Testing)
- **CPU**: 8 cores
- **RAM**: 16 GB
- **Disk**: 50 GB

## 🐛 Troubleshooting

### Services Won't Start
```bash
# Check logs
docker-compose -f docker-compose.dev.yml logs [service-name]

# Rebuild executables
cargo build --release --workspace

# Reset everything
./deploy.sh clean
./deploy.sh
```

### Port Conflicts
```bash
# Find what's using a port
lsof -i :50051

# Change ports in docker-compose.dev.yml
```

### Database Issues
```bash
# Reset databases
docker-compose -f docker-compose.dev.yml down -v
docker-compose -f docker-compose.dev.yml up -d postgres redis influxdb
```

## ✅ Summary

We've transformed a complex multi-file Docker setup into a **single-command deployment**:

- **1 docker-compose file** instead of 12
- **1 deploy script** for all operations
- **0 Docker builds** needed (uses compiled binaries)
- **13 services** deployed and networked
- **100% local development ready**

Just run `./deploy.sh` and your entire HFT system is ready for testing! 🚀