Files
foxhunt/docker-compose.yml
jgrusewski df64dbc04c 🚀 Wave 127 Phase 2: Protocol Translation + E2E Infrastructure (Agents 168-172)
## Summary
Major architectural fixes enabling E2E testing through protocol translation layer
and complete infrastructure resolution. Trading Service confirmed 100% implemented.

## Agents 168-172 Achievements

**Agent 168** - Port Configuration Fix:
- Fixed 3-layer port mismatch (tests→API Gateway→backends)
- Test files: localhost:50051 → localhost:50050
- Result: Infrastructure 100% correct, E2E testing unblocked

**Agent 169** - Root Cause Discovery:
- Confirmed Trading Service 100% implemented (all 11 methods exist)
- Identified protocol mismatch as root cause (TLI↔Trading proto)
- Documented all method implementations and field mappings

**Agent 170** - Protocol Translation Implementation:
- Implemented TLI↔Trading proto translation layer (+227 lines)
- Phase 2: 5 core methods (submit_order, cancel_order, get_order_status, get_account_info, get_positions)
- Phase 4: 2 streaming methods (subscribe_market_data, subscribe_order_updates)
- Dual proto compilation setup in build.rs

**Agent 171** - Backend Port Fix:
- Fixed API Gateway backend URLs (50051→50052, 50052→50053)
- Discovered authentication forwarding blocker
- Validated port connectivity working

**Agent 172** - Authentication Forwarding:
- Implemented auth metadata forwarding for all 7 translated methods
- Fixed gRPC Request ownership patterns (metadata clone before into_inner)
- Updated E2E test JWT secret for compliance (88-char base64)

## Files Modified

### API Gateway
- `services/api_gateway/build.rs`: Dual proto compilation
- `services/api_gateway/src/grpc/trading_proxy.rs`: +227 lines (translation + auth)
- `services/api_gateway/src/main.rs`: Port configuration
- `services/api_gateway/src/auth/interceptor.rs`: JWT validation
- `services/api_gateway/src/grpc/backtesting_proxy.rs`: Port updates

### Integration Tests
- `services/integration_tests/tests/trading_service_e2e.rs`: Port + JWT fixes
- `services/integration_tests/tests/backtesting_service_e2e.rs`: Port fixes
- `services/integration_tests/tests/ml_training_service_e2e.rs`: Port fixes

### Other Services
- `services/backtesting_service/src/main.rs`: Port configuration
- Multiple test files: Compliance, risk, pipeline tests

## Test Status
- E2E baseline: 6/54 (11.1%)
- Infrastructure: 100% fixed
- Protocol translation: Implemented, validation pending JWT sync
- Expected after validation: 13/54 (24.1%) with 7 methods working

## Technical Achievements
- Protocol adapter pattern (TLI↔Trading proto)
- gRPC metadata forwarding (5 auth headers)
- Dual proto compilation architecture
- Stream translation with unfold pattern
- Zero-copy enum pass-through

## Remaining Work
- JWT secret synchronization (in progress)
- Agent 170 Phase 5: 15 extended methods
- ML Training Service startup
- Backtesting Service route implementation (9 methods)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-08 19:35:59 +02:00

308 lines
8.9 KiB
YAML

version: '3.8'
services:
# PostgreSQL - Primary database for SQLx compilation and app data
postgres:
image: timescale/timescaledb:latest-pg16
container_name: foxhunt-postgres
environment:
POSTGRES_DB: foxhunt
POSTGRES_USER: foxhunt
POSTGRES_PASSWORD: foxhunt_dev_password
ports:
- "5432:5432"
volumes:
- postgres_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U foxhunt"]
interval: 10s
timeout: 5s
retries: 5
networks:
- foxhunt-network
# Redis - Caching and real-time data
redis:
image: redis:7-alpine
container_name: foxhunt-redis
ports:
- "6379:6379"
volumes:
- redis_data:/data
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 10s
timeout: 5s
retries: 5
networks:
- foxhunt-network
# InfluxDB - Time-series data for HFT metrics
influxdb:
image: influxdb:2.7-alpine
container_name: foxhunt-influxdb
environment:
DOCKER_INFLUXDB_INIT_MODE: setup
DOCKER_INFLUXDB_INIT_USERNAME: foxhunt
DOCKER_INFLUXDB_INIT_PASSWORD: foxhunt_dev_password
DOCKER_INFLUXDB_INIT_ORG: foxhunt
DOCKER_INFLUXDB_INIT_BUCKET: trading_metrics
DOCKER_INFLUXDB_INIT_RETENTION: 30d
ports:
- "8086:8086"
volumes:
- influxdb_data:/var/lib/influxdb2
healthcheck:
test: ["CMD", "influx", "ping"]
interval: 30s
timeout: 10s
retries: 5
networks:
- foxhunt-network
# HashiCorp Vault - Secrets management
vault:
image: hashicorp/vault:1.15
container_name: foxhunt-vault
environment:
VAULT_ADDR: http://0.0.0.0:8200
VAULT_DEV_ROOT_TOKEN_ID: foxhunt-dev-root
ports:
- "8200:8200"
volumes:
- vault_data:/vault/data
cap_add:
- IPC_LOCK
command: vault server -dev -dev-listen-address=0.0.0.0:8200
healthcheck:
test: ["CMD", "vault", "status"]
interval: 30s
timeout: 10s
retries: 5
networks:
- foxhunt-network
# Prometheus - HFT Metrics Collection
prometheus:
image: prom/prometheus:latest
container_name: foxhunt-prometheus
ports:
- "9090:9090"
volumes:
- prometheus_data:/prometheus
- ./config/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:ro
- ./config/prometheus/rules:/etc/prometheus/rules:ro
command:
- '--config.file=/etc/prometheus/prometheus.yml'
- '--storage.tsdb.path=/prometheus'
- '--storage.tsdb.retention.time=15d'
- '--web.enable-lifecycle'
- '--query.max-concurrency=50'
healthcheck:
test: ["CMD", "wget", "--no-verbose", "--tries=1", "--spider", "http://localhost:9090/-/healthy"]
interval: 30s
timeout: 10s
retries: 5
networks:
- foxhunt-network
# Grafana - HFT Trading Dashboards
grafana:
image: grafana/grafana:latest
container_name: foxhunt-grafana
ports:
- "3000:3000"
volumes:
- grafana_data:/var/lib/grafana
- ./config/grafana/dashboards:/var/lib/grafana/dashboards:ro
- ./config/grafana/provisioning:/etc/grafana/provisioning:ro
environment:
- GF_SECURITY_ADMIN_PASSWORD=foxhunt123
- GF_USERS_ALLOW_SIGN_UP=false
- GF_DASHBOARDS_DEFAULT_HOME_DASHBOARD_PATH=/var/lib/grafana/dashboards/hft-trading-performance.json
depends_on:
prometheus:
condition: service_healthy
healthcheck:
test: ["CMD-SHELL", "wget --no-verbose --tries=1 --spider http://localhost:3000/api/health || exit 1"]
interval: 30s
timeout: 10s
retries: 5
networks:
- foxhunt-network
# =========================================================================
# Application Services (gRPC microservices)
# =========================================================================
# Trading Service - Core trading logic (port 50052)
trading_service:
build:
context: .
dockerfile: services/trading_service/Dockerfile
container_name: foxhunt-trading-service
ports:
- "50052:50051" # Map external 50052 to internal 50051
- "9092:9092" # Metrics
environment:
- DATABASE_URL=postgresql://foxhunt:foxhunt_dev_password@postgres:5432/foxhunt
- REDIS_URL=redis://redis:6379
- VAULT_ADDR=http://vault:8200
- VAULT_TOKEN=foxhunt-dev-root
- RUST_LOG=info
- RUST_BACKTRACE=1
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
vault:
condition: service_healthy
healthcheck:
test: ["CMD", "/usr/local/bin/grpc_health_probe", "-addr=localhost:50051"]
interval: 10s
timeout: 5s
start_period: 30s
retries: 3
networks:
- foxhunt-network
restart: unless-stopped
# Backtesting Service - Strategy testing (port 50053)
backtesting_service:
build:
context: .
dockerfile: services/backtesting_service/Dockerfile
container_name: foxhunt-backtesting-service
ports:
- "50053:50052" # Map external 50053 to internal 50052
- "9093:9093" # Metrics
- "8083:8080" # Health check endpoint
environment:
- DATABASE_URL=postgresql://foxhunt:foxhunt_dev_password@postgres:5432/foxhunt
- REDIS_URL=redis://redis:6379
- VAULT_ADDR=http://vault:8200
- VAULT_TOKEN=foxhunt-dev-root
- BENZINGA_API_KEY=${BENZINGA_API_KEY:-demo_key_please_replace}
- RUST_LOG=info
- RUST_BACKTRACE=1
volumes:
- /tmp/foxhunt/certs:/tmp/foxhunt/certs:ro
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
vault:
condition: service_healthy
healthcheck:
test: ["CMD", "/usr/local/bin/grpc_health_probe", "-addr=localhost:50052"]
interval: 10s
timeout: 5s
start_period: 30s
retries: 3
networks:
- foxhunt-network
restart: unless-stopped
# ML Training Service - Model training (port 50054)
ml_training_service:
build:
context: .
dockerfile: services/ml_training_service/Dockerfile
container_name: foxhunt-ml-training-service
runtime: nvidia
ports:
- "50054:50053" # Map external 50054 to internal 50053
- "9094:9094" # Metrics
- "8095:8080" # Health endpoint (unique host port)
environment:
- DATABASE_URL=postgresql://foxhunt:foxhunt_dev_password@postgres:5432/foxhunt
- REDIS_URL=redis://redis:6379
- VAULT_ADDR=http://vault:8200
- VAULT_TOKEN=foxhunt-dev-root
- RUST_LOG=info
- RUST_BACKTRACE=1
- NVIDIA_VISIBLE_DEVICES=all
- NVIDIA_DRIVER_CAPABILITIES=compute,utility
- CUDA_VISIBLE_DEVICES=0
volumes:
- /tmp/foxhunt/certs:/tmp/foxhunt/certs:ro
- /tmp/foxhunt/models:/tmp/foxhunt/models
- /tmp/foxhunt/checkpoints:/tmp/foxhunt/checkpoints
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
vault:
condition: service_healthy
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
interval: 10s
timeout: 5s
start_period: 30s
retries: 3
networks:
- foxhunt-network
restart: unless-stopped
# API Gateway - Auth + routing (port 50051)
api_gateway:
build:
context: .
dockerfile: services/api_gateway/Dockerfile
container_name: foxhunt-api-gateway
ports:
- "50051:50050" # Map external 50051 to internal 50050
- "9091:9091" # Metrics
environment:
- GATEWAY_BIND_ADDR=0.0.0.0:50050
- DATABASE_URL=postgresql://foxhunt:foxhunt_dev_password@postgres:5432/foxhunt
- REDIS_URL=redis://redis:6379
- VAULT_ADDR=http://vault:8200
- VAULT_TOKEN=foxhunt-dev-root
- TRADING_SERVICE_URL=http://trading_service:50051
- BACKTESTING_SERVICE_URL=http://backtesting_service:50052
- ML_TRAINING_SERVICE_URL=http://ml_training_service:50053
- JWT_SECRET=dev_secret_key_change_in_production
- JWT_ISSUER=foxhunt-api-gateway
- JWT_AUDIENCE=foxhunt-services
- RATE_LIMIT_RPS=100
- ENABLE_AUDIT_LOGGING=true
- RUST_LOG=info
- RUST_BACKTRACE=1
volumes:
- /tmp/foxhunt/certs:/tmp/foxhunt/certs:ro
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
vault:
condition: service_healthy
trading_service:
condition: service_healthy
backtesting_service:
condition: service_started
healthcheck:
test: ["CMD", "/usr/local/bin/grpc_health_probe", "-addr=localhost:50050"]
interval: 10s
timeout: 5s
start_period: 30s
retries: 3
networks:
- foxhunt-network
restart: unless-stopped
volumes:
postgres_data:
redis_data:
influxdb_data:
vault_data:
prometheus_data:
grafana_data:
networks:
foxhunt-network:
driver: bridge