Files
foxhunt/docs/deployment/cloud-deployment.md
jgrusewski 7458f1be01 feat(wave12): E2E validation complete - 225-feature pipeline ready
 Validation Results:
- PPO training: 24.2s (1 epoch, 950 samples, dim=225)
- Feature extraction: 105μs/bar (9.5x faster than target)
- Model checkpoint: 293KB (147KB actor + 146KB critic)
- GPU memory: 145MB used (96.4% headroom)
- Zero dimension mismatches

📊 Success Criteria (5/5):
 Feature dimension = 225 (Wave C 201 + Wave D 24)
 Model state_dim = 225
 Training completed without errors
 Checkpoint saved successfully
 No dimension mismatch errors

📁 Training Data Ready:
- ES.FUT: 2.9MB, 180 days
- NQ.FUT: 4.4MB, 180 days
- 6E.FUT: 2.8MB, 180 days
- ZN.FUT: 65KB, 90 days (clean)

🚀 Next: Full production model retraining (4 models, ~10min GPU time)

🤖 Generated with Claude Code (https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-22 22:48:04 +02:00

384 lines
9.2 KiB
Markdown

# Cloud Deployment Guide
**Last Updated**: 2025-10-22
**Supported Platforms**: AWS, GCP, Azure
**Prerequisites**: Cloud CLI, Terraform 1.5+
**Deployment Time**: 45-60 minutes
---
## AWS EKS Deployment
### Prerequisites
```bash
# Install AWS CLI
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip && sudo ./aws/install
# Configure AWS credentials
aws configure
# AWS Access Key ID: YOUR_ACCESS_KEY
# AWS Secret Access Key: YOUR_SECRET_KEY
# Default region: us-east-1
# Install eksctl
curl --silent --location "https://github.com/weibaohui/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp
sudo mv /tmp/eksctl /usr/local/bin
```
### Create EKS Cluster
```bash
# Create cluster configuration
cat > foxhunt-eks-cluster.yaml <<EOF
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
name: foxhunt-prod
region: us-east-1
version: "1.28"
vpc:
cidr: 10.0.0.0/16
nat:
gateway: HighlyAvailable
nodeGroups:
- name: app-nodes
instanceType: c5.4xlarge
desiredCapacity: 6
minSize: 3
maxSize: 12
volumeSize: 100
volumeType: gp3
labels:
workload-type: application
tags:
Environment: production
Project: foxhunt
- name: data-nodes
instanceType: r5.2xlarge
desiredCapacity: 3
minSize: 2
maxSize: 6
volumeSize: 500
volumeType: gp3
labels:
workload-type: data
taints:
- key: workload-type
value: data
effect: NoSchedule
- name: gpu-nodes
instanceType: g4dn.xlarge # T4 GPU
desiredCapacity: 1
minSize: 0
maxSize: 3
volumeSize: 100
labels:
workload-type: gpu
accelerator: nvidia-t4
taints:
- key: nvidia.com/gpu
value: present
effect: NoSchedule
addons:
- name: vpc-cni
- name: coredns
- name: kube-proxy
- name: aws-ebs-csi-driver
iam:
withOIDC: true
serviceAccounts:
- metadata:
name: ebs-csi-controller-sa
namespace: kube-system
wellKnownPolicies:
ebsCSIController: true
cloudWatch:
clusterLogging:
enableTypes: ["api", "audit", "authenticator", "controllerManager", "scheduler"]
EOF
# Create cluster (30-40 minutes)
eksctl create cluster -f foxhunt-eks-cluster.yaml
# Verify cluster
kubectl get nodes
# NAME STATUS ROLES AGE VERSION
# ip-10-0-1-100.ec2.internal Ready <none> 5m v1.28.0
# ip-10-0-2-100.ec2.internal Ready <none> 5m v1.28.0
# ip-10-0-3-100.ec2.internal Ready <none> 5m v1.28.0
```
### Configure RDS PostgreSQL
```bash
# Create RDS subnet group
aws rds create-db-subnet-group \
--db-subnet-group-name foxhunt-db-subnet \
--db-subnet-group-description "Foxhunt PostgreSQL subnet group" \
--subnet-ids subnet-abc123 subnet-def456 subnet-ghi789
# Create RDS instance (TimescaleDB via custom AMI or Aurora PostgreSQL)
aws rds create-db-instance \
--db-instance-identifier foxhunt-postgres \
--db-instance-class db.r5.2xlarge \
--engine postgres \
--engine-version 15.4 \
--master-username foxhunt \
--master-user-password ${DB_PASSWORD} \
--allocated-storage 500 \
--storage-type gp3 \
--storage-encrypted \
--vpc-security-group-ids sg-abc123 \
--db-subnet-group-name foxhunt-db-subnet \
--backup-retention-period 30 \
--preferred-backup-window "03:00-04:00" \
--multi-az
# Wait for instance to be available
aws rds wait db-instance-available --db-instance-identifier foxhunt-postgres
# Get endpoint
aws rds describe-db-instances \
--db-instance-identifier foxhunt-postgres \
--query 'DBInstances[0].Endpoint.Address' \
--output text
# foxhunt-postgres.abc123.us-east-1.rds.amazonaws.com
```
### Configure ElastiCache Redis
```bash
# Create Redis cluster
aws elasticache create-replication-group \
--replication-group-id foxhunt-redis \
--replication-group-description "Foxhunt Redis cluster" \
--engine redis \
--cache-node-type cache.r5.large \
--num-cache-clusters 3 \
--automatic-failover-enabled \
--multi-az-enabled \
--cache-subnet-group-name foxhunt-cache-subnet \
--security-group-ids sg-def456 \
--at-rest-encryption-enabled \
--transit-encryption-enabled \
--auth-token ${REDIS_PASSWORD}
# Wait for cluster to be available
aws elasticache wait replication-group-available \
--replication-group-id foxhunt-redis
# Get endpoint
aws elasticache describe-replication-groups \
--replication-group-id foxhunt-redis \
--query 'ReplicationGroups[0].NodeGroups[0].PrimaryEndpoint.Address' \
--output text
# foxhunt-redis.abc123.cache.amazonaws.com
```
### Deploy Application
```bash
# Update Helm values for AWS
cat > values-aws-prod.yaml <<EOF
global:
environment: production
imageRegistry: <AWS_ACCOUNT_ID>.dkr.ecr.us-east-1.amazonaws.com
postgresql:
enabled: false # Using RDS
externalHost: foxhunt-postgres.abc123.us-east-1.rds.amazonaws.com
externalPort: 5432
redis:
enabled: false # Using ElastiCache
externalHost: foxhunt-redis.abc123.cache.amazonaws.com
externalPort: 6379
ingress:
enabled: true
className: alb
annotations:
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-type: ip
alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-1:123456789012:certificate/abc-123
EOF
# Deploy Helm chart
helm install foxhunt ./helm/foxhunt \
--namespace foxhunt \
--create-namespace \
--values values-aws-prod.yaml \
--timeout 15m \
--wait
# Verify deployment
kubectl get pods -n foxhunt
kubectl get ingress -n foxhunt
```
---
## GCP GKE Deployment
### Create GKE Cluster
```bash
# Set project
gcloud config set project foxhunt-prod
# Create cluster
gcloud container clusters create foxhunt-prod \
--zone us-central1-a \
--num-nodes 6 \
--machine-type n1-standard-8 \
--disk-type pd-ssd \
--disk-size 100 \
--enable-autoscaling \
--min-nodes 3 \
--max-nodes 12 \
--enable-stackdriver-kubernetes \
--addons HorizontalPodAutoscaling,HttpLoadBalancing,GcePersistentDiskCsiDriver
# Add GPU node pool
gcloud container node-pools create gpu-pool \
--cluster foxhunt-prod \
--zone us-central1-a \
--machine-type n1-standard-4 \
--accelerator type=nvidia-tesla-t4,count=1 \
--num-nodes 1 \
--min-nodes 0 \
--max-nodes 3 \
--enable-autoscaling
# Get credentials
gcloud container clusters get-credentials foxhunt-prod --zone us-central1-a
```
### Configure Cloud SQL PostgreSQL
```bash
# Create Cloud SQL instance
gcloud sql instances create foxhunt-postgres \
--database-version=POSTGRES_15 \
--tier=db-custom-8-32768 \
--region=us-central1 \
--storage-size=500 \
--storage-type=SSD \
--storage-auto-increase \
--backup \
--backup-start-time=03:00 \
--maintenance-window-day=SUN \
--maintenance-window-hour=04
# Set password
gcloud sql users set-password postgres \
--instance=foxhunt-postgres \
--password=${DB_PASSWORD}
# Get connection name
gcloud sql instances describe foxhunt-postgres \
--format='value(connectionName)'
# foxhunt-prod:us-central1:foxhunt-postgres
```
---
## Azure AKS Deployment
### Create AKS Cluster
```bash
# Create resource group
az group create --name foxhunt-prod --location eastus
# Create AKS cluster
az aks create \
--resource-group foxhunt-prod \
--name foxhunt-prod-aks \
--node-count 6 \
--node-vm-size Standard_D8s_v3 \
--enable-cluster-autoscaler \
--min-count 3 \
--max-count 12 \
--enable-addons monitoring \
--generate-ssh-keys
# Add GPU node pool
az aks nodepool add \
--resource-group foxhunt-prod \
--cluster-name foxhunt-prod-aks \
--name gpupool \
--node-count 1 \
--node-vm-size Standard_NC4as_T4_v3 \
--enable-cluster-autoscaler \
--min-count 0 \
--max-count 3
# Get credentials
az aks get-credentials --resource-group foxhunt-prod --name foxhunt-prod-aks
```
### Configure Azure Database for PostgreSQL
```bash
# Create PostgreSQL server
az postgres flexible-server create \
--resource-group foxhunt-prod \
--name foxhunt-postgres \
--location eastus \
--admin-user foxhunt \
--admin-password ${DB_PASSWORD} \
--sku-name Standard_D8s_v3 \
--tier GeneralPurpose \
--storage-size 512 \
--version 15 \
--high-availability Enabled
# Get connection string
az postgres flexible-server show \
--resource-group foxhunt-prod \
--name foxhunt-postgres \
--query "fullyQualifiedDomainName" \
--output tsv
# foxhunt-postgres.postgres.database.azure.com
```
---
## Cost Optimization
### AWS Cost Estimates
| Resource | Type | Monthly Cost |
|----------|------|--------------|
| EKS Control Plane | 1 cluster | $73 |
| EC2 App Nodes | 6x c5.4xlarge | $3,672 |
| EC2 Data Nodes | 3x r5.2xlarge | $1,814 |
| EC2 GPU Nodes | 1x g4dn.xlarge | $526 |
| RDS PostgreSQL | db.r5.2xlarge Multi-AZ | $1,248 |
| ElastiCache Redis | 3x cache.r5.large | $621 |
| Data Transfer | 1TB/month | $90 |
| **Total** | | **~$8,044/month** |
### Cost Optimization Strategies
1. **Reserved Instances**: Save 40-60% on EC2/RDS costs
2. **Spot Instances**: Use for non-critical backtesting workloads
3. **Auto-scaling**: Scale down during off-hours (weekends, nights)
4. **Storage Optimization**: Use gp3 instead of io1 for general workloads
5. **Right-sizing**: Monitor actual usage and adjust instance types
---
**End of Cloud Deployment Guide**