Files
foxhunt/docs/runbooks/service-restart.md
jgrusewski 7458f1be01 feat(wave12): E2E validation complete - 225-feature pipeline ready
 Validation Results:
- PPO training: 24.2s (1 epoch, 950 samples, dim=225)
- Feature extraction: 105μs/bar (9.5x faster than target)
- Model checkpoint: 293KB (147KB actor + 146KB critic)
- GPU memory: 145MB used (96.4% headroom)
- Zero dimension mismatches

📊 Success Criteria (5/5):
 Feature dimension = 225 (Wave C 201 + Wave D 24)
 Model state_dim = 225
 Training completed without errors
 Checkpoint saved successfully
 No dimension mismatch errors

📁 Training Data Ready:
- ES.FUT: 2.9MB, 180 days
- NQ.FUT: 4.4MB, 180 days
- 6E.FUT: 2.8MB, 180 days
- ZN.FUT: 65KB, 90 days (clean)

🚀 Next: Full production model retraining (4 models, ~10min GPU time)

🤖 Generated with Claude Code (https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-22 22:48:04 +02:00

1.4 KiB

Service Restart Runbook

Last Updated: 2025-10-22 Expected Downtime: 0-30 seconds per service


Graceful Restart Procedure

Single Service Restart

# Kubernetes rolling restart (zero downtime)
kubectl rollout restart deployment/foxhunt-trading-service -n foxhunt

# Wait for completion
kubectl rollout status deployment/foxhunt-trading-service -n foxhunt

# Verify health
kubectl get pods -n foxhunt -l app.kubernetes.io/component=trading-service

All Services Restart (Ordered)

# Order: Infrastructure → Backend → Frontend
services=(
    "foxhunt-postgresql"
    "foxhunt-redis"
    "foxhunt-trading-service"
    "foxhunt-backtesting-service"
    "foxhunt-ml-training-service"
    "foxhunt-trading-agent"
    "foxhunt-api-gateway"
)

for svc in "${services[@]}"; do
    echo "Restarting $svc..."
    kubectl rollout restart deployment/$svc -n foxhunt 2>/dev/null || \
    kubectl rollout restart statefulset/$svc -n foxhunt 2>/dev/null
    kubectl rollout status deployment/$svc -n foxhunt 2>/dev/null || \
    kubectl rollout status statefulset/$svc -n foxhunt 2>/dev/null
    sleep 10
done

Emergency Restart (Immediate)

# Delete pods (forces immediate recreation)
kubectl delete pods -n foxhunt -l app.kubernetes.io/component=trading-service

# Monitor recreation
watch kubectl get pods -n foxhunt -l app.kubernetes.io/component=trading-service

End of Service Restart Runbook