Files
foxhunt/docs/monitoring/prometheus-setup.md
jgrusewski 7458f1be01 feat(wave12): E2E validation complete - 225-feature pipeline ready
 Validation Results:
- PPO training: 24.2s (1 epoch, 950 samples, dim=225)
- Feature extraction: 105μs/bar (9.5x faster than target)
- Model checkpoint: 293KB (147KB actor + 146KB critic)
- GPU memory: 145MB used (96.4% headroom)
- Zero dimension mismatches

📊 Success Criteria (5/5):
 Feature dimension = 225 (Wave C 201 + Wave D 24)
 Model state_dim = 225
 Training completed without errors
 Checkpoint saved successfully
 No dimension mismatch errors

📁 Training Data Ready:
- ES.FUT: 2.9MB, 180 days
- NQ.FUT: 4.4MB, 180 days
- 6E.FUT: 2.8MB, 180 days
- ZN.FUT: 65KB, 90 days (clean)

🚀 Next: Full production model retraining (4 models, ~10min GPU time)

🤖 Generated with Claude Code (https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-22 22:48:04 +02:00

101 lines
2.0 KiB
Markdown

# Prometheus Setup Guide
**Last Updated**: 2025-10-22
**Version**: Prometheus 2.45+
---
## Installation
```bash
# Install Prometheus Operator
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
helm install prometheus prometheus-community/kube-prometheus-stack \
--namespace monitoring \
--create-namespace \
--set prometheus.prometheusSpec.retention=30d \
--set prometheus.prometheusSpec.storageSpec.volumeClaimTemplate.spec.resources.requests.storage=100Gi
```
---
## Configuration
### Service Monitors
```yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: foxhunt-services
namespace: foxhunt
spec:
selector:
matchLabels:
app: foxhunt
endpoints:
- port: metrics
interval: 15s
path: /metrics
```
### Scrape Configuration
```yaml
scrape_configs:
- job_name: 'foxhunt-api-gateway'
kubernetes_sd_configs:
- role: pod
namespaces:
names:
- foxhunt
relabel_configs:
- source_labels: [__meta_kubernetes_pod_label_app_kubernetes_io_component]
action: keep
regex: api-gateway
- source_labels: [__meta_kubernetes_pod_name]
target_label: pod
```
---
## Key Metrics
### Trading Service Metrics
```promql
# Request rate
rate(grpc_requests_total{service="trading-service"}[5m])
# Error rate
rate(grpc_requests_total{service="trading-service",code!~"2.."}[5m])
# Latency P99
histogram_quantile(0.99, rate(grpc_request_duration_seconds_bucket{service="trading-service"}[5m]))
# Order submission rate
rate(orders_submitted_total[5m])
# Position count
positions_open_total
```
### Database Metrics
```promql
# Connection pool usage
pg_stat_database_numbackends / pg_settings_max_connections * 100
# Query duration P95
histogram_quantile(0.95, rate(pg_stat_statements_mean_exec_time_bucket[5m]))
# Cache hit ratio
pg_stat_database_blks_hit / (pg_stat_database_blks_hit + pg_stat_database_blks_read) * 100
```
---
**End of Prometheus Setup Guide**