# Grafana Setup Guide

**Last Updated**: 2025-10-22
**Version**: Grafana 10.1+

---

## Installation

```bash
# Grafana is installed with kube-prometheus-stack
# Access Grafana
kubectl port-forward -n monitoring svc/prometheus-grafana 3000:80

# Get admin password
kubectl get secret -n monitoring prometheus-grafana -o jsonpath="{.data.admin-password}" | base64 --decode
```

---

## Dashboard Import

### Foxhunt Trading Dashboard

```json
{
  "dashboard": {
    "title": "Foxhunt Trading System",
    "panels": [
      {
        "title": "Order Submission Rate",
        "targets": [
          {
            "expr": "rate(orders_submitted_total[5m])"
          }
        ]
      },
      {
        "title": "API Gateway Latency P99",
        "targets": [
          {
            "expr": "histogram_quantile(0.99, rate(grpc_request_duration_seconds_bucket{service=\"api-gateway\"}[5m]))"
          }
        ]
      },
      {
        "title": "Active Positions",
        "targets": [
          {
            "expr": "positions_open_total"
          }
        ]
      },
      {
        "title": "Database Connection Pool Usage",
        "targets": [
          {
            "expr": "pg_stat_database_numbackends / pg_settings_max_connections * 100"
          }
        ]
      }
    ]
  }
}
```

---

## Alert Configuration

### Critical Alerts

```yaml
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: foxhunt-alerts
  namespace: foxhunt
spec:
  groups:
  - name: foxhunt-critical
    interval: 30s
    rules:
    - alert: HighErrorRate
      expr: rate(grpc_requests_total{code!~"2.."}[5m]) > 0.1
      for: 5m
      labels:
        severity: critical
      annotations:
        summary: "High error rate detected"
        description: "Error rate is {{ $value | humanizePercentage }}"

    - alert: HighLatency
      expr: histogram_quantile(0.99, rate(grpc_request_duration_seconds_bucket[5m])) > 1
      for: 5m
      labels:
        severity: warning
      annotations:
        summary: "High latency detected"
        description: "P99 latency is {{ $value }}s"

    - alert: DatabaseDown
      expr: up{job="postgresql"} == 0
      for: 1m
      labels:
        severity: critical
      annotations:
        summary: "PostgreSQL is down"
```

---

**End of Grafana Setup Guide**