7458f1be01
✅ Validation Results: - PPO training: 24.2s (1 epoch, 950 samples, dim=225) - Feature extraction: 105μs/bar (9.5x faster than target) - Model checkpoint: 293KB (147KB actor + 146KB critic) - GPU memory: 145MB used (96.4% headroom) - Zero dimension mismatches 📊 Success Criteria (5/5): ✅ Feature dimension = 225 (Wave C 201 + Wave D 24) ✅ Model state_dim = 225 ✅ Training completed without errors ✅ Checkpoint saved successfully ✅ No dimension mismatch errors 📁 Training Data Ready: - ES.FUT: 2.9MB, 180 days - NQ.FUT: 4.4MB, 180 days - 6E.FUT: 2.8MB, 180 days - ZN.FUT: 65KB, 90 days (clean) 🚀 Next: Full production model retraining (4 models, ~10min GPU time) 🤖 Generated with Claude Code (https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
87 lines
1.7 KiB
Markdown
87 lines
1.7 KiB
Markdown
# Alerting Rules Guide
|
|
|
|
**Last Updated**: 2025-10-22
|
|
|
|
---
|
|
|
|
## Alert Severity Levels
|
|
|
|
| Severity | Response Time | Notification |
|
|
|----------|---------------|--------------|
|
|
| Critical | 5 min | PagerDuty + Slack + Phone |
|
|
| Warning | 30 min | Slack |
|
|
| Info | Next day | Email |
|
|
|
|
---
|
|
|
|
## Critical Alerts
|
|
|
|
### System Down
|
|
|
|
```yaml
|
|
- alert: SystemDown
|
|
expr: up{job=~"api-gateway|trading-service"} == 0
|
|
for: 1m
|
|
labels:
|
|
severity: critical
|
|
annotations:
|
|
summary: "{{ $labels.job }} is down"
|
|
description: "Service {{ $labels.job }} has been down for 1 minute"
|
|
```
|
|
|
|
### High Error Rate
|
|
|
|
```yaml
|
|
- alert: HighErrorRate
|
|
expr: rate(grpc_requests_total{code!~"2.."}[5m]) / rate(grpc_requests_total[5m]) > 0.05
|
|
for: 5m
|
|
labels:
|
|
severity: critical
|
|
annotations:
|
|
summary: "High error rate: {{ $value | humanizePercentage }}"
|
|
```
|
|
|
|
### Database Connection Pool Exhausted
|
|
|
|
```yaml
|
|
- alert: DatabasePoolExhausted
|
|
expr: pg_stat_database_numbackends / pg_settings_max_connections > 0.9
|
|
for: 5m
|
|
labels:
|
|
severity: critical
|
|
annotations:
|
|
summary: "Database connection pool at {{ $value | humanizePercentage }}"
|
|
```
|
|
|
|
---
|
|
|
|
## Warning Alerts
|
|
|
|
### High Latency
|
|
|
|
```yaml
|
|
- alert: HighLatency
|
|
expr: histogram_quantile(0.99, rate(grpc_request_duration_seconds_bucket[5m])) > 0.5
|
|
for: 10m
|
|
labels:
|
|
severity: warning
|
|
annotations:
|
|
summary: "P99 latency is {{ $value }}s"
|
|
```
|
|
|
|
### High Memory Usage
|
|
|
|
```yaml
|
|
- alert: HighMemoryUsage
|
|
expr: container_memory_usage_bytes / container_spec_memory_limit_bytes > 0.9
|
|
for: 10m
|
|
labels:
|
|
severity: warning
|
|
annotations:
|
|
summary: "Memory usage at {{ $value | humanizePercentage }}"
|
|
```
|
|
|
|
---
|
|
|
|
**End of Alerting Rules Guide**
|