9c3d741a08
Move 17 library crates into crates/, CLI binary into bin/fxt, consolidate 10 test crates into testing/, split config crate from deployment config files. Root directory reduced from 38+ to ~17 directories. All Cargo.toml paths and build.rs proto refs updated. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
3.2 KiB
3.2 KiB
Load Testing Cheat Sheet
Prerequisites Check
# Docker services running?
docker ps | grep -E "(redis|postgres)" | wc -l # Should be 2+
# API Gateway running?
curl -f http://localhost:50050/health && echo "✅ Ready" || echo "❌ Not running"
# Load tests built?
ls -lah ../../target/release/load_test_runner && echo "✅ Built" || echo "❌ Not built"
Quick Commands
Run Tests
# Normal Load (1K clients, 60s)
cargo run -r --bin load_test_runner -- normal
# Spike Load (0→10K clients)
cargo run -r --bin load_test_runner -- spike
# Stress Test (find breaking point)
cargo run -r --bin load_test_runner -- stress
# All tests
cargo run -r --bin load_test_runner -- all
View Results
firefox *.html # Open all reports
Extract Metrics
# P99 Latency
grep -oP 'P99 Latency.*?value">\K[^<]+' normal_load_report.html
# Throughput
grep -oP 'Requests/Second.*?value">\K[^<]+' normal_load_report.html
# Error Rate
grep -oP 'Error Rate.*?value">\K[^<]+' normal_load_report.html
Performance Targets
| Metric | Target | Pass/Fail |
|---|---|---|
| P99 Latency | <10μs | ✅ / ❌ |
| Throughput | >100K req/s | ✅ / ❌ |
| Error Rate | <0.1% | ✅ / ❌ |
Troubleshooting
Connection Refused
# Start API Gateway
cargo run -r -p api_gateway
High Latency
# Check CPU
top -p $(pgrep api_gateway)
# Check DB connections
psql -h localhost -p 5433 -U postgres -c 'SELECT count(*) FROM pg_stat_activity;'
High Errors
# Check logs
docker logs foxhunt-api-gateway --tail 50
# Check backend health
curl http://localhost:50051/health # Trading
curl http://localhost:50052/health # Backtesting
curl http://localhost:50053/health # ML Training
5-Minute Validation
# 1. Start services
docker-compose up -d redis postgres
# 2. Start API Gateway
cargo run -r -p api_gateway &
# 3. Wait for startup
sleep 5
# 4. Run quick test
cd services/api_gateway/load_tests
cargo run -r --bin load_test_runner -- normal --num-clients 100 --duration-secs 30
# 5. Check results
firefox normal_load_report.html
Expected Results
Good Performance ✅
Requests/Second: >100,000
Error Rate: <0.1%
P99 Latency: <10μs
Needs Work ⚠️
Requests/Second: 1,000-10,000
Error Rate: 0.5%-5%
P99 Latency: 10-100ms
Files Generated
normal_load_report.html # Main dashboard
normal_load_report.rps.svg # RPS chart
normal_load_report.latency.svg # Latency chart
normal_load_report.errors.svg # Error chart
Custom Parameters
# Custom normal load
cargo run -r --bin load_test_runner -- normal \
--num-clients 500 \
--duration-secs 120
# Custom spike load
cargo run -r --bin load_test_runner -- spike \
--target-clients 5000 \
--ramp-up-secs 15
# Custom stress test
cargo run -r --bin load_test_runner -- stress \
--initial-clients 200 \
--increment 200 \
--max-p99-latency-ms 100.0
Monitoring Commands
# Real-time metrics
watch -n 1 "curl -s http://localhost:50050/metrics | grep requests_total"
# Docker stats
watch -n 1 "docker stats --no-stream foxhunt-api-gateway"
# System resources
watch -n 1 "ps aux | grep api_gateway | grep -v grep"