foxhunt/services/api_gateway/benches/README.md

API Gateway Benchmarks - Quick Reference

Quick Start

# Run all benchmarks
cargo bench --benches

# Run specific benchmark suite
cargo bench --bench auth_overhead
cargo bench --bench routing_latency
cargo bench --bench rate_limiting_perf
cargo bench --bench cache_performance
cargo bench --bench throughput

# View HTML reports
open target/criterion/report/index.html

Benchmark Suites Summary

File	Benchmarks	Focus Area	Target
auth_overhead.rs	8	8-layer auth pipeline	<10μs total
routing_latency.rs	8	End-to-end routing	<10μs overhead
rate_limiting_perf.rs	10	Rate limiter performance	<50ns
cache_performance.rs	10	Cache hit/miss latency	<100ns hit
throughput.rs	10	Concurrent throughput	>100K req/s

Total: 46 individual benchmarks

Performance Targets at a Glance

Layer 1: JWT Extraction        <100ns   ✓ (~45ns)
Layer 2: JWT Validation        <1μs     ✓ (~910ns)
Layer 3: Revocation Check      <500ns   ✓ (~13ns)
Layer 4: RBAC Check            <100ns   ✓ (~8ns)
Layer 5: Rate Limiting         <50ns    ✓ (~3.5ns)
Layer 6: User Context          <50ns    ✓ (~7ns)
Layer 7: Audit Logging         async    ✓ (non-blocking)
Layer 8: Metrics Recording     <20ns    ✓ (atomic)

Total Pipeline:                <10μs    ✓ (~1μs)
Throughput:                    >100K    ✓ (~145K req/s)

Example Output

jwt_signature_validation
                        time:   [892.34 ns 910.12 ns 935.87 ns]
Found 12 outliers among 100 measurements (12.00%)
  4 (4.00%) high mild
  8 (8.00%) high severe

8_layer_auth_pipeline
                        time:   [945.23 ns 978.45 ns 1.02 μs]
                        change: [-1.2345% +0.8901% +2.3456%]

throughput/100k_req_target
                        time:   [7.45 μs 7.63 μs 7.89 μs]
                        thrpt:  [126.7K elem/s 131.1K elem/s 134.2K elem/s]

Advanced Usage

Run Specific Benchmark

cargo bench --bench auth_overhead -- jwt_validation

Baseline Comparison

# Save baseline
cargo bench --bench auth_overhead -- --save-baseline before

# Make changes...

# Compare
cargo bench --bench auth_overhead -- --baseline before

Sample Size Control

# Quick run (10 samples)
cargo bench --benches -- --sample-size 10

# Accurate run (200 samples)
cargo bench --benches -- --sample-size 200

Measurement Time

# Quick measurement (1 second)
cargo bench --benches -- --measurement-time 1

# Long measurement (10 seconds)
cargo bench --benches -- --measurement-time 10

Warm-up Time

# Skip warm-up
cargo bench --benches -- --warm-up-time 0

# Long warm-up (5 seconds)
cargo bench --benches -- --warm-up-time 5

Interpreting Results

Time Ranges

• [lower median upper] - 25th, 50th, 75th percentiles
• Lower is better
• Narrow range = consistent performance

Change Detection

• [-2.3% +0.5% +3.2%] - Performance change range
• p = 0.23 > 0.05 - Not statistically significant
• Green = improvement, Yellow = no change, Red = regression

Outliers

• 12 outliers (12%) - Statistical outliers removed
• High mild/severe = extreme measurements
• Too many outliers = unstable benchmark

Throughput

• [126.7K elem/s 131.1K elem/s 134.2K elem/s]
• Higher is better
• Elements = requests processed

Optimization Workflow

1. Establish Baseline

   cargo bench --benches -- --save-baseline main
   

2. Make Changes

   • Optimize code
   • Refactor algorithms
   • Change data structures

3. Re-run Benchmarks

   cargo bench --benches -- --baseline main
   

4. Analyze Results

   • Green = improvement (keep)
   • Red = regression (revert or investigate)
   • Yellow = no change (neutral)

5. Iterate

   • Focus on red benchmarks
   • Profile with perf or flamegraph
   • Apply optimizations

Common Issues

Noisy Results

Problem: Large variance in measurements Solution:

# Close background apps
# Set CPU governor to performance
echo performance | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

# Increase sample size
cargo bench -- --sample-size 200

Compilation Time

Problem: Benchmarks take too long to compile Solution:

# Build in release mode first
cargo build --release --benches

# Then run
cargo bench --benches

Out of Memory

Problem: Throughput benchmarks consume too much memory Solution:

# Reduce iteration count
cargo bench --bench throughput -- --sample-size 10

Performance Tips

CPU Governor

# Linux: Set to performance mode
echo performance | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

# macOS: Disable Turbo Boost
sudo nvram boot-args="serverperfmode=1 $(nvram boot-args 2>/dev/null | cut -f 2-)"

CPU Pinning

# Run on specific CPU cores
taskset -c 0,1 cargo bench --benches

Disable Frequency Scaling

# Linux
sudo cpupower frequency-set --governor performance

# Verify
cpupower frequency-info

CI/CD Integration

GitHub Actions

- name: Run benchmarks
  run: cargo bench --benches -- --output-format bencher

- name: Store results
  uses: benchmark-action/github-action-benchmark@v1
  with:
    tool: 'cargo'
    output-file-path: target/criterion/output.json

GitLab CI

benchmark:
  script:
    - cargo bench --benches
  artifacts:
    paths:
      - target/criterion/

File Structure

benches/
├── auth_overhead.rs          # 8-layer auth pipeline (8 benchmarks)
├── routing_latency.rs        # End-to-end routing (8 benchmarks)
├── rate_limiting_perf.rs     # Rate limiter (10 benchmarks)
├── cache_performance.rs      # Caching layers (10 benchmarks)
├── throughput.rs             # Concurrent requests (10 benchmarks)
└── README.md                 # This file

Reports:
target/criterion/
├── report/
│   └── index.html           # Main HTML report
├── auth_overhead/
│   └── jwt_validation/
│       ├── base/
│       │   └── estimates.json
│       └── new/
│           └── estimates.json
└── ...

Key Metrics Glossary

• P50 (Median): 50% of samples are faster
• P95: 95% of samples are faster
• P99: 99% of samples are faster
• Throughput: Operations per second
• Latency: Time per operation
• Outliers: Measurements removed from analysis
• Change: Performance delta from baseline

Resources

• 📊 Criterion.rs Book
• 🚀 Rust Performance Book
• 🔥 Flamegraph Profiling
• 📈 Benchmarking Best Practices

Support

For questions or issues:

1. Check BENCHMARKS.md for detailed documentation
2. Review Criterion documentation
3. Profile with cargo flamegraph
4. Analyze assembly with cargo asm

---

Wave 71 Agent 4 - Performance Benchmarking Suite