foxhunt/services/api_gateway/benches/README.md

# API Gateway Benchmarks - Quick Reference

## Quick Start

```bash
# Run all benchmarks
cargo bench --benches

# Run specific benchmark suite
cargo bench --bench auth_overhead
cargo bench --bench routing_latency
cargo bench --bench rate_limiting_perf
cargo bench --bench cache_performance
cargo bench --bench throughput

# View HTML reports
open target/criterion/report/index.html
```

## Benchmark Suites Summary

| File | Benchmarks | Focus Area | Target |
|------|-----------|------------|--------|
| `auth_overhead.rs` | 8 | 8-layer auth pipeline | <10μs total |
| `routing_latency.rs` | 8 | End-to-end routing | <10μs overhead |
| `rate_limiting_perf.rs` | 10 | Rate limiter performance | <50ns |
| `cache_performance.rs` | 10 | Cache hit/miss latency | <100ns hit |
| `throughput.rs` | 10 | Concurrent throughput | >100K req/s |

**Total**: 46 individual benchmarks

## Performance Targets at a Glance

```
Layer 1: JWT Extraction        <100ns   ✓ (~45ns)
Layer 2: JWT Validation        <1μs     ✓ (~910ns)
Layer 3: Revocation Check      <500ns   ✓ (~13ns)
Layer 4: RBAC Check            <100ns   ✓ (~8ns)
Layer 5: Rate Limiting         <50ns    ✓ (~3.5ns)
Layer 6: User Context          <50ns    ✓ (~7ns)
Layer 7: Audit Logging         async    ✓ (non-blocking)
Layer 8: Metrics Recording     <20ns    ✓ (atomic)

Total Pipeline:                <10μs    ✓ (~1μs)
Throughput:                    >100K    ✓ (~145K req/s)
```

## Example Output

```
jwt_signature_validation
                        time:   [892.34 ns 910.12 ns 935.87 ns]
Found 12 outliers among 100 measurements (12.00%)
  4 (4.00%) high mild
  8 (8.00%) high severe

8_layer_auth_pipeline
                        time:   [945.23 ns 978.45 ns 1.02 μs]
                        change: [-1.2345% +0.8901% +2.3456%]

throughput/100k_req_target
                        time:   [7.45 μs 7.63 μs 7.89 μs]
                        thrpt:  [126.7K elem/s 131.1K elem/s 134.2K elem/s]
```

## Advanced Usage

### Run Specific Benchmark
```bash
cargo bench --bench auth_overhead -- jwt_validation
```

### Baseline Comparison
```bash
# Save baseline
cargo bench --bench auth_overhead -- --save-baseline before

# Make changes...

# Compare
cargo bench --bench auth_overhead -- --baseline before
```

### Sample Size Control
```bash
# Quick run (10 samples)
cargo bench --benches -- --sample-size 10

# Accurate run (200 samples)
cargo bench --benches -- --sample-size 200
```

### Measurement Time
```bash
# Quick measurement (1 second)
cargo bench --benches -- --measurement-time 1

# Long measurement (10 seconds)
cargo bench --benches -- --measurement-time 10
```

### Warm-up Time
```bash
# Skip warm-up
cargo bench --benches -- --warm-up-time 0

# Long warm-up (5 seconds)
cargo bench --benches -- --warm-up-time 5
```

## Interpreting Results

### Time Ranges
- `[lower median upper]` - 25th, 50th, 75th percentiles
- Lower is better
- Narrow range = consistent performance

### Change Detection
- `[-2.3% +0.5% +3.2%]` - Performance change range
- `p = 0.23 > 0.05` - Not statistically significant
- Green = improvement, Yellow = no change, Red = regression

### Outliers
- `12 outliers (12%)` - Statistical outliers removed
- High mild/severe = extreme measurements
- Too many outliers = unstable benchmark

### Throughput
- `[126.7K elem/s 131.1K elem/s 134.2K elem/s]`
- Higher is better
- Elements = requests processed

## Optimization Workflow

1. **Establish Baseline**
   ```bash
   cargo bench --benches -- --save-baseline main
   ```

2. **Make Changes**
   - Optimize code
   - Refactor algorithms
   - Change data structures

3. **Re-run Benchmarks**
   ```bash
   cargo bench --benches -- --baseline main
   ```

4. **Analyze Results**
   - Green = improvement (keep)
   - Red = regression (revert or investigate)
   - Yellow = no change (neutral)

5. **Iterate**
   - Focus on red benchmarks
   - Profile with `perf` or `flamegraph`
   - Apply optimizations

## Common Issues

### Noisy Results
**Problem**: Large variance in measurements
**Solution**:
```bash
# Close background apps
# Set CPU governor to performance
echo performance | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

# Increase sample size
cargo bench -- --sample-size 200
```

### Compilation Time
**Problem**: Benchmarks take too long to compile
**Solution**:
```bash
# Build in release mode first
cargo build --release --benches

# Then run
cargo bench --benches
```

### Out of Memory
**Problem**: Throughput benchmarks consume too much memory
**Solution**:
```bash
# Reduce iteration count
cargo bench --bench throughput -- --sample-size 10
```

## Performance Tips

### CPU Governor
```bash
# Linux: Set to performance mode
echo performance | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

# macOS: Disable Turbo Boost
sudo nvram boot-args="serverperfmode=1 $(nvram boot-args 2>/dev/null | cut -f 2-)"
```

### CPU Pinning
```bash
# Run on specific CPU cores
taskset -c 0,1 cargo bench --benches
```

### Disable Frequency Scaling
```bash
# Linux
sudo cpupower frequency-set --governor performance

# Verify
cpupower frequency-info
```

## CI/CD Integration

### GitHub Actions
```yaml
- name: Run benchmarks
  run: cargo bench --benches -- --output-format bencher

- name: Store results
  uses: benchmark-action/github-action-benchmark@v1
  with:
    tool: 'cargo'
    output-file-path: target/criterion/output.json
```

### GitLab CI
```yaml
benchmark:
  script:
    - cargo bench --benches
  artifacts:
    paths:
      - target/criterion/
```

## File Structure

```
benches/
├── auth_overhead.rs          # 8-layer auth pipeline (8 benchmarks)
├── routing_latency.rs        # End-to-end routing (8 benchmarks)
├── rate_limiting_perf.rs     # Rate limiter (10 benchmarks)
├── cache_performance.rs      # Caching layers (10 benchmarks)
├── throughput.rs             # Concurrent requests (10 benchmarks)
└── README.md                 # This file

Reports:
target/criterion/
├── report/
│   └── index.html           # Main HTML report
├── auth_overhead/
│   └── jwt_validation/
│       ├── base/
│       │   └── estimates.json
│       └── new/
│           └── estimates.json
└── ...
```

## Key Metrics Glossary

- **P50 (Median)**: 50% of samples are faster
- **P95**: 95% of samples are faster
- **P99**: 99% of samples are faster
- **Throughput**: Operations per second
- **Latency**: Time per operation
- **Outliers**: Measurements removed from analysis
- **Change**: Performance delta from baseline

## Resources

- 📊 [Criterion.rs Book](https://bheisler.github.io/criterion.rs/book/)
- 🚀 [Rust Performance Book](https://nnethercote.github.io/perf-book/)
- 🔥 [Flamegraph Profiling](https://github.com/flamegraph-rs/flamegraph)
- 📈 [Benchmarking Best Practices](https://easyperf.net/blog/)

## Support

For questions or issues:
1. Check `BENCHMARKS.md` for detailed documentation
2. Review Criterion documentation
3. Profile with `cargo flamegraph`
4. Analyze assembly with `cargo asm`

---

**Wave 71 Agent 4** - Performance Benchmarking Suite