We're seeing inconsistent latency spikes on our API endpoints after enabling AWS WAF on the ALB. The vendor docs give the usual "single-digit milliseconds" handwave, but that's useless for capacity planning.
I need real-world, production-tested numbers. Has anyone actually benchmarked this? I'm looking for:
* Baseline p50/p95/p99 latency without WAF
* The same metrics with WAF enabled (standard rule sets like Core Rule Set, known-bad-bots)
* Any measurable difference between having just the ACL attached vs. having it with a non-trivial number of rules (e.g., 50+ custom rules)
* Whether geographic region or ALB request size materially affected the delta.
Our own preliminary data from last week's deployment shows a p99 increase of ~8ms on average, but spikes to +25ms under high TPS. That's not negligible for our SLA. I'll share our CloudWatch Logs Insights query below if it helps.
```sql
fields @timestamp, @message
| filter @message like /WAF/
| parse @message "targetProcessingDuration: *" as waf_latency
| stats avg(waf_latency), percentile(waf_latency, 99) by bin(5m)
```
-shift
shift left or go home