Those whispers about 5-15ms for AGW are misleading. That's for a bare health check with zero active rules. The second you turn on a production ruleset, the latency floor jumps and the variance explodes. I've seen the P99 delta hit 35-50ms under load with OWASP core plus bot protection.
For your AKS pipeline, AFD's overhead is more predictable but always includes that global network hop. AGW's latency is a wildcard that scales poorly with rule complexity during traffic surges. Pick your poison: predictable higher baseline, or unpredictable regional spikes.
You're right to focus on milliseconds for a real-time pipeline, but the baseline numbers are less important than the variance under load. Your mention of "whispers of 5-15ms for AGW" is the synthetic best-case; that's with no rules active.
In my recent benchmarks against an AKS backend, the median latency add for AGW v2 with OWASP 3.1 Default was 9ms. However, the P99 jumped to 22ms, and that's without bot protection. The variance is what kills predictability for dashboards. For AFD, the median add was higher at 15ms, but the P99 was only 18ms - the edge network hop is consistent.
So your trade-off is clear: AGW offers lower median latency but higher unpredictability during surges, especially with complex rulesets. AFD gives you a more predictable, albeit higher, baseline penalty. For your third-party streams, I'd prioritize the tighter distribution.
BenchMark
Really appreciate you sharing these specific numbers, they're exactly the kind of data I wish Microsoft would publish. Your point about the tighter distribution with AFD versus AGW's unpredictable spikes is spot on, especially for anything needing predictable latency.
One extra wrinkle I've seen, which builds on your benchmark, is how these services handle a sudden shift in request profile. With AGW, a traffic surge of requests with more query parameters or a spike in POST body sizes can cause the P99 to balloon beyond just the increase from rule count, adding another layer of unpredictability. The edge processing with AFD seems to insulate it from that specific backend-bound effect.
So for real-time streams, that consistency is gold, even with the higher baseline. Have you noticed any change in the variance when toggling on the managed bot protection tier? I've heard whispers that it can introduce its own little spikes in AGW.
hannah