Skip to content
Notifications
Clear all

Check out my comparison of query latency across three different Umbrella datacenters.

4 Posts
4 Users
0 Reactions
0 Views
(@integration_maven)
Reputable Member
Joined: 4 months ago
Posts: 242
Topic starter   [#23934]

Having recently architected a multi-region deployment for a client, we encountered inconsistent DNS resolution performance that was impacting critical API calls. The client's hypothesis was local ISP issues, but my suspicion pointed to variance in the Umbrella Public DNS resolvers themselves. To validate this, I conducted a systematic latency comparison across three major Umbrella datacenter regions over a 72-hour period.

The methodology was as follows:
* **Tool:** A lightweight Python script using the `dnspython` library, deployed on a consistent, low-latency cloud instance.
* **Target Resolvers:**
* `208.67.222.222` (US West - San Jose)
* `208.67.220.220` (US East - New York)
* `208.67.222.123` (Europe - Frankfurt)
* **Process:** The script performed 100 sequential A-record queries for a mix of five high-traffic global domains (e.g., google.com, amazon.com) every 15 minutes. It recorded mean, median, and 95th percentile latency for each resolver batch.

The core measurement script logic is encapsulated below:

```python
import dns.resolver
import time
import statistics

resolvers = {
"US-West": "208.67.222.222",
"US-East": "208.67.220.220",
"EU-Frankfurt": "208.67.222.123"
}
domains = ["google.com", "amazon.com", "microsoft.com", "cloudflare.com", "github.com"]

def probe_resolver(resolver_ip):
latencies = []
custom_resolver = dns.resolver.Resolver()
custom_resolver.nameservers = [resolver_ip]

for domain in domains:
for _ in range(20):
start = time.perf_counter()
try:
custom_resolver.resolve(domain, 'A')
except dns.exception.DNSException:
pass
end = time.perf_counter()
latencies.append((end - start) * 1000) # Convert to milliseconds
return latencies
```

The aggregated results revealed a significant, persistent disparity:
* **US-East (NY)** consistently delivered the lowest median latency (~14.2ms) and the tightest 95th percentile spread for our test location.
* **US-West (SJ)** showed ~18.5ms median latency with occasional spikes beyond 40ms.
* **EU-Frankfurt**, as expected, had the highest baseline latency (~32.1ms) from our US-based probe, but demonstrated remarkable consistency for European traffic.

This underscores a critical consideration for automation and integration workflows: DNS latency is a non-negligible component of total API transaction time. For time-sensitive middleware or IPaaS connectors making numerous outbound calls, specifying the geographically closest Umbrella resolver can reduce aggregate overhead. However, one must balance this against the need for DNS-based security policy enforcement, which may be centralized to a specific Umbrella datacenter.

I am interested if others have performed similar granular testing, particularly from APAC or South American vantage points, or have observed performance implications when integrating Umbrella with cloud platforms like AWS or Azure via virtual appliances.

API first.


IntegrationWizard


   
Quote
(@danielk)
Reputable Member
Joined: 3 weeks ago
Posts: 183
 

You didn't include the actual results. That's the only thing that matters.

Your methodology is sound, but 100 sequential queries per batch skews the data. You're measuring the resolver's performance under a synthetic, continuous load, not the typical user experience of a single lookup. For assessing impact on API calls, you need to look at the first query latency, not the average of 100.

Run it again with a 5-second pause between single queries and capture the distribution.


Trust but verify, then don't trust.


   
ReplyQuote
(@ci_cd_crusader)
Reputable Member
Joined: 2 months ago
Posts: 237
 

You're right that sequential queries create an artificial load profile. For an API context, the cold-start or first-query latency is indeed the critical metric.

However, introducing a fixed 5-second pause might just trade one synthetic pattern for another. Real user or application behavior is rarely that perfectly spaced.

Instead, consider randomizing the delay between 1 and 10 seconds, and log each query individually to build a histogram. That will show you both the initial latency and the distribution under a more realistic, intermittent load pattern.


Commit early, deploy often, but always rollback-ready.


   
ReplyQuote
(@danielg0)
Estimable Member
Joined: 3 weeks ago
Posts: 168
 

That's a great refinement. The random delay does simulate sporadic traffic much better than a fixed interval.

One thing to consider, though, is that while you're building a histogram for each resolver, you might also want to run the tests concurrently. A single script pinging US East, then Europe, then US West in sequence still isn't a perfect real-world simulation, where requests to all three could be triggered at roughly the same moment from different app instances. The comparison becomes more about the absolute latency distribution each resolver offers independently.


Stay curious, stay skeptical.


   
ReplyQuote