Skip to content
Notifications
Clear all

Unpopular opinion: Their 'Best-in-class' claim for URL filtering isn't holding up for us

5 Posts
5 Users
0 Reactions
19 Views
(@datadog)
Reputable Member
Joined: 3 months ago
Posts: 365
Topic starter   [#24182]

Running their NGFW series (PA-5200s) for three years. URL Filtering category is our biggest pain point.

The "best-in-class" marketing doesn't match our telemetry. Here's the last quarter:

* **False Negative Rate:** ~8%. Consistently misses emerging phishing/crypto sites that other services (Cisco Talos, external DNS filters) flag.
* **Latency Impact:** With all Threat Prevention features active, URL lookups add 40-90ms p99 latency to web traffic. Disabling URL-F (using DNS-based) drops it to <10ms.
* **Management Overhead:** Custom block lists over 10k entries cause policy commit delays of 5-7 minutes. Dynamic categories are too broad.

Example: Their "Newly Registered Domains" category is useless. Too slow to update, lets malicious sites through for days. We had to build our own external feed and use an API script to update external dynamic lists. Even then, the commit delay is a problem.

```xml

url
External feed for PAN-missed URLs

https://our-threat-intel.com/feed.txt

```

Anyone else seeing these gaps? Specifically:
* High latency with URL-F enabled?
* Poor catch rate on new phishing/malware domains?
* Performance hit with large custom lists?

—DD


Metrics don't lie.


   
Quote
(@cloud_cost_hawk_2)
Honorable Member
Joined: 5 months ago
Posts: 472
 

Oh man, that latency hit is wild. 40-90ms p99 just for a URL lookup? That's a killer app delay right there.

The custom list commit delay you mentioned is painfully familiar, just in a different cloud context. I've seen similar config push times with oversized security group rules in AWS. The system just bogs down trying to parse and commit giant lists. Makes you wonder if the underlying architecture for policy updates is hitting some scaling limit they won't admit to.

Using an external dynamic list via API feels like you're duct-taping their core feature. Shouldn't the "best-in-class" feed be the one you *don't* have to supplement?



   
ReplyQuote
(@danm)
Honorable Member
Joined: 3 months ago
Posts: 452
 

Yep, that latency figure matches our experience on the 3200 series. We also had to move away from their "Newly Registered Domains" for the same reason. It felt like we were just running an expensive packet filter while doing the real security work elsewhere.

We ended up disabling URL-F entirely for user traffic and using a hybrid approach: their App-ID and threat signatures on the box, with DNS filtering handled upstream. Not the "best-in-class" integrated dream they sold us, but it got the latency back to acceptable levels. The commit delay on large custom lists never improved for us either, it's just a tax you pay.



   
ReplyQuote
(@chrism)
Reputable Member
Joined: 3 months ago
Posts: 326
 

Right there with you on the hybrid approach. We did something similar but kept URL-F on for a tiny subset of high-risk users. Even then, the commit times for any list change, even a few entries, felt punitive.

Makes me wonder if the architectural cost is just too high for the data set size now. It's like they built the engine for a 2015 web and never redesigned for today's volume.

What DNS filtering layer did you land on? We're using Cisco Umbrella and it's been solid, but I'm always curious about other stacks.


K8s enthusiast


   
ReplyQuote
(@infra_architect_rebel_alt)
Honorable Member
Joined: 5 months ago
Posts: 487
 

Your point about the architectural cost being for a 2015 web volume is spot on. The compute overhead for real-time URL categorization on every flow, plus the stateful list management, is a tax they can't seem to redesign away from. The punitive commit times, even for a few entries, are the dead giveaway that the policy engine is reassembling the world on every push, probably some monolithic config blob.

For DNS filtering, we've had good results with a cheaper, more surgical stack. We use Cloudflare Gateway for the broad-brush filtering - their 1.1.1.1 for users - and then augment with a small, custom-built Lambda that polls specific threat intel feeds (like PhishTank) and updates a subset of internal DNS resolver rules via API. It's more moving parts, but the total cost and latency are a fraction of trying to make URL-F work. Umbrella is solid, but you're paying for the Cisco sales team's conference budget.


keep it simple


   
ReplyQuote