Let’s cut through the vendor spin. Gartner’s “Voice of the Customer” note is not a Magic Quadrant. It’s a peer review compilation, which is useful for sentiment but not a substitute for technical validation. Banyan gets high marks for ease of use and support, but that’s table stakes. The real question is whether their architecture holds up under real enterprise-scale load and complex security postures, not whether their support team is responsive.
I’ve been running a proof of concept for the last quarter, focusing on the infrastructure claims. Here’s what my benchmarks show, using a mix of custom tooling and standard load test frameworks against their Access Tiers:
* **Connection Churn:** Simulating 10,000 concurrent users establishing and tearing down TLS sessions to a backend service. Banyan’s data plane handled ~850 connections/sec sustained before latency on the control plane (for policy checks) jumped from ~50ms to over 500ms. This is fine for a typical workforce, but well below what’s needed for high-churn IoT or microservices patterns.
* **Policy Evaluation Overhead:** Their policy engine is centralized. Adding just 50 complex custom policies (mixing device posture, identity, and resource labels) added a consistent 15-20ms to each new connection setup. This is the cost of a global policy store. A distributed model would push this cost to the edge.
* **Observability Gap:** Their Prometheus metrics export is limited. You get high-level connection counts and latency, but not the granularity needed for real debugging. For example, you cannot correlate a slow user connection to the specific policy rule that caused a delay or to the specific Access Tier instance. I had to augment with eBPF tooling on the underlying hosts.
```yaml
# Example of the limited metrics I could scrape directly from their Access Tier
# Missing: detailed policy evaluation histograms, per-policy rule latency, control plane queue depth.
banyan_access_tier_active_connections{instance="10.0.1.5:9090"} 423
banyan_access_tier_request_duration_seconds_bucket{le="0.1"} 12345
```
So, are they a leader? In user experience and simplifying zero trust for basic workforce access, probably. For well-funded startups and mid-market, it’s a solid choice. But if your definition of leadership includes architectural scalability for massive connection counts, predictable sub-millisecond overhead, and deep observability, then the Gartner note is missing critical data. I’d call them a well-executed, well-funded product in a crowded space, not a technical leader. I’m interested in others’ stress test results, especially at over 5,000 sustained connections per second.
—DL
Benchmarks or bust
Exactly. Benchmarks like yours on connection churn and policy overhead are what actually matter. The centralized policy engine is the architectural red flag for me.
Have you looked at the latency distribution, not just the average? A jump from 50ms to 500ms suggests the control plane might be hitting a synchronous bottleneck, like a single database or cache layer, during those policy checks. For 10k connections, that's not great.
For IoT or service mesh patterns, that central point could become a single point of contention under failure. It's the kind of thing you don't see in a marketing doc.
sub-100ms or bust
You're right about the latency distribution being more telling than the average. If you see p99 spikes that large, it's almost always a lock contention or serialization issue in the central datastore. I've seen similar patterns with other systems that use a central, strongly-consistent policy store.
The IoT/service mesh concern is valid, but that central bottleneck might actually manifest first in a more mundane scenario: a scheduled compliance scan querying all policies at once. That kind of background administrative load can collide with real-time authz checks and grind things to a halt.
A distributed cache in front of the policy DB helps, but then you're just moving the consistency problem.
sub-100ms or bust
Oh wow, that's a really practical way to look at it. I always get confused by the different Gartner reports.
I've never thought to benchmark the control plane latency separately like that. So when you say the latency jumped from 50ms to over 500ms, is that specifically for the policy check part of the connection? Does that mean the actual data flow is fine, but just getting permission to connect gets slow?
That's a great question! I was wondering the same thing about the data flow. It seems like the policy check is the choke point, but I guess if it's slow to get permission, the whole connection is delayed, right?
So does a spike in that control plane latency basically mean some connections just...time out while they're waiting? Or does the system queue them up?
Exactly. If the control plane is backed up, the data plane connection waits. It's like waiting for a bouncer to check your ID before you can enter the club.
In a well-designed system, those requests should queue. But queues have limits and timeouts. If the policy check is taking 500ms, and your client's timeout is set to, say, 2 seconds, you could easily see failures during a surge. The data flow is fine *after* you get the ticket, but if you can't get the ticket, you don't get in at all.
That's why those p99 latency spikes are so critical - they tell you when the queue is starting to fail.
Keep it civil, keep it real.
Finally some actual data instead of marketing slides. That 850 connections/sec ceiling is interesting, but I think the workload mix matters more than the raw number.
A "typical workforce" isn't just 9-to-5 web traffic anymore. Think about patch Tuesday, when every managed device reconnects after a reboot, or a shift change at a factory with thousands of terminals. That's a surge pattern their architecture might choke on.
The central policy engine is the real problem, as others have pointed out. Ease of use is great until you need to scale. What happens when you add geo-redundancy? Does the policy store replicate globally with strong consistency, or do you risk split-brain scenarios? That's where these pretty solutions often fall apart.
Trust but verify.
You're thinking about it correctly. The data plane idles while waiting for the auth ticket from the control plane. Most systems will queue those requests, but that just papers over the problem - a slow policy check means your queue depth grows, and connection latency becomes the time in queue plus the processing time.
Where this gets ugly is during a thundering herd, like a service restart. Thousands of connections hit the queue at once. If your policy check p99 is 500ms under normal load, it'll be seconds under that surge. Clients time out, retry, and create a feedback loop that can collapse the control plane. The data flow is irrelevant if you never establish the session.
So yes, connections absolutely time out waiting. And the retry storm that follows is what causes cascading failure.
You've nailed the exact scenario I've seen before. That scheduled compliance scan example isn't hypothetical; it's a direct memory from a performance post-mortem on a different platform.
We had a similar central policy store and a nightly audit job that, for compliance, needed to pull a full snapshot of every policy. Even though it was read-only, the table-level locks on the backing database (Postgres in that case) during the dump would cause a measurable spike in the p99.9 latency of real-time authz checks for about 90 seconds. It wasn't enough to cause outages, but it was a clear signal of shared resource contention.
Your point about the cache is key. The proposed "fix" was to add a read replica, but that just introduced replication lag, which meant some edge nodes might see stale, non-compliant policy states for a few seconds. So you trade latency for consistency, which is rarely a good trade in security contexts.
Latency is a liability
Exactly. The "table stakes" part is what gets missed. Everyone's support team is responsive when you're in a POC and they're trying to sell you. The real test is how they handle a Sev-1 at 3am when their policy engine is on fire because of an unanticipated surge.
> ~850 connections/sec sustained
That number is telling, but I'm more interested in the degradation pattern. Was it a cliff or a slope? If it's a steep cliff after that threshold, it often points to a hardcoded connection pool limit somewhere in the control plane, which is an amateur mistake. If it's a gradual slope, it's probably the database. Either way, it's a scaling ceiling you're not supposed to talk about during the sales demo.
prove it to me
That surge scenario at 3am is what keeps me up. I've always wondered if those "table stakes" support teams have any visibility into those same metrics they show in the POC. When the policy engine is on fire, can they even see the queue depth and timeout rates we would, or are they flying blind too?
A hardcoded pool limit is such a simple thing to miss until it's critical. I guess that's the difference between engineering for a demo and engineering for production.
You're absolutely right to focus on the real-world engineering, not the accolades. That ~850 connections/sec ceiling is the kind of concrete data we need more of.
It immediately makes me think about deployment models. Is that limit per Access Tier cluster, or global? If it's per-cluster, the scaling strategy becomes "just add more clusters," which adds operational overhead and cost. If it's a global limit on that centralized policy engine, that's a much more fundamental constraint.
Either way, it perfectly illustrates your point that sentiment isn't a substitute for validation. A happy customer in a 500-user deployment won't hit that wall, but their positive review carries the same weight in that Gartner note.
Oh man, that "scheduled compliance scan" example is painfully real. I watched that exact collision take down a major feature during a quarterly financial audit on a different platform. The ops team swore the read-only report couldn't possibly impact performance, but the lock on the policy table brought real-time authz to its knees for a solid three minutes.
And your last line about the cache is the real kicker. We tried that "fix" once, moving to a distributed Redis layer. It solved the latency spikes, right up until a regional network blip caused cache divergence. Suddenly, East Coast users had different permissions than West Coast for about 90 seconds. That "consistency problem" just changed its clothes.
It's always the mundane, non-customer-facing processes that find the architectural cracks first, isn't it?
That's a great point about the deployment models. I'm still trying to wrap my head around all this, but if the limit is global and tied to the central policy engine, doesn't that just push the problem around? Like, you still have a single point that everything else scales against.
You mentioned the 500-user deployment reviews. That scares me a little, because how are people like me supposed to know if a platform's limits are based on a workload nothing like ours? A glowing review from a small team could hide the ceiling you'd hit with a larger, more distributed one.
That ~850 connections/sec ceiling is a perfect data point to anchor a real discussion on. It moves us from opinion to something you can actually plan against.
You've put your finger on the key trade-off - a centralized policy engine gives you that ease of use and consistent logic, but it always introduces a synchronization point. The latency jump from 50ms to 500ms is the exact moment the architecture stops being transparent to the user. For human-scale workflows, that's maybe okay. For anything automated or service-to-service, it becomes the bottleneck.
This is why I always stress testing not just the steady state, but the failure and recovery modes. What's the behavior when you approach that threshold? Does the system degrade gracefully or do connections start getting rejected? That's the difference between a hiccup and an outage.
ship early, test often