I've been running Banyan in a hybrid environment for about eight months, primarily to secure access to internal AI inference endpoints and our model evaluation tooling. The vendor documentation and several case studies mention seamless scaling to thousands of users, but my team's experience during a scheduled load test last quarter suggests the reality is more nuanced, particularly around the claimed "zero-trust" access proxy and the policy engine.
We designed a test to simulate a ramp from 100 to 500 concurrent, authenticated users performing typical operations: establishing secure tunnels to various backend services, with JIT privilege requests mixed in. The test harness was synthetic but modeled real behavior, with think times and varied request patterns. We monitored the Banyan Command Center (global policy plane) and our deployed Access Tier instances (data plane).
Here are the key observations when we crossed the ~350 concurrent user threshold:
* **Latency Degradation:** Average tunnel establishment time increased from ~120ms to over 900ms. This wasn't linear; it exhibited a sharp knee in the curve around 375 concurrent users.
* **Policy Evaluation Lag:** JIT privilege requests, which normally resolve in under 200ms, began queuing. We observed 2-3 second delays, which in a real scenario, would lead to user abandonment.
* **Access Tier Instability:** Two of our four Access Tier pods began cycling due to memory pressure. The logs pointed to sustained high connection churn and what appeared to be inefficient cleanup of terminated user sessions.
* **Command Center API Throttling:** Our automated monitoring scripts, which poll the Command Center API for health metrics, started receiving 429 (Too Many Requests) responses, indicating the global management plane was under stress.
Our configuration is fairly standard. We deployed Access Tiers on AWS m5.xlarge instances (4 vCPU, 16 GiB RAM) as per their sizing guide for up to 500 users.
```
# Access Tier Deployment Snippet (Helm values)
replicaCount: 4
resources:
limits:
cpu: "2"
memory: "4Gi"
requests:
cpu: "1"
memory: "2Gi"
autoscaling:
enabled: false # We wanted fixed capacity for this test
```
The vendor's response was that our "testing methodology didn't accurately reflect real-world usage patterns" and that we should consider horizontal scaling of Access Tiers sooner, or upgrade to larger instance types. While technically true, this contradicts the implied "scale effortlessly" messaging. The bottleneck appeared less about raw compute on the Access Tiers and more about coordination load on the Command Center and internal state management.
So, my question to the community: has anyone else conducted similar scale testing, particularly under sustained concurrent load? I'm interested in:
* At what user concurrency you observed significant performance degradation.
* Whether the bottleneck was primarily in the data plane (Access Tiers) or the control plane (Command Center).
* Any specific tuning parameters (beyond just throwing larger instances at it) that proved effective, such as adjusting session timeouts, connection pool settings, or database tuning for the self-hosted components.
* If the managed service offering exhibits the same characteristics, or if the scaling limitations are primarily for the self-hosted deployment model.
The marketing claims of "enterprise-scale" need to be backed by reproducible benchmarks. Without concrete data, we're all just operating on vendor promises.
Show me the benchmarks
That sharp knee in the curve around 375 users is really telling, and it mirrors something I've seen in other systems that claim massive horizontal scale. The policy engine often becomes the hidden bottleneck; it might be fine for steady-state traffic but chokes on concurrent evaluation spikes from JIT requests.
Your latency jumping from 120ms to 900ms is a huge signal. It makes me wonder about your Access Tier instance sizing and if the policy decisions are all getting funneled back to a single global controller node. Did you notice if the Command Center metrics showed elevated CPU or queue depth on the policy service at that same threshold? Sometimes the "seamless scaling" only applies to the data plane proxies, while the control plane has a much lower, undocumented ceiling.
We had a similar experience with a different tool where auto-scaling for the workers was aggressive, but the central logging and audit database couldn't keep up, causing a domino effect. The marketing case studies never seem to test the messy, concurrent JIT scenario.
don't spam bro
You're seeing the real bottleneck. It's never the tunnels. That latency spike is the centralized policy engine getting hammered by concurrent JIT requests. Their "thousands of users" marketing assumes a steady, predictable flow, not a stampede of authenticated sessions all needing decisions at once.
Did you check if your Access Tiers were actually scaling or just sitting idle while the global controller melted? I've seen this before where the data plane is fine but every request gets serialized waiting for a policy check. The documentation always omits that part.
Just saying.
Absolutely spot on about the control plane being the hidden ceiling. You're right that the data plane scaling gets all the attention.
In our setup, we saw exactly what you described: the Access Tiers were healthy, but the policy service queue latency graph looked like a cliff. The concurrent JIT requests were all serialized at the global controller, creating that exact domino effect. The marketing materials assume a nicely distributed arrival rate, not a surge where everyone needs a decision at the same moment.
It makes me wonder if the true "scale" number should always come with an asterisk for max concurrent policy evaluations per second, not just user connections. That's the metric we started tracking after our own meltdown.
Stay connected
That asterisk is the whole story. They sell you on user count, but the real constraint is policy evaluations per second, which they never publish.
It's even worse if you use any custom policy hooks or external authorization calls. Then your scaling limit is the slowest third party API your rules depend on, and your whole access platform queues up behind it.
So you're not scaling to thousands of users. You're scaling to whatever the single, non-redundant global controller can handle, which is a much smaller number.
Just my 2 cents