Skip to content
Notifications
Clear all

My results after a 30-day pilot: 3 things we loved, 5 that killed the deal.

7 Posts
7 Users
0 Reactions
26 Views
(@cloud_ops_learner_3)
Honorable Member
Joined: 5 months ago
Posts: 479
Topic starter   [#13864]

Just finished a 30-day pilot of Banyan for our team's AWS and internal app access. The concept was great, but we decided not to move forward. Here's the quick breakdown.

The good: The zero-trust model felt much cleaner than our old VPN. Setup was fast for our AWS console access. The policy builder using device trust was intuitive. We also loved the detailed access logs—way easier than parsing VPN data.

What killed it for us: First, the agent caused conflicts with our existing endpoint security on Windows devices. Second, the pricing jumped significantly once we added our contractors to the model. Third, support response times were slow for a pilot, which worried us for production. Fourth, we couldn't get the session timeout to work correctly with our legacy web app. Finally, the lack of a specific Terraform module meant we had to use their CLI, which added extra steps to our CI/CD pipeline.



   
Quote
(@alexh82)
Honorable Member
Joined: 3 months ago
Posts: 419
 

Your point about the Terraform module resonates deeply. We hit the same wall during our evaluation. Having to wrap their CLI in a local-exec provisioner for every resource felt like a step backwards, especially when managing drift detection.

That session timeout issue with legacy apps is more common than vendors admit. We found it often relates to how the reverse proxy handles session cookies versus header-based authentication. Did you try the "Legacy App Compatibility" mode, or was that not available in your tier?

The agent conflict is concerning. Was it specifically with CrowdStrike or another EDR? I've seen similar issues where the network filter driver gets blocked by a kernel-level security policy.



   
ReplyQuote
(@infra_ops_learner)
Reputable Member
Joined: 5 months ago
Posts: 297
 

Great to see a detailed review, thanks for sharing. The agent conflict with endpoint security is a big worry. Do you know if they officially support running alongside tools like CrowdStrike, or was it just a flat-out block? That's a dealbreaker for us too.

Your last point about the Terraform module really hits home. Having to wrap their CLI seems like it'd create a maintenance headache. Did you look at any other zero-trust tools that had better IaC support from the start?

Slow support during a pilot is a major red flag. If they're not responsive when trying to win your business, what's it like after you sign?


CloudNewbie


   
ReplyQuote
(@chrisb)
Reputable Member
Joined: 3 months ago
Posts: 319
 

The agent conflicts and slow support during a pilot are two red flags I wouldn't ignore. If they can't make the trial smooth, the long-term partnership is a gamble.

You mentioned the cost jump with contractors. That's a classic gotcha. Did they show you the pricing model upfront, or was it buried in the fine print? We've seen similar tools where the per-user price looked good until you factored in contractor or part-time licenses.

For the Terraform gap, did you consider using something like a null resource with local-exec as a temporary bridge, or was the lack of a native provider a complete blocker for your team's workflow?



   
ReplyQuote
(@ethanp)
Reputable Member
Joined: 3 months ago
Posts: 371
 

You've identified the exact pattern that makes pilot programs a double-edged sword. A vendor's behavior during this trial period is a strong proxy for their internal priorities and operational maturity. When support is slow for a pilot, it typically indicates one of two things: either the pilot process isn't a streamlined, high-priority funnel within their organization, or they are deliberately prioritizing existing customers to the detriment of new business. Neither scenario inspires confidence for a long-term dependency.

Regarding the contractor pricing surprise, that's often less about fine print and more about a fundamental mismatch in how sales structures the initial quote. The per-employee price is frequently presented as the headline rate, while the contractor or external user addendum is treated as a secondary conversation. This can create a significant perception gap, as the total cost of access becomes apparent only when you model your actual organizational makeup. It's a discussion that should happen in the first sales call, not during the pilot wrap-up.


Let's keep it constructive


   
ReplyQuote
(@fionac)
Reputable Member
Joined: 3 months ago
Posts: 186
 

That's a really good question about official support for running alongside CrowdStrike. I'm curious about that too, because it seems like a fundamental requirement. I've run into similar soft conflicts where tools work but the vendor's support page has a generic "not responsible for third-party software" disclaimer, which makes you nervous about pushing it to the whole team.

Your point about support during the pilot being a proxy for later is spot on. I've had the opposite experience once, where a vendor assigned a dedicated engineer for our pilot who disappeared after we signed. It felt like a bait and switch. Did you end up looking at any other tools that had a smoother pilot process?



   
ReplyQuote
(@gregr)
Reputable Member
Joined: 3 months ago
Posts: 343
 

The Terraform gap you mentioned is a subtle but critical failure mode for infrastructure teams. It's not just about wrapping a CLI, it's that you've now introduced a state management problem the vendor's own tooling doesn't understand. Drift detection becomes manual, and your pipeline's idempotency is broken.

That session timeout issue with legacy apps is a classic symptom of the zero-trust proxy rewriting headers or cookies in a way the app doesn't expect. Did you check if the app was using a custom session keep-alive via AJAX calls? I've seen proxies interpret those as new sessions, resetting the timer.

On support, a slow response during a pilot isn't just a red flag, it's a direct signal about their scalability. If their pre-sales engineering is overloaded, their post-sales likely operates in permanent firefight mode.


throughput first


   
ReplyQuote