Alright, gather 'round the corporate campfire. So the higher-ups have seen the buzzwords and the glossy sales decks and now you're tasked with running a Zscaler PoC. The goal, presumably, is to prove it's the magical cloud security panacea before signing a hefty contract. Let's be real: their onboarding team will give you a metrics wishlist that makes them look good. Your job is to capture data that tells the actual story.
First, ditch the idea that this is just about "seeing if it works." You need to define what "works" even means for your specific chaos. Is it about offloading VPN capacity? "Reducing attack surface"? Their marketing will happily define success for you, but don't let them. Start by instrumenting your existing edge. Get a baseline for at least two weeks *before* you turn on Zscaler. Capture:
* **VPN concentrator load** (peak concurrent users, bandwidth)
* **Direct internet egress traffic** volume and destinations (you might be shocked)
* **Time to resolve** internal help desk tickets for "web filtering" or "site access" issues
* A sample of **latency** to key SaaS apps (Office 365, Salesforce, etc.) from several typical user locations
Then, during the PoC, measure the exact same things. The delta is your actual value, not some pretty dashboard they provide. Pay special attention to the "free alternative" you already have: the internet. Compare Zscaler's latency to a direct connection for those SaaS apps. If their "nearest cloud node" adds 50ms, that's a tangible user experience cost.
And for the love of licenses, **test the exceptions workflow**. Every org has legacy junk that needs to bypass the proxy. How painful is it to create a PAC file bypass or an IP exclusion? How many clicks to whitelist a mistakenly blocked internal app? That's your future administrative overhead right there. Document the time it takes.
Finally, the metric nobody wants to talk about: the **soft cost of lock-in**. Once you route all your traffic through their cloud, what's your exit strategy? The PoC is your one chance to feel that viscosity. Try simulating a partial rollback for a department. Is it a switch flip or a multi-week support ticket?
Their metrics will show green checkmarks. Your job is to find the hidden price tags.
― Finn
FOSS advocate
Couldn't agree more on the baseline instrumentation, but you're still thinking like an engineer who believes the data will be the deciding factor. Let me offer a cynical addition: you also need to baseline the political capital and internal friction.
The metrics you capture will be weaponized in the post-PoC meeting. The VPN load graph will look great, but the latency sample to Salesforce will have one outlier from that guy in Boise on his home DSL. That's the slide the incumbent firewall vendor's account rep will be handed by the network team who hates this cloud thing. Your real PoC isn't just about the tech, it's about gathering ammunition for the internal turf war that's about to start. So, document the hell out of your "before" state, especially the pain points everyone already complains about, because you'll need to remind them why you're doing this when the new, shiny pain points appear.
monoliths are not evil
Oh man, this is so true. You've just described every major tool evaluation I've ever been part of.
That "one outlier from Boise" will absolutely become the poster child for the opposition. Been there. What saved us once was having a pre-agreed "success criteria" doc, signed off by stakeholders *before* the PoC started. It included what metrics mattered, acceptable thresholds, and crucially, how we'd handle anomalies (e.g., "We'll discard the top/bottom 5% of latency samples").
Even with that, you still need the political ammo. I'd add: take screenshots of your current ticket system for VPN complaints or app performance issues *now*. When someone starts complaining about a slight hiccup during the PoC, you can show them the weekly fire drill we're trying to eliminate.
Pipeline Pilot
You're right about the outlier becoming the weapon, but I think the pre-agreed success doc is a trap. It gives the illusion of objectivity, but the goalposts will move as soon as the data comes in. The team that hates it will suddenly discover that "user experience" or "operational nuance" wasn't captured by your pretty thresholds.
The real tactic isn't to document the "before" state. It's to get the loudest complainer from the network team on the PoC testing call when that Boise latency spike happens. Let them hear the user on their DSL complaining about the current VPN dropout, not the Zscaler hop. It reframes the outlier from "proof it's broken" to "proof the old problem still exists, and now we can actually see it."
Otherwise, you're just building a better dataset for your opponents to cherry pick from.
Brilliant tactical point. I'm stealing that "get the complainer on the call" move for the next vendor bake-off. It reframes the conversation instantly.
But you still need the data, because that same person will demand "hard numbers" later. The trick is to make the metrics inherently comparative. Don't just log latency to Salesforce during the PoC, graph it against the baseline period on the same chart. That outlier spike? Plot it right next to the 15 bigger spikes from last month on the old VPN. The story becomes "look, we still have this problem, but the average is way better and the *frequency* is down."
Otherwise, you're stuck in a debate about absolute perfection versus their idealized version of the current setup. The data's job isn't to be objective, it's to tell the comparative story visually.
pipeline all the things