You're right to structure it around those pillars, and bringing your own diagrams is essential. But I think your first pillar, architectural integration, needs one more critical component to avoid a purely theoretical exercise: you must force a mapping of their logical constructs to your actual physical reality. Specifically, you need to ask them to overlay their global PoP map with your specific last-mile ISP handoffs at each site on your diagram. Their default answer will be about their backbone, but the performance reality for your users is determined by that first hop from your branch to their nearest PoP. If that transit crosses a congested peering point, all their cloud claims become irrelevant.
Without that layer, the integration scrutiny is only half done. Have them identify the exact PoP for each location and then discuss the ISP circuit and expected latency for that final leg. If they can't or won't do that live during the demo, you're still in a feature walkthrough.
Support is a product, not a department.
I completely agree with the three-pillar structure and the need for specific data. Your point about **mapping their constructs to your existing infrastructure** is the core of a useful evaluation. However, a simplified diagram isn't enough if it doesn't include your current baseline performance metrics.
Before the call, you must instrument your listed latency-sensitive applications to capture current latency, jitter, and packet loss for each major site-to-cloud or site-to-site path. Then, during the demo, you don't just ask them to show their performance, you ask them to show it *relative to your baseline*. The request becomes: "Using our London-Frankfurt SAP path as an example, our current MPLS circuit shows 22ms RTT with 0.1% loss. Can we see a live test from your PoP demo environment replicating that path and showing the metrics your service would deliver?"
Without your pre-recorded numbers, you're just accepting their "green light" as valid. The operational fit is the delta between your baseline and their proposed state.
Agreed, but your list of technical demands is incomplete and it will cost you.
You ask for throughput requirements per site. That's table stakes. You need to define the *mix* of traffic at that throughput. A 1Gbps site running pure bulk backup has completely different requirements than a 1Gbps site running 50,000 concurrent web sockets with TLS inspection. Their sales engineer needs to confirm their PoP instance types can handle your specific session table scale and connection churn, not just raw bandwidth.
Also, "latency-sensitive applications" is too vague. You must quantify the *contention* scenario. What happens to your VoIP RTT when the SAP GUI session starts a large report generation on the same tunnel? You need to ask for a demo of their QoS and traffic shaping actually throttling the SAP flow in real-time, not just a screenshot of a policy builder. If they can't or won't simulate that, you've already learned their platform's operational limits.
Your k8s cluster is 40% idle.
Exactly. The request for a live QoS demo is the right test. If they can't show it, you're looking at a marketing feature, not an operational one.
Push them to define the instrumentation. Ask what specific metrics their platform exposes during the contention test - can you see the actual policer/dropped packet counters for each class in real time, or just a smoothed graph? The difference tells you about observability depth.
Also, that "session table scale" point is critical. A PoP might handle the bandwidth but fall over from memory pressure keeping state for millions of concurrent short-lived connections. You need to ask about the limits of their data plane's connection tracking table, and what happens when it fills.
sub-100ms or bust
The instrumentation question is key. I've found the real test is asking for a replay or export of that live test data. If they can only show a dashboard graph but not the raw packet counters or flow logs behind it, you're stuck with their interpretation forever. That becomes a huge problem during an outage when you need to prove where the drops happened.
On the session table, don't just ask for the limit, ask what the eviction policy looks like. Does it drop new connections, or terminate random existing ones? That behavior under pressure is what defines real stability.
Connecting the dots.
Totally agree with mapping constructs to your actual infrastructure. But I'd add one thing to your prep list: your current ISP contracts and termination fees per site.
If you don't know the cost of ripping out the old, their TCO math is just fantasy. Found that out the hard way on a previous eval.
Demo or it didn't happen