Great questions to kick things off. Since you're new to SASE, I'd watch out for them assuming the "single pane" concept is obvious. Ask them to *show* the exact same policy rule, but then display the backend configuration it generates for an Azure Security Group versus an AWS Security Group side by side. If they're just a fancy front-end for two separate rule sets, you'll feel the pain later.
For routing, push beyond the happy path. Ask them to simulate a cloud region failure live in the demo, and watch how the path convergence works. If the logic or timing is different between AWS and Azure, you've got a hidden operational gap from day one.
On cost visibility, ask to filter for one business application that uses resources in both clouds. Can they show you a unified cost for that logical service? If it's just repackaged separate cloud bills, it's not giving you the insight you need for true multi-cloud management. Good luck with the demo
Stay curious, stay skeptical.
That's a great practical test. Pushing them to show the latency and cost impact of a new path immediately shows if their control plane is just reacting or actually making intelligent routing decisions. A world tour for your traffic isn't just a performance hit, it's a budget surprise.
I'd add one thing to the policy object check. When they build that rule, watch how they select the source and destination. If they have to click through a nested folder structure of accounts and VPCs to find the specific "development" tag, that's a red flag. The real efficiency gain is being able to type "dev-AWS" into a unified search bar and having the system understand it across clouds. If they can't do that, you're looking at a lot of manual upkeep.
Keep it civil, keep it real
The unified search bar is an excellent benchmark. That abstraction layer is the true test of a control plane's data model.
What often happens is the search works for asset discovery but fails for dynamic objects like security tags. A system might let you type "dev-AWS" to find an EC2 instance, but can you use that same query as a source object in a live policy rule? The demo needs to show the policy being applied and then immediately enforced on traffic, proving the query wasn't just a one-time lookup but a binding, real-time classification.
If the search is merely a pre-flight step that materializes a static list into the rule, you're right back in manual upkeep territory. The rule should contain the query itself, not its cached result.
brianh
That's a solid starting list. One thing I'd add for the multi-cloud policy check is to see how they handle "user" vs "infrastructure" identity in one rule. Like, can a rule let a person in an Azure AD group access a server tagged with an AWS IAM role? If they have to build two separate rules for that, the "single pane" is already cracked.
And on cost, make them show a forecast view. Not just "this is what you spent," but "if you move this app's backup traffic, here's the predicted bill impact." Otherwise it's just a fancy receipt 😅
Still learning.
You're spot on about the field mapping. Even if they ingest the CUR, the translation layer is where it falls apart. They'll show you a beautiful chart labeled "AWS Costs," but it's just their internal metric multiplied by a static list price.
I'd push them to show the mapping table live. Ask them to pull up a specific SKU from the CUR, like "Data Transfer Out to Internet (Asia-Pacific)," and then have them point to the exact tunnel and application in their model that corresponds to it. If they hesitate or start talking about averages, you've got your answer.
That click from spike to line item is the only real proof.
You've put your finger on the weakest point in any cloud cost integration: the data model mapping. The static price list is a huge red flag, because you'll see massive drift after a year as providers change rates and introduce new SKUs.
Your test to pull up a specific SKU is perfect. I'd take it one step further and ask them to show you the *timestamp* on the price data feed they're using. If it's not updated at least weekly via an automated, auditable sync from the cloud providers' own pricing APIs, then any forecast they show is built on sand.
The real proof is when you ask them to re-run a month's report using last month's price list versus the current one. If the numbers don't change, their system is decorative.