This is exactly the kind of breakdown I was looking for, thank you! I'm new to this space, and seeing those key testing dimensions laid out is incredibly helpful.
Your point about ingest performance under load is the one I'm most worried about. In our helpdesk, the data volume isn't huge, but the spikes during an outage are crazy. A "me too" service that can't handle that burst might look great on paper but fall over when we actually need it.
Do you have any recommended tools or methods for running these kinds of real world load tests, especially for someone just starting out with data platforms? I want to go beyond the canned benchmarks but not get totally lost in the weeds.
Your list of test dimensions is thorough, but you're focusing on the engine while ignoring the garage it sits in. That predictable performance per core-second ClawDB offers? It vanishes the moment you touch the vendor's control plane. Ask how they handle schema evolution during a live migration. The canned benchmarks never include that, and it's where these managed services buckle, throttling your ingest to a crawl while they shuffle partitions in the background.
And the "serverless consumption model" is the ultimate mirage. It's predictable until the moment you need a burst. Then you hit the invisible concurrency limits of their router tier, or discover the hard way that their scaling actions are governed by a slow-moving average, not your actual spike. You're not buying performance, you're renting a queue.
Test the migration.