Just started a new role where my first project was picking a BI tool for the whole company. My main worry was performance—we have a lot of teams that would hit the dashboards at the same time.
I set up a test with 50 concurrent users running typical queries on our main dataset (about 5 million rows). I tried Power BI, Tableau, Looker, Metabase, and Sigma. The results were... surprising. One tool really struggled with more than 10 users at once 😬 Another handled it fine but the cost was wild.
Has anyone else done this kind of load testing? I'm curious if my experience with dashboard latency under load matches what others have seen.
I did something similar last year when we were scaling up our sales dashboards. The tool that choked at 10 users - was it Metabase? We saw exactly that with their default setup, though tweaking the database connection pooling helped a bit.
The cost vs performance trade-off is real. We had a tool that handled load beautifully, but the quote made our finance team wince. Ended up going with a mid-tier performer that had predictable pricing, because consistent latency for 30 people was better than lightning speed for 5.
Would you consider sharing which dataset type you used? In our case, columnar storage made a huge difference under concurrent load versus row-based, regardless of the BI layer.
✌️
Great to see someone doing real-world load testing! That 10-user choke point is usually about connection pooling, like user575 mentioned. We saw it with Metabase's default setup too until we tuned the app server threads and database max connections.
For your 5 million row dataset, was it a live connection or an extract? With Power BI especially, pushing an extract to their service can handle concurrent users way better than live queries hitting your warehouse directly. The pricing shocker was probably Looker or Tableau - their per-user costs add up fast at 50 seats.
Would love to see your actual latency numbers. We logged average response times during our tests and found one tool that looked fine on paper had huge latency spikes every few queries.
Always A/B test.
50 users is a solid test, that's a lot of concurrency to handle. Which one was the 10-user choker for you? I'm guessing Metabase since that seems to be a common theme.
I'm actually about to set up a similar test at my place. Did you use a specific tool for generating the load, like k6 or locust? And were all the tools hitting the exact same backend database? That'd make the comparison super interesting.
Containers are magic, but I want to know how the magic works.
The choke was indeed Metabase on default config. Tableau also had bad latency above 20 users, but Metabase fell over first.
I used k6 for load testing. It scripts well and logs response times. All tools pointed at the same Snowflake instance, same warehouse size. That's the only way to isolate the BI layer overhead.
Biggest caveat: your database connection pooling matters more than the BI tool itself sometimes. But Metabase's app server still queued requests even with a tuned DB pool.
Benchmarks don't lie.
Metabase's app server queue is the real bottleneck. Their whole architecture is naive for concurrency.
>pointed at the same Snowflake instance, same warehouse size
That's key. Too many people blame the database when their BI layer can't pool or cache.
If you're hitting Snowflake directly, why even use a heavy BI server? A simple dashboard framework with a good connection pool can handle this. You're paying for complexity that's failing under load.
Simplicity is the ultimate sophistication