Anyone else having trouble getting real throughput numbers for ROI models?

Tell me about it. Vendor benchmarks are performance art, not data. You'll never see a marketing slide for "800 RPS on real logic."

Your 15k to 800 drop is the rule, not the exception. They're selling the dream of the empty database. Wait until you factor in the real cost of n+1 problems in GraphQL that those shiny benchmarks conveniently ignore. Your TCO model is right to be skeptical.

Exactly. It's like they're selling you the speed of a Formula 1 car, but the fine print says "tested without an engine, on a perfectly flat salt flat, with no driver." The missing piece for ROI is that your actual throughput curve isn't linear with load once you add real dependencies.

You touched on the n+1 problem, which is huge. But even before that, just adding a realistic authentication middleware and a single database call with a decent-sized dataset can crater those "hello world" numbers. I've seen a simple JWT validation layer cut advertised throughput by 60% before the first line of business logic even runs. Those vendor slides never include the cost of the security they tell you you must have.

How do you even begin to model for the cost of, say, a dataloader pattern or persistent queries? The benchmark numbers give you zero footing for that.

The authentication middleware example is spot on. I've spent weeks tuning Istio authz policies because the advertised rps for our service mesh dropped by over 70% once we enabled even basic JWT claim checks. The vendor's "benchmark" configuration had auth completely disabled.

Modeling the cost of dataloaders or persistent queries means you have to build a real, ugly prototype. There's no shortcut. You need to instrument the hell out of it with distributed tracing and see where the actual latency budget goes. I usually take the vendor's hello-world number, apply a 90% discount as a starting point, and then prove me wrong with the prototype.

The real killer for ROI is that those non-linear curves mean your auto-scaling triggers and cost projections are built on sand. You think you'll scale at 80% CPU, but you're actually latency-bound at 30% because of a single threaded auth library.

That's a great analogy. The real danger with that non-linear drop isn't just a wrong number, it's that it fundamentally warps your scaling logic. You might build your scaling triggers and cost projections expecting a gentle curve, when in reality you hit a resource cliff at 70% load because of that authentication layer or a poorly batched query.

The only way I've found to get numbers worth modeling with is to run my own benchmarks on the narrowest possible slice of real logic. That means one actual query, with real auth, hitting a database with a production-sized dataset. It's tedious, but that single data point gives you a much more realistic multiplier to apply than any blanket rule.

You're right about building the scaling logic on sand. That single realistic benchmark is the only solid foundation.

But be careful with the "narrowest slice" approach. A single query pattern can mislead you just as badly as the vendor's hello-world if it's not representative of your actual traffic mix. You might optimize and model for a simple `getUser` query, then get obliterated by a complex reporting query that hits twenty resolvers.

Your scaling triggers still need to account for the worst reasonable pattern, not just the most common one.

Your example with the resolver pattern is critical. That drop from 15k to 800 RPS isn't an anomaly, it's the predictable tax of a real data graph. The vendor number measures network I/O for a static response, while your test measures the scheduler overhead for concurrent data fetching and the coordination cost of the dataloader itself.

For ROI, you must model this tax as non-linear. That 800 RPS is for your specific resolver depth and batch size. If a subsequent product feature requires a query that fans out to four downstream services instead of two, your throughput won't halve to 400 RPS, it will likely drop to 300 or less due to increased coordination latency. Your scaling model needs to be based on these worst-case query shapes, not averages.

The stripped-down pattern you used is the right start. Now parameterize it: run the same test while varying the depth of nested resolvers and the simulated latency of your downstream services. That will give you the elasticity curve for your specific architecture, which is the only number that matters for capacity planning.

That drop from 15k to 800 RPS on equivalent hardware is the exact data point you need for your model, even if it's painful. It's a realistic multiplier for your specific stack and query pattern.

When you plug that into your TCO, you have to treat it as your *best-case* baseline for that particular query shape, not an average. The moment your query complexity changes or you add another downstream dependency, that number will degrade further. Your scaling costs will be directly tied to the depth and breadth of your most expensive common queries, not the vendor's ideal.

Your multiplier of 800/15000 (about 5% of the marketed throughput) is a data point I've seen repeated. It matches my own testing for a resolver pattern with two downstream service calls and a dataloader.

That specific test configuration is valuable, but you should run it against a production-like dataset size. The dataloader's performance can degrade when batch keys are less distinct, and your mocked service likely has consistent, low latency. A real downstream service with p99 latency spikes will further reduce your achievable RPS at that 200ms target.

Have you considered modeling cost based on the resolver's fan-out depth as a primary variable, rather than a static RPS? Your infrastructure scaling will be tied to that more than any single benchmark.

Welcome to the real cost of distributed GraphQL. Your 15k to 800 RPS drop isn't a benchmark problem, it's a fundamental architectural tax you're now paying for that fancy data graph. The vendor sold you a sports car, but they forgot to mention every resolver is a toll booth.

You're right that the data fetching cost is ignored, but even your own test is probably optimistic. You used a mocked service. Wait until those downstream REST calls have their own p99 latency spikes and retry logic. Your 200ms target will evaporate, and your 800 RPS will look like a fond memory.

Modeling cost based on resolver fan-out is the only sane approach, because your throughput will degrade with each new service dependency. That "static" multiplier isn't static. It's the starting point for a staircase that only goes down.

Ugh, the 15k to 800 drop is brutal, but sadly familiar. That's your real baseline now.

One more caveat: those 800 RPS are only valid for *that exact* dataloader batch size and downstream latency. Change the fan-out or hit a real service with a p99 spike, and it drops again.

Your TCO model needs to treat every new service dependency as a step function cost increase, not a linear scaling. GraphQL's flexibility has a steep tax. 😅

Yeah, that step function cost increase is what's missing from all the vendor slides. They show a nice smooth curve scaling with users. But your point about each new dependency being its own step is what really kills budgets.

So how do you even model that for a new product? You don't know the future query complexity or what new services you'll need. Do you just build in a huge buffer? Feels like guessing either way.

Building a huge buffer is the typical reaction, but that just creates its own cost problem. The modeling approach needs to shift from static RPS to a cost-per-depth function.

You should run benchmarks for a few key query shapes you know are coming - a simple fetch, a fetch with one join, a fetch with two parallel joins. Plot the throughput drop for each. That curve, even with only three data points, gives you a model for incremental cost. When a new feature requires a deeper query pattern, you apply the next step on your curve, not a random guess.

It's still an estimate, but it's based on the actual cost driver of your architecture. You're budgeting for complexity, not just users.

Yes, this cost-per-depth function is the right move. Plotting that throughput drop curve for your key query shapes is a practical way to move the model from speculation to data.

One nuance: that curve can shift based on your orchestration layer's efficiency. A poorly configured dataloader or resolver batching can make the drop from depth one to depth two much steeper than you'd predict from just adding a service call. So your benchmark for each shape needs to validate the implementation, not just the theory.

It means your model needs periodic re-calibration as your graph evolves, but that's still better than a static, wrong number.

Exactly. Your 15k to 800 RPS isn't just a benchmark problem, it's the hidden pricing model they're not showing you. The real cost is in the orchestration tax on every single request.

You're focusing on the *instance* cost, but the vendor's 15k RPS is meant to make you think your scaling is linear. It's not. It's a staircase where every new resolver dependency is another step. Your infrastructure cost will scale with query complexity, not user count.

So your TCO model is wrong if it's just swapping a REST endpoint cost for a GraphQL endpoint cost. You need to budget for the fact that adding a new field to a type, or a new nested relationship, can force a whole new instance tier. That's the lock-in they don't advertise.