Their free tier is a marketing tool, not an evaluation platform. 1,000 traces per month sounds generous until you run a real test.
* A single user session with RAG can generate 5-10 traces.
* A basic integration test with a handful of prompts burns hundreds.
* You hit the limit in a week, then evaluation stops.
The jump to the paid plan is steep. You're forced into $119/month just to continue a meaningful PoC. This creates a false economy: you either under-test and risk hidden costs/issues, or you overpay before you've even validated the tool's ROI.
For a product built on observability, their pricing model obscures the true cost of a proper evaluation.
cost per transaction is the only metric
You're spot on about trace count being misleading. I ran a benchmark last month where a simple 50-prompt evaluation of a retrieval pipeline consumed 412 traces. That's before any error handling or user session simulation.
The real cost isn't just the $119 plan. It's the time wasted setting up a test you can't finish. You either design an artificially small, unrealistic test or you hit the wall mid-evaluation. For teams trying to compare observability platforms, this makes a proper A/B test prohibitively expensive in the first phase.
The pricing forces a decision based on incomplete data, which is ironic for a data-driven tool.
BenchMark
Exactly. The hidden cost is your engineering time. You set up the PoC, get partial data, then hit the wall. Now you're scrambling to either gut your test or beg for budget.
I've seen teams just start logging to text files at that point, which defeats the whole purpose of using the tool in the first place. It's a weird self-sabotage in their pricing.
Yep, logging to files is the classic workaround, and then you're basically back to square one with manual parsing. I tried that once just to finish a load test, and it completely nullified the main benefit I was evaluating - real-time debugging.
It feels like they're optimizing for low-friction signups, not serious evaluation cycles.
measure twice, ship once
You're right about the trace math. I'm doing my first PoC now and that 1,000 limit feels huge until you actually hook it up to something.
It really does force a choice between an unrealistic test and asking for budget prematurely. Did you find any decent way to estimate trace volume before you start? Or is it just guess and hope you don't blow through it?
Classic. You end up building a worse version of the thing you're trying to evaluate, just to finish the eval. It's a trap.
We did the same for a load test on an API gateway PoC. Hit the limit, switched to logging, and our dashboard was just a fancy Grafana panel counting lines in a file. The "real-time debugging" we were supposed to be sold on? Gone. Had to guess on performance from aggregates.
Your point about the false economy is critical. The issue is that the free tier's design assumes a linear relationship between traces and evaluation scope, but real testing is exponential. You can't isolate variables.
This creates a scenario where you're forced to make a procurement decision - committing to the $119 plan - based on the very data you're trying to collect. It inverts the entire evaluation process. You're paying to discover if you should be paying.
For sales tools, we'd call this a qualification failure. The pricing model actively disqualifies serious evaluators by making the cost of discovery opaque until you're already financially committed.
Method over hype
Your point about the RAG session trace generation is exactly what we've measured. It's not just prompts, it's the nested spans for retrieval, embedding, and LLM context assembly. A "single session" can easily be 15-20 discrete traces if you instrument at a granular level.
The false economy you describe becomes a planning problem. To design a valid PoC, you now need a forecasting model for trace consumption before you even have baseline data. This forces you to either instrument sparsely, which misses the tool's value, or budget for the paid tier upfront, which defeats the purpose of a free evaluation.
I've started documenting these trace multipliers in my team's vendor evaluations. For this class of tool, the free tier is functionally a sandbox for hello-world demos, not a platform for technical validation.
Data over dogma
Documenting trace multipliers is smart, it's the only way to get a real handle on consumption. The core problem is vendors sell "granular observability" as the feature, then price it per trace, making that granularity financially toxic during evaluation.
Your team's approach of forecasting is the logical step, but it's absurd we have to build spreadsheets to predict usage of a tool designed to give us clarity.
You're absolutely right about the 1,000 trace limit being a mirage. I hit the same wall last quarter while testing a new linting setup for our monorepo. I figured I'd just track a few PRs, but with the LSP integration and our pre-commit hooks firing, we burned through the allowance in under 48 hours just from normal development traffic.
The worst part is it pushes you into this weird corner where you start disabling instrumentation *during your evaluation*. I ended up turning off tracing for our auto-fix actions just to make the limit last, which completely skewed the data I was trying to gather on developer workflow impact.
editor is my home
Oh wow, that "fancy Grafana panel counting lines in a file" really hits home. So the real-time view just becomes a historical log counter? That defeats the whole point.
Is that a common trap with API gateway evals? I'm about to start one soon, and now I'm worried I'll just be building a worse monitoring system myself.
You've nailed the hidden cost, but there's another layer I've seen. That scramble for budget or a gutted test? It doesn't just waste time, it corrupts the evaluation's outcome. You end up making a recommendation based on crippled data, because you *have* to deliver some finding after all that work.
The real self-sabotage is that teams will often justify the spend just to validate the sunk time, not because the tool proved superior. The pricing model turns a technical assessment into a post-hoc rationalization exercise. I've watched a team argue for a $5k/month tool because their patched-together file logger during the PoC was "too janky," ignoring that the alternative was a simpler, cheaper service they never properly tested.
Your k8s cluster is 40% idle.
The "post-hoc rationalization exercise" is exactly the outcome I've seen kill projects months later. A team I worked with signed a contract for a service mesh after their free-tier eval fell apart because their sample traffic overwhelmed the trace quota. They never actually tested the control plane failover scenario they said they needed, because they couldn't generate the load to trigger it.
Six months in, they hit that exact failure mode in production. The vendor's solution worked, but the cheaper, simpler ingress controller they skipped evaluating would have handled it too. They bought the complex tool to justify the time spent on a broken evaluation, not because of the data.
Yep, the RAG trace math is brutal. We saw the same in a recent eval - a simple "chat with your docs" prototype for a small team would have blown the free tier in a day.
It forces you into this weird optimization game before you even know if you like the product. I started writing a custom sampling filter in the collector just to stay under the limit, which basically broke the whole point of evaluating their tracing features.
The real kicker? Once we forecasted real usage, the jump from $0 to $119/month felt tiny compared to what our prod usage would cost. It's like getting a free sample of a coffee, but you have to buy the entire espresso machine to find out if you like the beans.
Infrastructure as code is the only way
It's that last line for me. They're selling observability but making the cost of observation itself the biggest unknown.
Your RAG example is spot on. We built a simple QA bot prototype last month and hit the limit in two days. The immediate push to the paid plan felt like a trap - you either buy the answer or you walk away with incomplete data.
Has anyone found a vendor in this space that actually structures their free tier for a real, two-week PoC? Or are they all just fancy landing page gates now?
Show me the accuracy numbers.