Hey everyone! I’ve been trialing Freeplay for the last few weeks on a side project where we’re building a small AI feature. I’m really excited about the platform for managing prompts and evaluating outputs—it feels like it could be a game-changer for our workflow.
But I’m hitting a wall with the pricing, specifically the trace costs. In their docs and plan estimator, it seemed like our usage would fit comfortably in the Team plan. However, our actual trace costs in the dashboard are already way higher than the estimate after just a few days of active development. We’re not even in production yet! 😅
I’m wondering if anyone else has experienced this? Are there specific settings or best practices to actually keep trace costs down? Like, are we logging too much data per trace by default? Or is the estimator just super optimistic for dev/testing phases?
I want to advocate for us to adopt Freeplay, but I need to go to my team with a realistic cost picture. Any tips from people using it daily would be super helpful! Thx!
Yeah, that's a familiar story, and it's a really important point to raise before you bring it to the team. The estimator often assumes a production-level efficiency that's just not there during active development and testing, when you're sending many more traces as you iterate.
One thing to check is whether you're using their sampling or trace filtering settings. During development, you can often log only a percentage of traces or exclude certain debug or test runs, which can make a big difference. The default can be quite verbose.
It might also be worth reaching out to their support with a screenshot of your dashboard. They can sometimes point out a specific setting that's inflating your counts. Getting that realistic picture from them directly could strengthen your advocacy.
—HR
That's a solid point about development inefficiency versus the estimator's production assumptions. The key I've found is that you need to formalize those debug/test exclusions as early as possible, otherwise they become a manual, error-prone task.
For example, we set up a dedicated environment variable or header for our integration tests that triggers a rule to drop all associated traces at the collection point. Without a structured rule like that, you're right that the verbosity can balloon quickly during iteration cycles.
Have you seen any patterns in what specifically gets over-logged during development? For us, it's often internal validation steps that generate their own sub-traces.
Formalizing exclusions early is absolutely critical. I'd extend your environment variable example by suggesting a multi-stage filtering strategy based on telemetry maturity. We categorize traces by three metadata dimensions: environment (dev/staging/prod), request purpose (integration_test, user_flow, evaluation), and trace criticality (debug, monitor, audit). A pre-collection processor applies rules to each dimension.
During development, we've quantified that validation sub-traces and automatic prompt template renders account for roughly 40% of our trace volume. The key isn't just dropping them, but sampling them at a lower rate, say 10%, because they still provide value for debugging specific failures. You can implement this with a head-based sampling rule before the trace leaves your application.
The pattern we see most is not just sub-traces, but entire traces from automated CI/CD runs that iterate through numerous test permutations. Without filtering, a single pipeline run can generate thousands of traces that mirror production volume but have zero analytical value.
Data first, decisions later.
Reaching out to support to "strengthen your advocacy" assumes the estimator is a good-faith tool. I'd treat it more like a marketing asset. Have you compared the estimator's assumptions about "a trace" with what your SDK is actually sending? The unit cost often hides a dozen nested spans.
Their sampling settings are a start, but they're applied after the trace leaves your system, so you're still paying for the network egress and initial ingestion. You need to sample client-side, in your application, before you call their API. Otherwise you're just filtering a bill that's already been run up.
- Nina