Skip to content
Notifications
Clear all

How do you deal with mobile app burst traffic blowing your APM budget?

2 Posts
2 Users
0 Reactions
2 Views
(@cost_cutter_99)
Estimable Member
Joined: 4 months ago
Posts: 124
Topic starter   [#13590]

We all know the scenario: a new mobile app feature goes viral, or there's a holiday sale spike. Congrats on the traffic! But then the next APM bill arrives and it's... painful. The per-host or per-gigabyte pricing that was predictable for steady-state web traffic doesn't hold up when you have millions of mobile sessions sending events concurrently.

I'm specifically wrestling with this on Datadog APM (but the principles apply to New Relic, Dynatrace, etc.). Their pricing is based on "APM Hosts" or "APM Spans per Indexed Hour," and a burst of mobile users can spin up hundreds of ephemeral backend containers or Lambda invocations, each counted as a host, or generate a tidal wave of spans.

My current spreadsheet breakdown shows our baseline cost gets 3-4x'd during a 48-hour traffic event. We've tried:
* Dropping sampling rates dynamically, but that's tricky to get right without losing critical error paths.
* Filtering out health-check and certain high-volume, low-value spans (like specific analytics calls).
* Moving to trace-level sampling instead of head-based, but the math on cost savings is murky.

What I'm curious about:
* Are teams using a sidecar or agent configuration to pre-filter or aggregate before the APM ingest?
* Is anyone successfully using a different pricing model (like ingest credits) that better absorbs bursts?
* Have you moved to an open-source APM (like SigNoz, OpenTelemetry Collector with a SaaS backend) specifically for this unpredictability? What's the operational trade-off?

The goal is to keep the visibility we need during an incident without the bill becoming an incident itself.



   
Quote
(@cloud_ops_learner)
Reputable Member
Joined: 2 months ago
Posts: 143
 

Totally feel your pain on the Datadog APM host spikes. We saw something similar last Black Friday.

What about using a different APM for the mobile backend during those bursts? Like, keep Datadog for steady-state but route a percentage of spans from new containers to a cheaper, usage-based tool during the event. Might be complex to set up though.

Do you think the vendors will ever offer true burst-friendly pricing, or is that against their whole model?


Still learning


   
ReplyQuote