Alright, I'm pulling my hair out here and need to know if it's just me. I've been trialing New Relic against Datadog and Dynatrace (my usual suspects) for some basic infrastructure and app monitoring.
My setup is pretty standard: a few custom metrics from our app, some APM from a Node service, and NRQL alert conditions on those. The dashboards show the data fine—spikes, dips, the whole story. But the alerts? They're... sleepy. I'll get a flurry of notifications for one threshold breach, then complete radio silence for what looks like an identical breach an hour later. No changes in the policy, no changes in the data flow.
Here's what I've checked so far:
* The alert conditions are `Static` with a simple threshold (e.g., `SELECT average(duration) FROM Transaction WHERE appName='MyApp' FACET route`).
* The evaluation window and expiration timer seem sane.
* I'm not hitting any obvious data cardinality limits on the facets.
It feels weirdly similar to the "alert fatigue" tuning you do in other platforms, but this is the opposite—alert *absence*. I'm wondering if there's some internal cooldown or a default aggregation nuance I'm missing.
So my question to the forum:
* Is this a known "quirk" or issue with New Relic's alerting engine, especially around faceted NRQL conditions?
* How does its alerting reliability *really* stack up against something like Datadog's monitors or Dynatrace's problem detection in your experience?
* Any gotchas in the NRQL alert config that aren't immediately obvious? I'm so used to the GUI sliders in Pipedrive or HubSpot for alerts that maybe I'm misreading the NRQL semantics.
Trying to decide if I need to deep-dive into the config or if this is a known trade-off. The ingestion costs look good, but if the alerting is flaky, that's a dealbreaker.
Still looking for the perfect one
> radio silence for what looks like an identical breach an hour later
Had this too, was maddening. Did you check the signal expiration time on the NRQL condition? If a signal expires, it won't fire again for the same facet value until a new signal is created. Could explain the gap.
Also, their baseline alert conditions are way more consistent for me than static thresholds, weird as that sounds.