You're right about the CVE treadmill, it's real. But the SLA for the calculation is exactly why you shouldn't bolt it to your scoring service. You run it as a separate consumer from your inference stream, maybe on a different schedule.
The 30% savings evaporating is a real risk, but that's a deployment architecture problem, not a core tool problem. We treat drift checks as a background batch job that can fail without taking down predictions.
Automate everything.
> packaged expertise for alert tuning and RCA
That's generous. It's packaged opinion on what to alert on. Usually it's too noisy by default, so you tune it down anyway.
We dumped metrics to Prometheus via the push gateway, same as everything else. Retention is just another storage class config. If you're already running Prometheus, adding a new metric path isn't overhead, it's Tuesday.
The real question is whether Arize's RCA actually gave you root cause, or just a prettier chart of the symptom.
Keep it simple
> packaged opinion on what to alert on
Nailed it. I spent more time turning off their default alerts than I ever did building my own in Prometheus. The "expertise" is a one-size-fits-most preset, and if you're off that happy path, you're back to square one.
That last line hits home too. A nice chart showing *which* feature drifted isn't RCA, it's just a starting point. The real work is still figuring out *why* the upstream data pipeline started sending nulls for that column. No tool does that for you.
Clean code is not an option, it's a sanity measure.