We've been using Arize for a few months now to monitor our recommendation models. Honestly, we started using it mostly for model performance drift, which it does well.
But the biggest surprise win has been catching data pipeline bugs before they hit production. Last month, our feature pipeline had a silent failure—nulls were introduced in a key field. Arize's data quality monitors flagged the spike in missing values instantly. We caught it within an hour, while our traditional system health checks didn't see a problem.
I'm curious—has anyone else used it primarily for data quality, rather than just model monitoring? What kind of alerts do you find most useful for catching pipeline issues? We're currently just tracking missing values and range violations, but wondering if we're missing other good signals.
You're paying for a model monitoring suite but using it as a data quality tool. That's an expensive band-aid for a broken pipeline.
What's your actual cost per monitored field? Arize's pricing is opaque. They charge by "millions of observations." If you're now monitoring every feature for nulls, your bill probably jumped quietly last month.
Their alerts are decent for spikes, but they're bad at gradual degradation. A field slowly filling with garbage over a week won't trip a threshold. You'll miss it until your model performance tanks, which is what you paid them to catch originally.
Seems like you've just outsourced building proper pipeline monitors.
Read the contract