Just starting to look into ML monitoring and Arize AI keeps coming up. Their free tier looks generous on paper, but I need to know if it's actually functional for a real pilot project.
Has anyone here run a proper pilot on the free plan? I'm thinking:
* Connecting a single model pipeline
* Tracking basic drift & performance
* Maybe setting up a few simple dashboards
Or is it too limited? I don't want to waste a week setting things up only to hit a dealbreaker wall. Looking for hands-on experience!
Trial first, ask later.
Yes, it's functional for a pilot. I ran a churn prediction model through their free tier for a three-month proof of concept. The main constraint isn't the feature set for basic monitoring, it's the data retention and granularity. You'll get 13 months of metric history, but only at a daily roll-up, not the raw event-level logs.
That means your drift detection will work on a daily basis, which is fine for most pilot scenarios where you're looking for trend validation. The wall you might hit is if your pilot requires analyzing performance per inference batch or tracing a specific prediction's path. For setting up dashboards on a single model with basic drift and performance, you'll have what you need.
Just be precise about your pilot's success criteria. If it requires sub-daily analysis or exporting raw data for a custom audit, you'll need to upgrade.
Data doesn't lie, but folks sometimes do.
That's a helpful distinction about daily roll-up vs event-level logs. I'm also looking at a pilot and the daily granularity is probably fine, but I'm unsure about the setup effort.
When you say "tracing a specific prediction's path" is limited, does that mean the free tier won't let you drill down from a dashboard anomaly to see the individual inferences that caused it? You'd just know something spiked on a given day?
Still learning.