Hello everyone. I've been conducting a detailed evaluation of Arize AI for a client's MLOps stack, specifically focusing on its monitoring and observability capabilities. While the platform's core metrics tracking and automated anomaly detection are quite robust, I keep hitting a friction point that's impacting my overall efficiency scorecard: the user interface for cohort analysis.
My workflow typically involves slicing model performance by specific segments—think user geography, input value ranges, or a particular model version. In Arize, I find the process of defining, saving, and reusing these cohorts to be unnecessarily convoluted. For instance, the multi-step process to create a compound filter (e.g., `prediction_confidence > 0.8 AND user_region = 'EU'`) feels like it requires more clicks and modal navigation than it should. Once a cohort is created, applying it across different dashboards or views doesn't feel as seamless as I'd expect from a tool at this price point and market position.
From a procurement and vendor evaluation standpoint, this touches on key criteria in my standard UX efficiency matrix:
* **Learnability:** How quickly can a new team member perform this core task?
* **Operational Efficiency:** How many steps (clicks, navigations) are required to complete the task?
* **Consistency:** Is the interaction pattern for cohort management consistent with other filter/slice operations in the platform?
I'm curious if others in the community have had similar experiences. Specifically:
* Have you developed any workarounds or best practices to streamline cohort analysis within Arize?
* In your comparisons with other observability platforms (e.g., WhyLabs, Fiddler, custom Grafana setups), how did Arize's UI for this specific function weigh in your decision matrix?
* Has anyone had success in feeding this specific UX feedback through their account channels, and if so, what was the responsiveness like?
Understanding these practical, day-to-day interactions is crucial for building a complete vendor assessment, beyond just checking feature boxes on a capabilities list.
null