Hey everyone! I'm diving into Arize for the first time and feeling a bit overwhelmed. We've got a text classification model in production that's been running smoothly, but I want to set up proactive monitoring for concept drift. The documentation is comprehensive, but as someone new to the platform, I'm not sure about the best entry point.
I've already:
* Containerized our model service with Docker
* Set up a basic CI/CD pipeline that logs predictions to a data warehouse
* Familiarized myself with the core concepts of embeddings and drift in NLP
My main questions are:
* For an NLP use case, should I start with the Python SDK or jump straight into the UI to configure my first project?
* What's the minimal set of data I need to send over to get a useful drift signal? Is it just the text input, prediction, and actual label, or do I absolutely need to send model embeddings from day one?
* Are there any specific pitfalls in the initial setup for text data that I should avoid? I've heard timestamp formatting can be tricky.
I'm excited to get this integrated and share my setup process back with the community. Any guidance from those who've been through this would be fantastic.
Ship fast, measure faster.
Start with the SDK. The UI project setup will just ask for things you haven't sent yet, so it's a chicken-and-egg problem. Get data flowing first.
> minimal set of data
You need timestamps, prediction IDs, features (your raw text), predictions, and actuals. You can skip sending embeddings initially - Arize can generate them for common model types. But check their supported list. If you're using a custom transformer, you'll have to provide them later.
Biggest pitfall? Timestamps in your log table aren't necessarily the "prediction timestamp" Arize needs. If your pipeline batches logs, you're feeding it stale timestamps and your drift detection is garbage from day one. Send the time the model actually made the call.
show me the bill