While reviewing Arize AI's documentation for a compliance audit requirement, I discovered a feature that isn't prominently highlighted but is critically important for teams operating in regulated environments or those needing to perform independent analysis: the ability to export raw inference logs via their Python SDK and API.
This moves beyond the platform's standard visualizations and automated drift reports, providing direct access to the underlying prediction and feature data sent to Arize. For anyone needing to verify the data used in their model monitoring, reconcile counts, or perform their own statistical analysis outside the UI, this is a game-changer. It effectively turns Arize into a queryable log store for your model's inferences.
The process involves using the `get_prediction_records` and `get_actual_records` methods. You need to specify a time range and can paginate through results. The key is ensuring your `prediction_id` linkage is robust when sending data, as that is the primary key for retrieval.
Here's a basic example of pulling a batch of prediction logs:
```python
from datetime import datetime
import arize
arize_client = arize.Client(api_key=ARIZE_API_KEY, space_key=ARIZE_SPACE_KEY)
response = arize_client.get_prediction_records(
model_id="your_model_id",
start_time=datetime(2024, 1, 15),
end_time=datetime(2024, 1, 16),
limit=100,
before_prediction_id=None # Use for pagination
)
if response.status_code == 200:
records = response.result.data
# 'records' is now a list of dictionaries containing your raw inference data
for record in records:
print(f"Prediction ID: {record.prediction_id}, Features: {record.features}")
else:
print(f"Failed: {response.error}")
```
**Important Considerations & Methodology:**
* **Rate Limits:** Be mindful of API rate limits when pulling large volumes of data. Implement pagination and backoffs for production scripts.
* **Data Completeness:** This only retrieves data that was successfully sent and ingested by Arize. Any failed `log` calls from your production service won't appear here.
* **Audit Trail:** Combining these logs with your own application logs (using the `prediction_id` as a correlation key) can create a complete, verifiable trail from user request to model prediction to Arize observation.
* **Cost Implications:** While the API access itself doesn't carry a direct extra charge, you are limited by your plan's monthly event volume. Pulling logs doesn't incur new events, but be aware of your ingestion caps.
For our use case, this capability allowed us to independently verify the drift metrics Arize was calculating and feed the same raw data into an internal benchmarking system. It shifts the platform from a closed monitoring solution to an open component in the MLOps stack.
-ck