Hey everyone, wanted to share something our team ran into while testing Cartesia's topic clustering feature on a set of customer support transcripts. We found what we believe is a reproducible bug in the output structure.
We were using the API to cluster about 500 text snippets. The clusters themselves seemed logically sound, but the JSON response had an issue with the `cluster_id` assignment. For a small subset of items (around 5%), the `cluster_id` was returned as `null` even though the `topic_label` was correctly populated and the item was clearly grouped within a larger cluster in the `clusters` array.
Here's a simplified example of the problematic output snippet:
```json
{
"clusters": [
{
"cluster_id": 2,
"topic_label": "Billing Inquiry",
"items": [
{"id": "msg_001", "text": "Why was I charged?"},
{"id": "msg_002", "text": "Invoice not received"}
]
}
],
"unclustered_items": [
{"id": "msg_003", "text": "Need a refund", "cluster_id": 2, "topic_label": "Billing Inquiry"}
]
}
```
Notice `msg_003` is in `unclustered_items`, but it has a `cluster_id` and `topic_label` matching cluster 2. This seems like an internal assignment error.
We've already submitted this to their support with our full request/response payload and are waiting on a ticket number. In the meantime, we built a temporary post-processing filter to reassign these items based on matching `topic_label`.
Has anyone else noticed inconsistencies in the clustering output, particularly with ID assignment? Curious if it's specific to larger batches or certain text characteristics.
hth
Interesting find, but have you checked if this actually costs you anything? A 5% null rate in a clustering feature might not impact your bottom line unless you're processing enormous volumes. I'd be more curious about the compute time wasted on those orphaned items.
If you're paying per API call, a bug that forces you to re-cluster or post-process data could inflate your bill. Did you notice any pattern in *which* items get the null assignment? Low-confidence snippets, maybe? That'd be a different kind of problem.
cost_observer_42
You're missing the point. The cost isn't the 5% null rate, it's the workaround.
Now I have to build and maintain a post processing layer to clean this data before my app can use it. That's development time, ongoing compute for the cleanup job, and increased complexity. Every API call now needs a downstream fix. That's pure waste.
Pattern in the items? It's random. That's worse. If it was low confidence snippets I could filter them upstream and pay for less input. Random bugs just trash the output's reliability.
show me the bill
Completely agree about the downstream cost being the real issue. That post-processing layer isn't just a one-time script, it's a permanent, brittle component in your data pipeline.
You've nailed the reliability problem, but I'd add that the "randomness" also blocks any meaningful SLA. You can't guarantee data cleanliness to stakeholders when 5% of items might fail silently. It forces you into a defensive, quality-check stance for every single job, which defeats the purpose of using a managed API.
My team hit a similar wall with a different vendor's sentiment analysis years ago. We eventually had to switch because the operational overhead of sanitizing outputs consumed more engineering cycles than building a simpler, less accurate model ourselves.
Support is a product, not a department.