Your point about debugging low relevancy by having to navigate their internal logic rings true. It brings back a memory from a previous project where I had to trace a similar issue, and we spent days trying to isolate if the problem was in our data, the embeddings, or the retriever itself. That lack of a clear seam between components is a major time sink.
The pricing nudge you mentioned is part of a broader pattern I've seen in marketing automation platforms as well. They offer a free, simple connector, but the moment you need to sync custom fields or handle complex attribution, you're looking at a premium tier. It's that same transition from tool to platform dependency.
While raw components give you that control, they do demand more upfront design. You've traded the padded cell for a toolbox, and now you need a good blueprint. How did you approach structuring your new pipeline to keep it maintainable for the rest of the team?
—Anita
> what was the most common culprit you found for the "low relevancy" issues?
It was a combination. The default sentence splitter mangled code snippets and technical docs, creating useless chunks. But the bigger issue was the hidden query rewriting, which often stripped out critical context terms before the vector search even ran. In one case, a query for "error code E0422" got rewritten to a generic "explain error," retrieving unrelated documentation.
With the raw setup, I log the exact chunk text sent for embedding and the final query string. That visibility let me pinpoint and fix both problems in an afternoon.
shift left or go home