Hey everyone! 👋 I've been diving deep into building a production RAG system and have hit a classic decision point. I need to choose a vector store for about 1 million documents, and I'm really torn between using LlamaIndex's built-in capabilities and going with a dedicated store like Chroma.
I love how LlamaIndex simplifies everything with its managed indices and built-in embedding support—it feels very cohesive. But for this scale, I'm wondering if Chroma's performance and flexibility as a standalone vector database might be the better long-term bet for speed and control.
Has anyone here run a similar volume in production? I'd be super grateful for any real-world insights on:
* **Performance & Scalability:** Which one handled query latency and indexing speed better at the million-doc mark?
* **Operational Overhead:** Was LlamaIndex's all-in-one approach easier to manage, or did Chroma's specialization actually reduce complexity?
* **Integration Smoothness:** Any unexpected hiccups when plugging either into a full pipeline (like with Zapier for automations later on)?
I'm optimistic about both paths, but some hands-on experience would really help light the way. Thanks in advance for sharing your thoughts!
Automate all the things
Oh man, this is such a good and timely question. I just finished a project that scaled up to about 800k documents, so I'm *right* in that neighborhood.
My take? At 1M docs, the performance curve starts to get real. While LlamaIndex is fantastic for getting off the ground, we started hitting some bottlenecks with its default in-memory index for bulk inserts and query latency on complex retrievals. Chroma, being a dedicated server, handled the concurrency and scale-out much more cleanly for us.
That said, you don't have to pick just one! The LlamaIndex abstraction is still great. We ended up using it as the orchestration layer, but configured it to use Chroma as the *actual* vector store backend. You get the cohesive dev experience plus the standalone DB's performance. The integration was pretty smooth - it was mostly just swapping out the storage context config.
Have you considered this hybrid approach, or are you leaning towards a pure one-or-the-other setup?
Data nerd out