Used it for a POC on our logging pipeline. Fast to wire up, got semantic search over error clusters in a day.
Tried to move it to production. Fell apart.
* **Ingestion is brittle.** Their `SimpleDirectoryReader` choked on a malformed log file and the whole pipeline died. No dead-letter queue, no skip option.
* **Query performance is unpredictable.** Fine with 10k documents, latency spiked at 50k. Their "auto-vectorization" picked a terrible chunk size.
* **Observability is non-existent.** Can't trace a query through retrievers/LLM calls. No built-in metrics export to Prometheus. You're blind.
Example: tried to add a simple request duration metric.
```python
from prometheus_client import Histogram
REQUEST_DURATION = Histogram('llama_index_request_duration_seconds', 'Request duration')
@REQUEST_DURATION.time()
def query_index(question):
# your query engine call
return engine.query(question)
```
Had to wrap everything myself. The abstraction leaks everywhere when you need real reliability.
Great for a weekend project. For anything serving real traffic? Use the patterns, not the framework. Build your own retrieval stack with LangChain or directly with the vector DB client.
just the metrics
Totally feel this, especially on the observability point. That "wrapping everything myself" step is the hidden tax on moving from POC to production that nobody talks about.
You mentioned building your own retrieval stack - for your logging pipeline, did you consider using the vector DB's native APIs directly (like Pinecone or Weaviate) for the core retrieval, and only using LlamaIndex's patterns for the chunking and LLM interaction? That's a compromise I've seen work: you get the rapid prototyping for the document processing chain, but hand off the critical query path to something you can instrument properly.
The auto-vectorization picking a terrible chunk size is a classic example of a "helpful" abstraction that backfires at scale. I've had to drop down to manually defining text splitters and metadata strategies for any serious volume, which makes you wonder what the framework is saving you at that point.
Stay connected