Hello everyone. I've been exploring LlamaIndex for a few months now, specifically for building specialized RAG agents in a B2B context. I wanted to share a concrete application I've built and deployed internally: a contract analysis agent.
The agent is designed to help our legal and procurement teams quickly surface key obligations, deadlines, and unusual clauses from a repository of vendor contracts. The core stack is LlamaIndex with a mix of vector and keyword search, over a carefully chunked document set. What made the difference was the query engine routing—simple queries like "termination date" go straight to a metadata filter, while complex ones like "summarize the indemnification obligations" use a more nuanced summarization index.
The live demo I'm running has been surprisingly effective, but the real learning was in the tuning. Getting consistent answers meant spending significant time on prompt engineering for the response synthesizer and setting strict metadata extraction rules to avoid hallucination. It also highlighted the importance of a clean, pre-processed corpus; garbage in, garbage out is still very true.
I'm curious if others here have built similar document agents for compliance or legal workflows. What were your biggest challenges with accuracy and consistency? Did you find the built-in node parsers sufficient for complex PDFs, or did you have to build custom pre-processing pipelines?