Hi everyone! 👋 I've been lurking for a bit, trying to learn the ropes. I'm in the middle of evaluating different tools for a project at my company, and I just finished a small proof-of-concept that I'd love some feedback on.
I built a tool that lets our finance team query a set of quarterly PDF reports using natural language. The goal is to answer questions like "What was the total marketing spend in Q3?" without manually digging through dozens of pages.
Here's the basic architecture I went with:
* I used LlamaIndex to ingest and chunk the PDFs (mostly text-based financial statements).
* For the vector store, I went with Pinecone because we have a small trial credit there. I'm a bit unsure if this is the most cost-effective choice long-term.
* The query engine is pretty standard, using GPT-4 for the final answer synthesis.
My main question is about the chunking strategy. I used the default `SentenceSplitter`, but I'm worried it might cut tables or important figures in half. Has anyone here worked with similar financial documents? Did you find a better node parser or chunking settings that preserved context better?
Also, I'm curious about alternatives to Pinecone for a production setup. We're on a tight budget, so I'm weighing options like Chroma or even a simple FAISS index. Any experiences with those for a few thousand documents?
Great use case, and I've been down a similar road with sales reports! Your chunking concern is spot on. With financial tables, the default sentence splitter can butcher the data.
You might want to try a hybrid approach. For a past project, I used a dedicated PDF table extractor first (like Camelot or Tabula) to pull structured data before chunking the surrounding text. Then you can keep the tables as whole units in your index. LlamaIndex has some custom node parser examples for this.
On Pinecone, it's great for a trial. For long term cost, check out ChromaDB (open source, runs locally) or even Weaviate. They can handle a finance team's query volume unless you're talking millions of embeddings.
How are you handling the numbers from the PDFs? I found that OCR on scanned statements sometimes misreads decimals, which is a disaster for finance.