Need the AI to stop hallucinating and actually show its work. Tired of "according to experts" with zero citations. Want every factual claim to automatically include a source link or reference.
Talking about a RAG setup, probably with something like LangChain or LlamaIndex. The agent itself is likely a custom chain. The key is forcing the citation *every time*, not just when it feels like it.
Here's a minimal `chain.py` concept using LangChain's `RetrievalQAWithSourcesChain`. The trick is in the prompt and the forced `return_source_documents=True`.
```python
from langchain.chains import RetrievalQAWithSourcesChain
from langchain.chat_models import ChatOpenAI
from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import Chroma
# Load your indexed knowledge base
vectorstore = Chroma(persist_directory="./chroma_db", embedding_function=OpenAIEmbeddings())
# Build the chain with explicit instructions
qa_chain = RetrievalQAWithSourcesChain.from_chain_type(
llm=ChatOpenAI(model="gpt-4", temperature=0),
chain_type="stuff",
retriever=vectorstore.as_retriever(search_kwargs={"k": 4}),
reduce_k_below_max_tokens=True,
return_source_documents=True, # Non-negotiable
)
# Custom prompt to hammer home the requirement
system_prompt = "You MUST cite the source ID and a brief excerpt for ANY factual statement. Format: [Source: ] ."
# Integrate this into the chain's prompt template
```
The output parsing then appends something like:
```
Answer: The sky is blue due to Rayleigh scattering. [Source: doc_id_42][Source: doc_id_77]
```
Without the `return_source_documents` flag and a strict prompt, it'll get lazy. Also, your retriever needs to be tuned—too few docs and it makes stuff up, too many and it gets noisy.
Anyone got a working config for this, maybe with a custom output parser that *guarantees* a source for each sentence? Saw a hack using `ResponseSchema` in LangGraph but it got messy.