Motivation
LLMs hallucinate and have no knowledge of your private documents. The fix is retrieval-augmented generation: fetch the most relevant passages first, then let the model answer using only that context. I wanted a chatbot that could reason across multiple documents at once and return precise, grounded answers.
Architecture
Documents are chunked, embedded and indexed in FAISS. At query time the system retrieves top-k passages and passes them as context to the Cohere LLM, which generates a grounded answer.
📄Documentsmulti-source
→
✂️RCTS Chunkingrecursive split
→
🔢HF Embeddingsvectorize
→
🧭FAISS Indextop-k search
→
🤖Cohere LLMgrounded answer
Retrieve-then-generate: context is fetched before the model ever answers.
What I built
- A FAISS-backed vector store for fast similarity search across all indexed documents.
- HuggingFace embeddings for high-quality semantic retrieval.
- A Cohere LLM integrated into the retrieval pipeline for answer generation.
- Recursive Character Text Splitting (RCTS) tuned for chunk size and overlap to maximize retrieval quality.
Results
95%Multi-doc answer accuracy
−35%Latency via chunking
Multi-docCross-document reasoning
Challenges
- Chunking trade-offs: chunks too large dilute relevance; too small lose context. Tuning RCTS size/overlap was the highest-impact lever — and cut latency 35%.
- Retrieval precision: picking embeddings and top-k that surfaced the right passages without flooding the prompt.
- Grounding: shaping prompts so the model answered from retrieved context instead of its own priors.
Learnings
- Retrieval quality caps answer quality — most RAG wins come from the retrieval half, not the model.
- Chunking strategy is a real hyperparameter worth systematic tuning.
- A thin, well-structured prompt with good context beats a clever prompt with poor context.