Retrieval-Augmented Generation (RAG) in LangChain

A consolidated RAG view: the architecture, workflow steps, vector database options, working examples with FAISS and persistent Chroma, key parameters and best practices.

12Retrieval Augmented Generation (RAG) in LangChain 13LangChain with Vector Databases 15Embeddings and Semantic Search

16.1

What is RAG?

RAG combines retrieval (searching relevant documents) with generation (LLM) to produce accurate and contextual answers.
It helps LLMs use your private data instead of just relying on pre-trained knowledge.
Reduces hallucinations and provides source-based answers.
Widely used in chatbots, Q&A systems, document search and enterprise knowledge systems.

16.2

Why Use RAG?

Provides accurate and up-to-date information from your own documents.
Reduces hallucination by grounding answers in real data.
Works with PDFs, websites, databases and more.
Supports source citations (where the information came from).
Essential for enterprise applications and domain-specific knowledge (policies, manuals, FAQs, etc.).

16.3

RAG Architecture with LangChain

Data SourcesPDF, TXT, web, database

Document Loader + Text SplitterLoad and split into chunks

EmbeddingsOpenAI / HuggingFace

Vector DatabaseFAISS / Chroma / Pinecone / Qdrant

RetrieverFind relevant chunks

LLMGenerate answer using context

Final ResponseWith source

16.4

RAG Workflow Steps

1Load documents from various sources (PDF, website, DB, etc.).
2Split documents into smaller chunks.
3Create embeddings and store them in a vector database.
4When a user asks a question, retrieve relevant chunks.
5Pass the retrieved context + query to the LLM.
6Generate the final answer with source information.

16.5

Popular Vector Databases for RAG

Vector DB	Type	Key Features	Use Case
FAISS	In-memory (by Meta)	Fast, simple, open-source	Local development, small to medium data
Chroma	Embedded	Easy to use, persistent storage	Simple projects, prototyping
Pinecone	Cloud (SaaS)	Managed, scalable, high performance	Production, large scale applications
Qdrant	Self-hosted / Cloud	High performance, filtering, payload	Production, enterprise use
Weaviate	Self-hosted / Cloud	Hybrid search, GraphQL, schema	Advanced search, enterprise applications

16.6

Example: Simple RAG using FAISS

python

from langchain.document_loaders import PyPDFLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.vectorstores import FAISS
from langchain.embeddings import OpenAIEmbeddings
from langchain.chains import RetrievalQA
from langchain.chat_models import ChatOpenAI

# 1. Load and split document
loader = PyPDFLoader("data/company_policy.pdf")
docs = loader.load()
splitter = RecursiveCharacterTextSplitter(chunk_size=1000,
    chunk_overlap=200)
chunks = splitter.split_documents(docs)

# 2. Create embeddings and store in FAISS
embeddings = OpenAIEmbeddings()
vectorstore = FAISS.from_documents(chunks, embeddings)

# 3. Create retriever and QA chain
retriever = vectorstore.as_retriever(search_kwargs={"k": 3})
qa_chain = RetrievalQA.from_chain_type(
    llm=ChatOpenAI(model="gpt-3.5-turbo", temperature=0.2),
    retriever=retriever
)

# 4. Ask a question
response = qa_chain.invoke({"query": "What is the leave policy?"})
print(response["result"])

16.7

Example: RAG with Chroma (Persistent)

python

from langchain.document_loaders import DirectoryLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import Chroma
from langchain.chains import RetrievalQA
from langchain.chat_models import ChatOpenAI

# 1. Load documents
loader = DirectoryLoader("data/")
docs = loader.load()
splitter = RecursiveCharacterTextSplitter(chunk_size=1000,
    chunk_overlap=200)
chunks = splitter.split_documents(docs)

# 2. Create embeddings and store in Chroma (persistent)
embeddings = OpenAIEmbeddings()
vectorstore = Chroma.from_documents(
    documents=chunks,
    embedding=embeddings,
    persist_directory="chroma_db"
)

# 3. Create QA chain
retriever = vectorstore.as_retriever(search_kwargs={"k": 3})
qa_chain = RetrievalQA.from_chain_type(llm=ChatOpenAI(),
    retriever=retriever)

# 4. Query
response = qa_chain.invoke({"query": "Summarize this document."})
print(response["result"])

16.8

Key Parameters in RAG

Parameter	Description	Example
chunk_size	Size of each chunk (in characters/tokens)	1000
chunk_overlap	Overlap between chunks	200
search_kwargs (k)	Number of relevant chunks to retrieve	3
temperature	Controls randomness in LLM response	0.0 - 0.5
model_name	LLM model to use	gpt-3.5-turbo

16.9

Best Practices and Key Takeaways

Use high-quality and clean documents.
Choose the right chunk size and overlap.
Use a reliable vector database based on your scale and use case.
Combine RAG with metadata filtering for better results.
Include source citations in the final response.
Fine-tune retriever and LLM parameters for accurate answers.
RAG is essential for building enterprise chatbots, knowledge bases and document Q&A systems.