Retrieval Augmented Generation (RAG) in LangChain

What RAG is and why it is used, the pipeline architecture and steps, popular vector stores, a full PDF example and advanced techniques.

13LangChain with Vector Databases 14Document Loaders and Text Splitters 16Retrieval-Augmented Generation (RAG) in LangChain

12.1

What is RAG?

RAG (Retrieval Augmented Generation) combines retrieval (search) and generation (LLM).
It fetches relevant information from external data sources (documents, databases, etc.) and provides it to the LLM as context.
This helps in getting accurate, up-to-date and domain-specific answers.
Useful for chatbots, document Q&A and internal knowledge systems.

12.2

Why Use RAG?

Provides accurate and factual answers.
Uses your private data (PDFs, docs, databases).
Reduces hallucinations in LLM responses.
Keeps information up-to-date.
Works with large domain-specific knowledge.
Cost effective (you don't need to fine-tune the model).
Useful for enterprise applications and chat with internal documents.

12.3

RAG Architecture in LangChain

DocumentsPDF, TXT, DB, Web, etc.

Document LoaderLoad data

Text SplitterSplit into small chunks

EmbeddingsConvert to vectors

Vector StoreStore vectors

RetrieverSearch similar chunks

LLMGenerate answer using context

Final ResponseTo user

The user query goes into the retriever, and the relevant documents it finds are passed to the LLM as context.

12.4

Steps in RAG with LangChain

1Load Documents - load data from PDFs, text files, web pages, databases, etc.
2Split Documents - split into smaller chunks.
3Create Embeddings - convert chunks into vector embeddings.
4Store in Vector DB - store embeddings in a vector database (FAISS, Chroma, Qdrant, etc.).
5Retrieve Relevant Chunks - search similar chunks for the user query.
6Generate Answer - send retrieved context to the LLM and generate the final response.

12.5

Popular Vector Stores for RAG

Vector Store	Description	Use Case
FAISS	In-memory vector store by Meta	Local development, small to medium data
Chroma	Persistent vector store, easy to use	Simple and open-source solution
Qdrant	High performance vector database	Production-ready, scalable applications
Pinecone	Managed vector database (cloud)	Scalable, enterprise use
Weaviate	Open-source vector DB with additional features	Hybrid search, production use

12.6

Example: Simple RAG Pipeline (PDF)

shell

# Install: pip install langchain langchain-community faiss-cpu pypdf

python

from langchain_community.document_loaders import PyPDFLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_community.embeddings import HuggingFaceEmbeddings
from langchain_community.vectorstores import FAISS
from langchain.chains import RetrievalQA
from langchain_openai import ChatOpenAI

# 1. Load PDF
loader = PyPDFLoader("company_policy.pdf")
docs = loader.load()

# 2. Split into chunks
splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000, chunk_overlap=200)
chunks = splitter.split_documents(docs)

# 3. Create embeddings
embeddings = HuggingFaceEmbeddings(
    model_name="all-MiniLM-L6-v2")

# 4. Store in Vector DB (FAISS)
vectorstore = FAISS.from_documents(chunks, embeddings)

# 5. Create retriever
retriever = vectorstore.as_retriever(search_kwargs={"k": 3})

# 6. Create RAG chain
llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0.7)
qa_chain = RetrievalQA.from_chain_type(
    llm=llm,
    retriever=retriever,
    return_source_documents=True
)

# 7. Ask a question
query = "What is the leave policy?"
result = qa_chain.invoke({"query": query})
print(result["result"])
print(result["source_documents"])

12.7

Advanced RAG Techniques

Use metadata filtering for better retrieval.
Hybrid search (semantic + keyword search).
Use re-ranking models to improve result quality.
Chunk optimization (size, overlap).
Use context compression to fit large data.
Combine multiple data sources (PDF + DB + API).
Use conversation memory for multi-turn RAG.
Evaluate RAG with metrics (relevance, accuracy).

12.8

Key Takeaways

RAG combines retrieval and generation for better results.
LangChain provides easy tools to build RAG pipelines.
Choose the right vector store based on your use case.
Proper chunking and embeddings are important.
Use metadata and filtering for accurate retrieval.
RAG is widely used in chatbots, enterprise search and internal tools.
It helps in building reliable and domain-specific AI applications.