RAG Architecture: How Real Retrieval Systems Are Built, From the Source Code

RAG Systems: full architecture breakdowns

• PageIndex Architecture — Python
  vectorless, reasoning-based RAG
• AnythingLLM Architecture — JavaScript
  Stop renting your intelligence. Own it with AnythingLLM. Everything you need for a powerful local-first agent experience
• GraphRAG Architecture — Python
  A modular graph-based Retrieval-Augmented Generation (RAG) system
• LightRAG Architecture — Python
  [EMNLP2025] LightRAG: Simple and Fast Retrieval-Augmented Generation
• RAGFlow Architecture — Go
  RAGFlow is a leading open-source Retrieval-Augmented Generation (RAG) engine that fuses cutting-edge RAG with Agent capabilities to create a superior context l…
• Langchain-Chatchat Architecture — Python
  Enterprise-grade LLM knowledge base and chatbot system

The RAG pipeline, stage by stage, in four real codebases

Defaults below were read from each project's source code, not its docs. All of them are configurable; the defaults are what most deployments actually run.

Decisions every RAG architecture has to make

Chunk size is a product decision

Small chunks retrieve precisely and lose context; large chunks keep context and dilute the match. Langchain-Chatchat settles on 750 characters with a splitter tuned for Chinese punctuation, while Open WebUI uses a generic 1,000. PageIndex's whole pitch is that any fixed boundary cuts through meaning, so it never chunks at all.

Vectors alone miss exact matches

Embeddings are good at paraphrase and bad at error codes, product names and IDs, which is exactly what people search for. Both vector-based systems here run BM25 keyword search alongside the vector index and merge the two ranked lists. Langchain-Chatchat does it by default on FAISS; Open WebUI makes it a switch.

The vector store is a plug-in, not the architecture

Langchain-Chatchat supports seven vector stores and Open WebUI thirteen, each behind a single interface. That is the tell: the database choice matters far less than chunking, hybrid retrieval and reranking, which is where both projects put their real logic.

Similarity is not relevance

Vector search returns text that looks like the question, which is not always text that answers it. PageIndex bets on structure plus reasoning instead: build a tree index once, then spend LLM calls at query time to navigate it. The trade is query-time cost and latency for retrieval you can explain, and it pays off on long, structured documents like reports and filings.

Graphs answer questions no single chunk can

Ask a vector RAG system for the main themes across a thousand documents and it fails, because no chunk contains that answer. GraphRAG pays at index time instead: an LLM call per chunk to extract entities and relationships, then community detection and an LLM-written report per community, so global questions can be answered from the reports. LightRAG keeps the entity and relationship extraction but skips communities entirely, and by default mixes graph lookups with plain vector search over chunks. That cuts indexing cost, and it is the trade-off to weigh before reaching for GraphRAG.

Reranking is the step teams skip

A cross-encoder reranker reads the query and each candidate together and is far more accurate than raw similarity scores, at the cost of extra compute per query. Open WebUI exposes cross-encoder and ColBERT reranking. Langchain-Chatchat ships a reranker module and leaves the call commented out, which tells you how often this step gets cut for latency.

LLM apps with a RAG pipeline built in

• open-webui Architecture — Docker, JavaScript, Node.js
  ChatGPT-like interface for local and deployed LLMs

Frequently asked questions

What is RAG architecture?

RAG (retrieval-augmented generation) architecture is the system around an LLM that finds relevant pieces of your own data and puts them in the model's context before it answers. It has two halves: an indexing pipeline that runs ahead of time (load, chunk, embed, store) and a query pipeline that runs per question (retrieve, optionally rerank, then generate with the retrieved context).

What are the components of a RAG pipeline?

Document loaders, a text splitter, an embedding model, an index (usually a vector database), a retriever, an optional reranker, and a prompt template that injects the retrieved context for the LLM. The comparison table above shows how Langchain-Chatchat, Open WebUI, GraphRAG and PageIndex implement each one.

What is hybrid search in RAG?

Hybrid search runs keyword search (usually BM25) and vector search on the same query and merges the two ranked lists, typically with reciprocal rank fusion. Keyword search catches exact terms like IDs and error codes that embeddings blur; vector search catches paraphrases that keywords miss. Langchain-Chatchat and Open WebUI both implement it with a 50/50 default weighting.

What is GraphRAG?

GraphRAG is Microsoft's graph-based RAG system. At index time an LLM extracts entities and relationships from every chunk, hierarchical Leiden clustering groups them into communities, and the LLM writes a summary report for each community. At query time, local search follows the graph around the entities in a question, and global search map-reduces over the community reports, which lets it answer questions about a whole dataset that vector search cannot. The cost is many LLM calls during indexing.

What is vectorless RAG?

Vectorless RAG retrieves without embeddings or a vector database. PageIndex builds a hierarchical tree index from a document's structure and has an LLM reason its way through that tree to the relevant sections. It trades query-time LLM calls for retrieval that follows the document's own organization and can cite exact sections.

Do I need a vector database for RAG?

No. For a small corpus an in-process index like FAISS is enough, which is Langchain-Chatchat's default, LightRAG stores vectors and its graph in local files out of the box, and PageIndex shows retrieval can work with no vectors at all. A dedicated vector database earns its place when you need scale, filtering, multi-tenancy or persistence across services.

What chunk size should I use for RAG?

There is no universal number, but real defaults are a useful starting point: Langchain-Chatchat uses 750 characters with 150 overlap and Open WebUI uses 1,000 with 100 overlap, and GraphRAG uses 1,200 tokens with 100 overlap because each chunk also feeds entity extraction. Shorter chunks suit precise lookups; longer ones suit questions that need surrounding context. Test on your own documents and questions before tuning further.

Can I get this analysis for my own codebase?

Yes. These pages are generated by Revibe's codebase analysis. Upload a repository or import it from GitHub and you get the same architecture diagrams, module maps and execution flows for your own project, including your RAG pipeline.