π Complete Guide to RAG Technology 2025
Detailed explanation of Retrieval-Augmented Generation technology
What is RAG?
RAG (Retrieval Augmented Generation) is a technology that combines large language models (LLM) with external knowledge bases. It allows AI to answer questions based on the latest and accurate information by retrieving relevant documents before generating answers, effectively solving the knowledge deadline problem and "illusion" problem of LLM. RAG has become a core technology for enterprise-level AI applications.
πRAG Workflow
π οΈ RAG development framework
πΎ Vector database
π’Embedding model
| Model | Dimensions | Provider | Features | Open Source |
|---|---|---|---|---|
| text-embedding-3-large | 3072 | OpenAI | Best performance, supports dimensionality reduction | β |
| voyage-3 | 1024 | Voyage AI | Search optimization, MTEB takes the lead | β |
| bge-m3 | 1024 | BAAI (Chiyuan) | Multilingual, sparse + dense | β |
| e5-mistral-7b | 4096 | Microsoft | Based on LLM, excellent results | β |
| jina-embeddings-v3 | 1024 | Jina AI | 8K context, multitasking | β |
| mxbai-embed-large | 1024 | Mixedbread | Matryoshka, binary quantization | β |
π Core Concepts
π Chunking strategy
Split long documents into small pieces suitable for retrieval. Common strategies: fixed size, semantic chunking, recursive chunking. Block size is typically 200-1000 tokens, with overlap taken into account.
π Hybrid Search Hybrid Search
Combines vector similarity search (semantic) and keyword search (BM25), taking into account semantic understanding and accurate matching, and the effect is better than single retrieval.
π Reranking reordering
Use Cross-Encoder to reorder initial inspection results to improve recall quality. Commonly used models: Cohere Rerank, bge-reranker.
π Query Transformation
Rewrite user query to improve search performance. Technologies include: HyDE (Hypothesis Document), Query Expansion, Multi-Query.
π§ Contextual Retrieval
Anthropic proposes to add contextual descriptions to each chunk to significantly improve retrieval accuracy and reduce retrieval failures by 49%.
π Evaluation Evaluation
Key indicators: recall rate, precision rate, MRR, NDCG. Assessment framework: RAGAS, TruLens. An annotation dataset needs to be constructed.
π» Code Example
β‘ Advanced RAG Technology
π² Graph RAG
Combined with the knowledge graph, capture the relationships between entities. For complex queries that require inference, Microsoft has open sourced GraphRAG.
π Agentic RAG
Using AI Agent to dynamically plan retrieval strategies, you can iterative retrieval, multi-hop reasoning, and handle complex problems.
π Self-RAG
The model self-reflects on retrieval needs and result quality, decides whether retrieval is needed, and evaluates whether retrieval results are relevant.
π― CRAG (Corrective RAG)
Evaluate the relevance of retrieved documents, trigger web search supplements, and automatically correct low-quality search results.
π‘ RAG Best Practices
- Chunking strategy: Select the chunking method according to the document type, use syntax tree for code, and semantic chunking for articles.
- Embedding selection: For multi-language use bge-m3, for English use voyage-3 or OpenAI
- Mixed search: Vector search + BM25 usually improves the performance by 10-20% compared with single search
- RerankingοΌ Initial inspection of top-20, and top-5 after rearrangement, balancing recall and precision
- Prompt Project: Explicitly instruct the model to answer based only on the retrieved content, admitting that it does not know
- Evaluation first: First build the evaluation data set and use data-driven optimization to avoid blind parameter adjustment.
- Monitoring online: Record search logs, track low-quality answers, and continue iterative improvements
π Start building your RAG app
Choose the appropriate tool stack to quickly build an intelligent question and answer system