RAG Explained

RAG stands for Retrieval-Augmented Generation. The short version: instead of asking the AI to remember everything, you let it look things up first.

How it works

1. You have a collection of documents (PDFs, notes, a knowledge base).
2. When someone asks a question, the system finds the relevant chunks from those documents.
3. Those chunks get passed to the AI along with the question.
4. The AI answers using both its training AND the retrieved context.

When do you need it?

Use RAG when:

• Your data changes frequently (product docs, internal wikis)
• You need the AI to cite sources
• The topic is too niche to be in the AI's training data

Don't use RAG when a fine-tuned model or simple prompt works just as well.

A concrete example

Imagine you run a SaaS. Your help docs are 200 pages long. Instead of hoping the AI memorised your product, you:

1. Embed all your docs into a vector database
2. When a user asks "how do I export data?", retrieve the relevant doc sections
3. Pass those sections to the AI with the question

The AI gives a specific, accurate answer — not a generic one.

What to use

• Vector DBs: Pinecone, Supabase pgvector, Chroma (local)
• Embedding models: OpenAI text-embedding-3-small, Cohere embed
• Orchestration: LangChain, LlamaIndex, or build it yourself in 50 lines

RAG is one of those things that sounds complex until you actually build one. Start with a small document set and a local vector store.