The problem it solves
A language model knows nothing about your business. Asked directly, it will produce a plausible, confident, invented answer. Retrieval-augmented generation changes the question: search your own material first, find the passages that bear on the question, and hand those to the model with an instruction to answer using only this.
How it works
- Preparation. Documents are split into passages and stored in a searchable index of meaning.
- A question arrives and is converted the same way.
- Retrieval. The index returns the closest passages — search by meaning, not just keywords.
- Generation. The model answers from the supplied passages, and should say so when the answer is not there.
- Citation. Because you know which passages were used, the answer can link to its sources.
RAG or fine-tuning?
Fine-tuning adjusts behaviour — tone, format, specialised style. It is poor at teaching facts, and those facts cannot be updated without training again. For business knowledge the answer is almost always RAG. If your policy changes on Monday, you want the system correct on Monday.
The four things that determine whether it works
- What you index. Feed it three superseded versions of a policy and it will cite the wrong one.
- How you split documents. Splitting along sections and headings beats cutting every thousand characters.
- How you retrieve. Combine meaning-based search with keyword search, then re-rank.
- What happens when nothing is found. The system must say it does not know.
RAG is strong on questions whose answer sits in one or two passages. It is weak on questions that require aggregating everything — those are database queries wearing a question mark. For the wider picture, see AI for business.