Retrieval-augmented generation (RAG)

RAG reduces hallucination and keeps proprietary data out of model training, which is why it dominates enterprise AI architectures. The retrieval layer typically uses embeddings and a vector database.

The security consequence that gets missed is that RAG turns your document store into part of the prompt. Anything the retriever can reach can end up in front of the model, which makes the retrieval layer an access control boundary whether or not anyone designed it as one. If the index was built by crawling a shared drive, the permissions of that drive do not travel with the text. A user who could never open the compensation spreadsheet can often ask a question whose best-matching chunk came from it.

The fix is to enforce permissions at retrieval time rather than at ingestion time: filter candidate documents by the querying user’s entitlements before they reach the prompt, and rebuild the index when access changes. This is unglamorous identity work rather than AI work, and it is the single most valuable control on a RAG system.

The second consequence is that RAG widens the prompt injection surface. A retrieved document is untrusted input that the model reads as authoritative, so an attacker who can place text into any indexed source (a wiki page, a support ticket, an uploaded PDF) can plant instructions that execute when that chunk is retrieved. Grounding improves accuracy, but it does not make the content safe, and treating retrieved text as data rather than instruction is a design decision the architecture will not make for you.