RAG:retrieve evidencebefore generation.
RAG:find relevant passagesbefore it answers.
Retrieval-augmented generation selects query-specific passages and adds them to model context before generation. It does not retrain the model, and its value depends on retrieval quality.RAG looks up useful passages and places them beside the question before the AI answers. It does not permanently teach the AI those pages, and it only helps when search finds the right evidence.
- RETRIEVELOOK UPcandidate evidencepossibly useful passages
- AUGMENTADD NOTESthe model contextbeside the question
- GENERATEANSWERa fresh answer
Adjustable desks: reimbursable up to $600 with manager approval.
Ergonomic furniture is eligible for the home-office benefit.
File under Home Office Furniture; attach the receipt.
Yes. Current policy covers an adjustable desk up to $600 with manager approval. Submit it as Home Office Furniture. [E-01] [E-03]
Check that each claim is supported by the selected passages.Retrieval selects the evidence. The model generates from it.Search finds the passages. The AI writes the answer.
RAG is a pipeline that combines retrieval, context assembly, and generation. An answer can be grounded only in evidence that the pipeline retrieves and supplies.RAG is three steps: find useful pages, place them beside the question, and write a new answer. The AI can only use evidence the search actually found.
Search an external corpus for passages that might answer this particular query.Search your trusted sources for the pages that could answer this question.
Place the selected passages, their provenance, and clear instructions into the model’s context.Put the useful passages beside the question, with their sources attached.
Ask the model to compose a fresh answer from that evidence—and admit when the packet is insufficient.Ask the AI to write a fresh answer from those notes—or say when the notes are not enough.
The context assembled for this request. New or private knowledge can be supplied at inference time.The reading packet for this answer. It can include current or private information.
The model’s weights. RAG is not training, and retrieved text does not become permanent model memory.The underlying AI. Reading a page for one answer does not teach it that page forever.
Truth. A stale source, missed passage, or poorly grounded generation can still produce a confident wrong answer.Truth. Old sources, missed pages, or a careless answer can still produce a confident mistake.
SEPARATE RESPONSIBILITIESRetrieval decides what evidence enters the room. Generation decides what to say about it.Search chooses which notes enter the room. The AI decides what to say about them.
Change the relevance threshold and inspect the evidence.
Lower the threshold to admit more candidates. Raise it to demand a stronger relevance score. The scores and documents here are illustrative; the trade-off is real.
Retrieval quality is not one number. This lab isolates one knob so you can see its causal effect.
Adjustable desks may be reimbursed up to $600 with manager approval.
Ergonomic furniture is eligible under the home-office benefit.
Submit furniture under Home Office Furniture and attach the receipt.
Home-office furniture reimbursements are capped at $250.
Five ways to improve posture while working from home.
Hotel workspaces should include a desk and suitable lighting.
Result: retrieval needs enough recall to include the decisive passage, then enough filtering and ranking to keep stale or distracting passages out. “Stuff in more chunks” is not a quality strategy.
Prepare the corpus before requests arrive, then retrieve at request time.
Production RAG has a preparation loop and a serving loop. If you only diagram “question → vector DB → answer,” the important engineering disappears in the arrow labels.
Load the documents you are allowed to search. Preserve source IDs, versions, owners, and effective dates.
Split at boundaries that preserve a complete idea. The retriever returns individual chunks rather than understanding a whole document at once.
Represent passages for semantic similarity, often alongside keyword search. Store enough metadata to filter safely.
- 01Query
Start with the user’s information need. Rewrite or expand only when it improves retrieval.
- 02Retrieve
Fetch a broad candidate set using semantic, keyword, or hybrid search.
- 03Filter + rerank
Enforce permissions and freshness, then select the passages most useful for this question.
- 04Pack the prompt
Attach evidence, provenance, and rules for answering or abstaining.
- 05Generate + cite
Compose a fresh answer and map its citations back to the exact supplied passages.
A chunk is the unit of evidence.
Too large, and the relevant sentence swims in unrelated text. Too small, and the sentence loses its qualifier, table header, or section meaning. Overlap can help at boundaries; it cannot repair thoughtless splitting.
An embedding is a search coordinate, not compressed truth.
It makes semantic neighbors easier to find—even when wording differs. Exact names, IDs, dates, and rare terms often benefit from keyword or hybrid retrieval too.
Provenance must survive every handoff.
Carry stable chunk and source IDs from index to prompt to response. A citation is reliable only when it maps back to a supplied passage with a stable identifier.
Corpus quality and retrieval design determine what the model can use.
Model choice matters. But many RAG failures are mundane: bad boundaries, stale indexes, missing permissions, weak queries, and no test set for the retriever.
First retrieve broadly enough to avoid missing evidence. Then filter, rerank, and budget context so the final packet stays useful. The best settings depend on your corpus and questions—measure them.
Write retrieval evals first
For a set of real questions, label which passages are sufficient. Measure whether those passages appear in the candidate set before grading eloquence.
Filter before the prompt
Tenant, role, geography, effective date, and document status belong in retrieval filters. Do not retrieve forbidden text and ask the model to politely ignore it.
Make freshness operational
Updates and deletions must propagate into the index. Track source version, indexed-at time, and failure state so “current” means something inspectable.
Let the model abstain
Tell it what counts as sufficient evidence and what to do when sources conflict or the answer is absent. Then test that behavior.
Citations make answers inspectable, but they do not prove correctness.
Citations make answers inspectable when they point to the passages actually used. They do not prove the source is current, the passage supports the claim, or the model read it faithfully.
The answer exists. Retrieval misses it.
The query uses different language, the chunk boundary hid the meaning, or the correct source was never indexed.
The archive remembers an obsolete world.
An old policy outranks its replacement, or a deleted page survives in the index.
A great semantic match is not permission.
Similarity search crosses tenant, team, region, or role boundaries and hands private text to the generator.
Too many clippings blur the answer.
A large top-k fills the context with near-matches, duplicate passages, and contradictory versions.
The model answers beyond the evidence.
The right passage arrives, but the generator adds unsupported details, ignores a qualifier, or attaches the wrong citation.
Retrieved text contains instructions.
Documents are untrusted input. A malicious passage can try to override rules or trigger actions if your system treats it as authority.
Retrieval expands the model’s input surface. Apply least privilege in the search layer, treat retrieved content as untrusted data, and keep consequential actions behind explicit authorization and validation.
Use RAG when answers depend on external text that varies by question.
RAG is a strong fit when the answer lives in external text and the right text varies by question. It is an awkward substitute for a calculator, database query, workflow tool, or behavior change.
- Answers depend on a large private or domain-specific corpus.
- Knowledge changes often enough that retraining would be clumsy.
- Readers need to inspect sources or citations.
- Only a small slice of the corpus is relevant to each request.
Typical cases: internal handbooks, support knowledge, research libraries, technical documentation, contracts, and case archives.
Put the whole thing in context.
Query the database or call a tool.
Use prompting or consider fine-tuning.
Fix the knowledge supply first.
Keep each pipeline stage observable.
Keep the seams visible. You want to know whether a bad answer came from search, filtering, context assembly, generation, or citation mapping.
// 1. Search broadly, inside the user’s permissions
const candidates = await search({
query,
filter: {tenantId, roles, status: "current"},
topK: 20,
});
// 2. Select a small, useful evidence packet
const evidence = (await rerank(query, candidates))
.filter(isRelevant)
.slice(0, 5);
// 3. Generate from passages with stable IDs
const answer = await generate({
instructions: "Use only EVIDENCE. Cite [chunk_id]. Abstain if missing.",
input: formatQuestion(query, evidence),
});
return {
answer,
citations: verifyCitations(answer, evidence),
trace: {query, candidates, evidence},
};Keep provenance attached, enforce permissions before generation, remove stale material, and allow the model to abstain when the supplied evidence is insufficient.