RAG:give the modelan open book.
Retrieval-augmented generation builds a tiny, query-specific reading packet before the model answers. The model is not retrained. It gets evidence—assuming your retrieval system found the right pages.
- RETRIEVEcandidate evidence
- AUGMENTthe model context
- GENERATEa fresh answer
Adjustable desks: reimbursable up to $600 with manager approval.
Ergonomic furniture is eligible for the home-office benefit.
File under Home Office Furniture; attach the receipt.
Yes. Current policy covers an adjustable desk up to $600 with manager approval. Submit it as Home Office Furniture. [E-01] [E-03]
Grounded, not guaranteed. Check the clippings.The model takes the exam. Retrieval packs the open book.
RAG is a pipeline, not a special kind of database or a magic model mode. Its answer can only be grounded in evidence the pipeline actually retrieves.
Search an external corpus for passages that might answer this particular query.
Place the selected passages, their provenance, and clear instructions into the model’s context.
Ask the model to compose a fresh answer from that evidence—and admit when the packet is insufficient.
The context assembled for this request. New or private knowledge can be supplied at inference time.
The model’s weights. RAG is not training, and retrieved text does not become permanent model memory.
Truth. A stale source, missed passage, or poorly grounded generation can still produce a confident wrong answer.
THE BOUNDARYRetrieval decides what evidence enters the room. Generation decides what to say about it.
How picky should the retriever be?
Lower the threshold to admit more candidates. Raise it to demand a stronger relevance score. The scores and documents here are illustrative; the trade-off is real.
Retrieval quality is not one number. This lab isolates one knob so you can see its causal effect.
Adjustable desks may be reimbursed up to $600 with manager approval.
Ergonomic furniture is eligible under the home-office benefit.
Submit furniture under Home Office Furniture and attach the receipt.
Home-office furniture reimbursements are capped at $250.
Five ways to improve posture while working from home.
Hotel workspaces should include a desk and suitable lighting.
The lesson: retrieval needs enough recall to include the decisive passage, then enough filtering and ranking to keep stale or distracting passages out. “Stuff in more chunks” is not a quality strategy.
Stock the archive offline. Search it at request time.
Production RAG has a preparation loop and a serving loop. If you only diagram “question → vector DB → answer,” the important engineering disappears in the arrow labels.
Load the documents you are allowed to search. Preserve source IDs, versions, owners, and effective dates.
Split at boundaries that preserve a complete idea. The retriever returns chunks, not mystical whole-document understanding.
Represent passages for semantic similarity, often alongside keyword search. Store enough metadata to filter safely.
- 01Query
Start with the user’s information need. Rewrite or expand only when it improves retrieval.
- 02Retrieve
Fetch a broad candidate set using semantic, keyword, or hybrid search.
- 03Filter + rerank
Enforce permissions and freshness, then select the passages most useful for this question.
- 04Pack the prompt
Attach evidence, provenance, and rules for answering or abstaining.
- 05Generate + cite
Compose a fresh answer and map its citations back to the exact supplied passages.
A chunk is the unit of evidence.
Too large, and the relevant sentence swims in unrelated text. Too small, and the sentence loses its qualifier, table header, or section meaning. Overlap can help at boundaries; it cannot repair thoughtless splitting.
An embedding is a search coordinate, not compressed truth.
It makes semantic neighbors easier to find—even when wording differs. Exact names, IDs, dates, and rare terms often benefit from keyword or hybrid retrieval too.
Provenance must survive every handoff.
Carry stable chunk and source IDs from index to prompt to response. A decorative footnote generated from memory is not a citation system.
The glamorous part is mostly document hygiene.
Model choice matters. But many RAG failures are mundane: bad boundaries, stale indexes, missing permissions, weak queries, and no test set for the retriever.
First retrieve broadly enough to avoid missing evidence. Then filter, rerank, and budget context so the final packet stays useful. The best settings depend on your corpus and questions—measure them.
Write retrieval evals first
For a set of real questions, label which passages are sufficient. Measure whether those passages appear in the candidate set before grading eloquence.
Filter before the prompt
Tenant, role, geography, effective date, and document status belong in retrieval filters. Do not retrieve forbidden text and ask the model to politely ignore it.
Make freshness operational
Updates and deletions must propagate into the index. Track source version, indexed-at time, and failure state so “current” means something inspectable.
Let the model abstain
Tell it what counts as sufficient evidence and what to do when sources conflict or the answer is absent. Then test that behavior.
A citation is a trail, not a force field.
Citations make answers inspectable when they point to the passages actually used. They do not prove the source is current, the passage supports the claim, or the model read it faithfully.
The answer exists. Retrieval misses it.
The query uses different language, the chunk boundary hid the meaning, or the correct source was never indexed.
The archive remembers an obsolete world.
An old policy outranks its replacement, or a deleted page survives in the index.
A great semantic match is not permission.
Similarity search crosses tenant, team, region, or role boundaries and hands private text to the generator.
Too many clippings blur the answer.
A large top-k fills the context with near-matches, duplicate passages, and contradictory versions.
The model answers beyond the evidence.
The right passage arrives, but the generator adds unsupported details, ignores a qualifier, or attaches the wrong citation.
Retrieved text contains instructions.
Documents are untrusted input. A malicious passage can try to override rules or trigger actions if your system treats it as authority.
Retrieval expands the model’s input surface. Apply least privilege in the search layer, treat retrieved content as untrusted data, and keep consequential actions behind explicit authorization and validation.
Use RAG for missing knowledge—not every missing capability.
RAG is a strong fit when the answer lives in external text and the right text varies by question. It is an awkward substitute for a calculator, database query, workflow tool, or behavior change.
- Answers depend on a large private or domain-specific corpus.
- Knowledge changes often enough that retraining would be clumsy.
- Readers need to inspect sources or citations.
- Only a small slice of the corpus is relevant to each request.
Typical cases: internal handbooks, support knowledge, research libraries, technical documentation, contracts, and case archives.
Put the whole thing in context.
Query the database or call a tool.
Use prompting or consider fine-tuning.
Fix the knowledge supply first.
A minimum viable loop with places to inspect.
Keep the seams visible. You want to know whether a bad answer came from search, filtering, context assembly, generation, or citation mapping.
// 1. Search broadly, inside the user’s permissions
const candidates = await search({
query,
filter: {tenantId, roles, status: "current"},
topK: 20,
});
// 2. Select a small, useful evidence packet
const evidence = (await rerank(query, candidates))
.filter(isRelevant)
.slice(0, 5);
// 3. Generate from passages with stable IDs
const answer = await generate({
instructions: "Use only EVIDENCE. Cite [chunk_id]. Abstain if missing.",
input: formatQuestion(query, evidence),
});
return {
answer,
citations: verifyCitations(answer, evidence),
trace: {query, candidates, evidence},
};Retrieve evidence, not vibes. Keep provenance attached, permissions enforced, stale pages out, and “I don’t know” available.