RAG for SOPs: Giving Staff Accurate Answers With Traceable Sources

Accuracy depends heavily on document quality and the question set, so no universal guarantee exists. Practitioner reports range widely, from roughly 70–75% on early prototypes to the mid-80s and 90s on tuned systems with cleaned sources. Treat any figure as the outcome of one specific configuration rather than a benchmark you can expect by default. The reliable path is to build an evaluation set from real employee questions and measure retrieval recall, answer correctness, groundedness and citation accuracy before launch.

The failure usually sits before the language model: the needed information is missing from the index, the right fragment never surfaces among the top results, or an outdated edition or irrelevant passage reaches the context. When a rule and its exception live in different fragments, the assistant can state the norm without the caveat. A model can smooth over some retrieval defects, but it cannot reliably reconstruct missing or outdated information, so fix document quality, versioning and hybrid retrieval before judging generation.

There is no universal correct size. Larger fragments carry more context but more noise; smaller fragments match precisely but can drop the conditions of a rule. Guidance commonly cites starting points from a few hundred tokens up to ranges like 500–800 tokens, often with modest overlap, but these should be validated on your own documents and real queries. The governing principle is that one semantic block should be self-contained enough to retrieve without losing its main context, with the rule and its exception kept together.

Three things are required. First, fragment metadata must include document title, section, clause, page and version. Second, the prompt instructs the model to answer only from the provided sources and to state which fragment supports each claim. Third, citations must be validated against the index, because a model can produce a coherent answer with a wrong or invented reference. When no passage confirms the answer, the model should say so and point to a named source rather than invent content.

Solve it on the content side and in the pipeline together. On content: remove duplicates, move old editions to an archive, explicitly record status (current, draft, superseded), and assign an owner and a review date. In the pipeline: filter documents by status before semantic matching, deduplicate at indexing, and reindex when the source is changed on the day an edit is approved. A technical filter cannot substitute for content work — it has nothing to filter if a document's status is recorded nowhere.

Hybrid retrieval combines vector (semantic) search with lexical keyword search, often with metadata filters and reranking. Semantic search finds content under rephrased questions, while lexical search handles exact product codes, form numbers, abbreviations and internal jargon. This matters for SOPs because an employee may ask 'how do I approve an invoice' when the procedure is named differently, or may quote a form code such as K-42 that semantic search cannot tie to the right section. Hybrid retrieval covers both cases.

Build an evaluation set of dozens to hundreds of real questions as employees phrase them, not idealized wording. Measure four dimensions: retrieval recall (did the right passage surface), answer correctness, groundedness or faithfulness (does the answer stay within retrieved context), and citation accuracy (does each reference support the claim). Complement machine metrics and judge models with sampled human review, since automated judges can be biased. In production, monitor unanswered-question rate, latency, citation quality and how often a correct answer still needed a human check.