How to Audit a RAG System

A corporate assistant answers a policy question correctly and provides a citation. The demo looks convincing. That is not enough evidence to conclude that the RAG system is controlled.

In retrieval-augmented generation (RAG), the final response depends on a chain: which sources enter the knowledge base, how they are chunked and indexed, what each user is allowed to retrieve, which documents a query actually retrieves, what context reaches the model, and whether the generated answer remains faithful to authorized and current evidence.

The audit thesis is therefore straightforward: a RAG system is not controlled because it cites sources. It is controlled when the full evidence chain can demonstrate that retrieval is relevant, authorized, current, traceable, and supported by predictable behavior when sufficient evidence is unavailable.

RAG moves the control surface

A general-purpose language model generates from its parameters and the context it receives. RAG adds a retrieval step that searches an external knowledge base and supplies selected content to the model before generation.

That can improve grounding, but it also introduces failure modes that do not sit inside the model itself. A superseded document may outrank the current version. A weak access filter may expose a restricted fragment. Poor chunking may strip away the context needed to interpret a policy. A citation may point to a real source that does not support the claim. And malicious instructions embedded in retrieved content can attempt to redirect the system through indirect prompt injection.

The audit object is therefore broader than the LLM. At minimum, it includes the source corpus, ingestion and indexing pipeline, metadata and permissions, embeddings or search mechanism, filters, ranking or reranking, context assembly, system instructions, generation, citation handling, monitoring, and change management.

Start with the evidence chain, not the model

The Global Internal Audit Standards provide a useful foundation for test design. Standard 14.1 requires internal auditors to gather information that is relevant, reliable, and sufficient and to evaluate whether it provides a reasonable basis for findings and conclusions. Its implementation considerations also point to currency and corroboration when assessing reliability. Standard 14.6 requires engagement documentation sufficient for an informed and competent person to repeat the work and obtain the same results.

For RAG, that translates into traceability between a query, the retrieved passages, and the answer. A screenshot of a successful chat is rarely sufficient evidence.

ISO/IEC 42001:2023 adds the management-system layer: risk assessment and treatment, controls, documented information, lifecycle management, data, suppliers, monitoring, and change. It does not prescribe a RAG-specific checklist. Its reference controls are applied through the organization’s risk-treatment process and statement of applicability; not every Annex A control automatically applies to every system. Audit criteria should therefore reflect the organization’s risks, external requirements, and controls determined as necessary.

NIST provides more GenAI-specific depth. NIST AI 600-1 includes actions concerning data provenance, source and citation review, grounding of RAG data, ongoing monitoring, and prompt-injection risk. As of September 2026, NIST states that AI RMF 1.0 is being revised, so auditors should record the exact framework version used as criteria.

Test seven control surfaces

Control surface	What can fail	Useful evidence	Audit procedure
Sources and corpus	Unapproved, incomplete, duplicated, or low-quality sources enter the knowledge base	Source inventory, owners, provenance, onboarding criteria	Sample sources and trace approval, origin, integrity, and coverage
Retrieval	Relevant content is missed or irrelevant content dominates	Chunking/search/ranking configuration, reference test set, retrieval logs	Run known-answer queries and compare expected vs. retrieved documents
Permissions	Users retrieve content they are not entitled to see	Access matrix, groups, metadata filters, identity logs	Repeat equivalent queries under different identities and verify restricted content never enters context
Citations and grounding	A citation exists but does not support the claim	Retrieved text, answer, references, traces	Check whether each material claim is actually supported by its cited source
Prompt injection	Instructions inside a source override intended behavior	Ingestion controls, separation of instructions from content, security tests	Use controlled adversarial cases and verify retrieved content cannot redefine instructions or trigger unauthorized disclosure
Freshness	Indexes retain old versions or fail to reflect deletions	Source dates, version history, synchronization targets, indexing logs	Update, replace, and retire test documents and measure propagation into retrieval
Output validation	The system answers confidently without adequate evidence	Abstention rules, evaluations, feedback, incidents, monitoring	Test answerable, unanswerable, ambiguous, and conflicting questions; assess accuracy, abstention, and escalation

This is not a universal checklist. A RAG application supporting regulatory decisions deserves different depth from a low-impact internal assistant. Scope should reflect intended use, users, consequences of error, source sensitivity, and third-party dependencies.

Build a test pack around risk, not demos

The strongest audit testing usually begins with a risk-based reference set, not improvised questions during a walkthrough. The set should represent the decisions and failure modes that matter for the use case.

Include questions with known answers, questions for which the corpus contains no answer, conflicting documents, current and superseded versions, content restricted to selected roles, and cases where a source contains untrusted instructions. For each scenario, define in advance which evidence should be retrieved and what acceptable behavior looks like: answer, abstain, ask for clarification, or escalate.

Preserve the trace for each execution. Record the corpus or index version, user identity or role, query, retrieved passages, ranking information where available, relevant configuration version, response, citations, and evaluation result. If the system is inherently variable, perfect reproducibility may be unrealistic; audit documentation should still make clear what was tested, under which configuration, and against which criteria.

Not every procedure requires a specialist. An AI-literate internal auditor can assess corpus ownership, access design, change controls, logs, monitoring, and source governance. More technically capable auditors can structure retrieval, citation, and adversarial-behavior tests. Penetration testing, exploit development, and deep security architecture assessments will normally require specialist support.

Use metrics as evidence, not as magic thresholds

RAG exposes measurable components, but no single score proves that the system is controlled. Useful measures can include whether the expected document appears among the top retrieved results, citation validity, the proportion of material claims supported by evidence, change-propagation time, abstention behavior when evidence is missing, and containment of prompt-injection scenarios.

Permissions require a different mindset. A strong average performance rate cannot offset one disclosure of sensitive content. The audit team should define which failures are tolerable, which are critical, and how thresholds relate to risk and intended use.

It is also important to separate retrieval quality from answer quality. A correct answer produced from the wrong documents may be accidental. Correct retrieval followed by an unsupported conclusion is a different failure. Measuring both layers prevents a polished final response from hiding a broken control.

Conclude on trustworthiness, not demonstration quality

A useful RAG finding connects criteria, condition, cause, risk, and action. “The AI may hallucinate” is too generic. “The index retains superseded policies for several days, allowing the assistant to cite outdated expense rules” identifies a testable condition and a concrete exposure.

The audit conclusion should address whether the system can repeatedly demonstrate that it uses governed sources, enforces permissions before sensitive content reaches the model, retrieves relevant evidence, keeps knowledge current, withstands reasonably foreseeable manipulation, cites faithfully, and validates outputs in proportion to the use case’s impact.

RAG can reduce some risks of an isolated LLM, but it does not remove the need for professional skepticism. The strongest evidence is not that the chatbot appears knowledgeable. It is that the organization can reconstruct and control why it answered the way it did.