Embeddings Without the Math: How Semantic Retrieval Changes the Audit Surface

An employee asks an internal AI assistant, “What did we conclude last year about Project Atlas?” The system answers in seconds and cites a confidential audit finding. The answer is relevant. The problem is that the user was never authorized to see it.

That scenario is why embeddings matter to auditors. They are not merely a search optimization technique. In many enterprise AI applications, embeddings sit inside the layer that determines which information is treated as relevant and which content is passed to a model. When that layer performs poorly, the user gets weak answers. When authorization fails, the same architecture can expose restricted information.

The core control idea is simple: a retrieval system must answer two separate questions: “Is this relevant?” and “Is this user allowed to see it?” Embeddings can help with the first. They must never replace the second.

What an embedding is, without the mathematics

An embedding is a numerical representation of content designed to preserve useful relationships between items. Instead of treating a sentence only as a sequence of words, an embedding model converts it into a representation that can be compared with representations of other sentences, documents, images, or other objects, depending on the model.

Think of a conceptual map. “Segregation of duties in accounts payable” should sit closer to “conflicting payment roles for suppliers” than to “annual leave policy.” The two audit-related phrases do not need to use identical words for the system to recognize that they are related.

That is what makes semantic search possible. Google AI’s embedding documentation describes embeddings as numerical representations used for semantic search, retrieval, classification, and clustering.

This does not mean the system understands meaning the way a person does. The representation reflects patterns learned by the model and can produce excellent, mediocre, or misleading matches. For auditors, that distinction matters: semantic similarity is not the same as business relevance, sufficient evidence, or authorization.

How embeddings change enterprise search

A typical retrieval-augmented generation, or RAG, architecture can be reduced to six steps:

The organization selects documents or records for the knowledge base.
Those sources are split into smaller chunks.
Each chunk is converted into an embedding and stored in a vector index or store with metadata.
The user's question is also converted into an embedding.
The system searches for the semantically closest chunks.
The retrieved content is passed to a generative model to produce an answer.

This can make knowledge retrieval far more useful than literal keyword matching. It also moves part of the risk into the retrieval pipeline. The NIST AI Risk Management Framework and its Generative AI Profile treat AI as a system whose risks should be managed across its lifecycle and call for risk reassessment when models are adapted through techniques such as retrieval-augmented generation.

For an auditor, the unit of analysis is therefore not just “the model.” The source corpus, chunking logic, embedding model, metadata, vector index, filters, retrieval rules, and update process all become part of the audit surface.

The control failure that is easiest to miss: relevant does not mean authorized

Imagine a knowledge base containing general policies, contracts, investigation files, and internal audit workpapers. Two chunks can be very close semantically even though their access rights are completely different.

If a restricted document is chunked and its authorization metadata is lost, a legitimate user query may retrieve content the user is not entitled to see. A similar problem arises when permissions on the source document change but the vector index continues operating for days or weeks with stale access metadata or retained derived content.

The OWASP RAG Security Cheat Sheet recommends carrying authorization metadata into the chunks, enforcing access controls at retrieval time, and propagating permission changes and deletions to derived indexes. OWASP also identifies vector and embedding weaknesses as a distinct risk area for LLM applications.

That leads to a useful design rule for auditors:

Semantic retrieval decides what looks useful; authorization decides what is allowed into the conversation.

The permission check should happen before restricted content reaches the language model. Telling the model “do not reveal confidential information” is not a substitute for technical access control.

Four risks auditors should keep separate

The first is corpus quality. A retrieval engine can work perfectly and return an obsolete policy with impressive precision. ISO/IEC 42001:2023 addresses the management of data used by AI systems, including themes such as provenance, quality, security, intended use, and lifecycle management. The official ISO/IEC 42001 page confirms the standard's risk-management and continual-improvement orientation.

The second is retrieval quality. Poor chunking can separate an exception from the main rule, remove necessary context, or make an irrelevant clause look like the strongest match. Auditors should look for evidence that retrieval is tested against realistic user questions, not only polished demonstration examples.

The third is authorization. Permissions need to follow content through ingestion, chunking, indexing, and retrieval. If repositories with different sensitivity levels are combined, the design should prevent semantic similarity from crossing boundaries that identity and access management were intended to preserve.

The fourth is lifecycle traceability. When a source document is deleted, expires, or is reclassified, what happens to its chunks, embeddings, caches, and other derived artifacts? Data governance does not end when the original file disappears.

What Internal Audit can request and test

An engagement does not have to become a data-science review. It can begin with evidence that connects architecture, permissions, and actual retrieval outcomes.

Area	Audit question	Useful evidence or test
Corpus and freshness	Which sources can be retrieved, and how are they kept current?	Source inventory, owners, update dates, retention rules; test superseded or expired documents.
Retrieval	Does the system return genuinely relevant content for representative scenarios?	Test-query set, expected results, retrieval metrics, error analysis, and edge cases.
Authorization	Are user permissions enforced before content is passed to the model?	Chunk-level metadata, filter logic, tests with authorized and unauthorized roles, retrieval logs.
Change and deletion	Do source changes propagate to derived data?	Re-indexing evidence, cascading deletion, permission synchronization, and monitoring for orphaned chunks.

For an internal audit function using AI itself, the professional requirement is even more direct. Standard 5.2 of the Global Internal Audit Standards requires internal auditors to protect the confidentiality, privacy, and ownership of information, comply with applicable information-security requirements, and manage the risk of inadvertent disclosure. If Internal Audit uses RAG over prior reports, workpapers, or sensitive evidence, those duties do not disappear because the content has been converted into embeddings.

The IIA's Artificial Intelligence Auditing Framework provides a complementary lens for assessing AI governance, management, risk, and control processes. It does not turn every embedding issue into a standalone professional requirement, but it helps place retrieval architecture inside a broader assurance view.

The right question is not “Do we have a vector database?”

An organization can have a technically sophisticated architecture and still govern information poorly. It can also use a relatively simple managed retrieval service and maintain strong controls. Maturity is not determined by vector dimensions or product choice.

For auditors, the more valuable question is: Can the organization demonstrate that retrieved content is relevant, current, authorized, and traceable for the intended use?

Embeddings allow a machine to find relationships that literal search would miss. That is their value. But in an enterprise environment, finding something is not the same as having the right to use it. That is where the real control problem begins.