How Large Language Models Work: A Guide for Internal Auditors

An auditor uploads an 80-page policy to a generative AI tool and asks for a summary of the key controls. The response arrives in seconds. It is polished, specific, and convincing. Then the auditor checks the source and discovers that one of the controls does not exist.

It is tempting to say the model simply “read the document incorrectly.” That misses the more important point. A large language model, or LLM, is not a database that retrieves an exact answer and then adds fluent wording. It is a system that generates language from learned patterns and the information available in its current context.

That distinction explains both the power and the failure modes of LLMs. Internal auditors do not need the mathematics behind the models, but they do need four concepts: tokens, context, probabilistic generation, and validation. Together, they explain how the same technology can summarize hundreds of pages usefully and still produce a confident false statement.

An LLM generates language; it does not automatically query a source of truth

During training, a language model processes large volumes of text and adjusts its parameters to learn statistical relationships across sequences. Most modern LLMs build on the transformer architecture introduced by Google Research in Attention Is All You Need.

For audit purposes, runtime behavior is simpler to explain: the model receives a sequence, estimates plausible continuations, selects a next token, and repeats. The NIST AI 600-1 Generative AI Profile makes the risk implication explicit: LLMs predict the next token or word from statistical patterns, which can produce factually accurate output but can also produce inaccurate or internally inconsistent content.

This matters because fluency is not evidence, and specificity is not proof. A model may produce a correct statement because the information is in the prompt, because related patterns were learned during training, or because the generated continuation happened to be plausible and correct. The wording alone does not tell the auditor which one occurred.

Tokens are the units the model actually processes

An LLM does not necessarily receive words in the same form that a person sees them. Before processing, text is divided into tokens. A token may represent a whole word, part of a word, punctuation, spacing, or another text fragment, depending on the model and tokenizer.

This detail has practical consequences. Model input and output limits are usually measured in tokens, not pages. Long identifiers, account numbers, clause references, tables, or unusual terminology can be represented as multiple tokens rather than as one indivisible value.

The audit lesson is not to count tokens manually, but to recognize that the model transforms the material before using it. When the exact value of an invoice number, contractual clause, threshold, date, or regulatory citation matters, the generated output should be checked against the source or against a deterministic control.

The context window is working material, not total memory

The context window is the amount of tokenized information the model can process together for a particular interaction. In an enterprise application, that context may contain system instructions, the user's prompt, examples, conversation history, retrieved documents, and results returned by connected tools. Google Cloud defines a context window as the number of tokens a foundation model can process in a prompt.

A useful mental model is temporary working material, not perfect memory. The model carries learned patterns in its parameters, but specific current information for a task must arrive through context, retrieval, or a connected tool.

A larger context window does not guarantee that every included fact will be used correctly. A critical control requirement may be present in a long document set and still be overlooked or outweighed by other instructions or examples. That is why an audit of an enterprise LLM application should not stop at “Which model do you use?” A more revealing question is “What information reaches the model for this decision, and how is it selected?”

This is also the right way to think about retrieval-augmented generation, or RAG. RAG can place policies, contracts, standards, or other trusted documents into the model's context. It can improve grounding, but it does not convert the LLM into an infallible database. Output quality still depends on which documents were retrieved, whether they are current and authorized, how they were segmented, and how the model used them.

Probabilistic generation explains variability and hallucinations

At each generation step, the model has multiple possible continuations with different estimated probabilities. System settings influence how one continuation is selected. As a result, similar runs can produce different wording or different answers, even when the underlying task appears unchanged.

That flexibility is useful for drafting, summarizing, classifying, and exploring alternatives, but dangerous when users treat the response as an exact lookup from a controlled record.

NIST uses confabulation for outputs that confidently present erroneous or false content; “hallucination” is the common informal term. The important point is that confabulation is not merely an occasional bug found in poor systems. NIST connects it to the statistical nature of generative models. Better models, stronger prompts, retrieval, testing, and validation can reduce the risk, but critical processes should not be designed on the assumption that the risk will disappear.

LLMs are also context-sensitive. A change in instructions, document order, examples, or retrieved information can change the output. When the model is connected to tools, databases, or business applications, a generation error can move from incorrect text to an incorrect action.

What this changes for internal auditors

Understanding the mechanism leads to better control questions. An audit does not need to begin with model architecture. It can begin with the use case, the information entering the system, the validation design, and the consequence of error.

What happens in the LLM	Audit risk	What to examine
Text is tokenized and context is bounded	Relevant information may be omitted, truncated, or handled unreliably	Documented limits, long-document handling, and tests involving exact or material data
Output depends on available context	Missing, stale, or poorly retrieved sources can change the conclusion	Authorized sources, freshness, retrieval criteria, permissions, and traceability of context
Generation is probabilistic	Outputs can vary, confabulate, or reach plausible but wrong conclusions	Test sets, acceptance criteria, repeat testing, and independent validation
Models and configurations can change	Behavior may change even when the business process has not	Inventory, versioning, change management, pre-release testing, and post-release monitoring
People tend to trust fluent answers	Human review can become superficial approval	Review roles, training, escalation rules, and evidence of corroboration

For internal audit's own use of LLMs, the Global Internal Audit Standards provide an important boundary. Standard 4.3, Professional Skepticism, requires internal auditors to critically assess information reliability and seek additional evidence when information may be incomplete, inconsistent, false, or misleading. Standard 14.1, Gathering Information for Analyses and Evaluation, requires information supporting engagement work to be relevant, reliable, and sufficient.

Those responsibilities do not weaken because a sophisticated model produced the information. The IIA's 2026 guidance, Understanding Data Analytics for Internal Auditors, places LLMs within the broader analytics toolkit and emphasizes data quality, governance, and validation when analytics and AI are used in audit work.

Before relying on an answer, separate source, generation, and evidence

A simple five-step lens helps prevent a useful output from being mistaken for a supported conclusion:

Source: What approved, current information should support the task?
Context: Which portion of that information actually reached the model, and under what instructions?
Generation: What did the LLM summarize, infer, add, or reformulate?
Validation: Which material claims were checked against independent sources or deterministic controls?
Decision: Who remains accountable for the conclusion, and what happens when uncertainty remains?

ISO/IEC 22989:2022 provides common terminology for discussing AI concepts. The NIST AI Risk Management Framework and its Generative AI Profile move the conversation toward context of use, measurement, and risk management. For internal audit, the decisive step is translating those concepts into testable evidence.

An auditor does not need to inspect model parameters to ask meaningful questions. The auditor does need to understand what enters the system, how the model generates output, and where that process can fail.

The most useful mental model is simple: an LLM is a probabilistic generator conditioned on context, not an automatic authority. Once that is clear, the technology becomes less mysterious and its risks become far more auditable.