Prompt Injection: What Internal Auditors Should Know

A user can give an AI assistant a perfectly legitimate task and still trigger an attacker’s instruction. The malicious content may be sitting in a supplier website, an email, a PDF, or a retrieval source that the application reads while doing its job. The employee never typed the hostile prompt, yet the model may process it as part of the instruction stream.

That is why prompt injection is not just a model-behavior problem. For auditors, it is better understood as a trust-boundary and application-control problem. The key question is not only whether the model can be manipulated, but what the system is allowed to do after manipulation occurs.

The risk begins when instructions and untrusted data share context

The NIST Generative AI Profile describes prompt injection as modifying input so that a generative AI system behaves in an unintended way. For audit purposes, two attack paths matter most.

Pattern	How the instruction enters	Business example	Primary audit focus
Direct prompt injection	A user or attacker submits the malicious content directly through chat, a form, or an API.	A user tries to override restrictions and obtain information outside the intended scope.	Access controls, system-prompt design, output validation, and application boundaries.
Indirect prompt injection	The malicious instruction is embedded in content that the application later retrieves or processes.	An assistant reviews an email, PDF, web page, or RAG record containing hidden instructions to alter the result or expose data.	Untrusted sources, retrieval, files, web access, permissions, and downstream actions.

Indirect injection is especially important because the attacker does not have to interact with the target model. They only need influence over content that a legitimate user or automated workflow will later consume. NIST’s newer adversarial machine-learning taxonomy sharpens this concept by treating indirect prompt injection as a resource-control attack: the adversary manipulates something the application trusts enough to ingest.

Prompt injection should also be separated from neighboring failure modes. A hallucination can produce a false answer without an attacker. A jailbreak is generally aimed at bypassing behavioral restrictions. Prompt injection exploits the application’s interpretation of instructions and content. They can overlap, but the control logic is different.

Business impact is driven by permissions, not by how clever the prompt looks

A successful injection against an isolated chatbot may result in a misleading answer. The same technique against an AI agent connected to email, document repositories, CRM records, source code, or transaction systems can create a confidentiality, integrity, or authorization event.

OWASP LLM01:2025 highlights outcomes such as sensitive-information disclosure, manipulated content, unauthorized function access, and actions the user did not intend. NIST similarly notes that indirect injection can affect availability, integrity, and privacy and can hijack agents or workflows when the model is able to invoke tools.

This changes the auditor’s risk assessment. The severity of prompt injection is not determined by the wording of the malicious instruction. It is determined by the system’s blast radius. What data can the model reach? What tools can it call? Which actions can it complete without an independent check?

That perspective also fits familiar internal-control concepts. COSO’s guidance on internal control over GenAI addresses prompt-based manipulation, changing configurations, cyber exposure, and other GenAI risks through the existing logic of control objectives, risk assessment, control activities, information, and monitoring. Prompt injection is therefore not a reason to abandon established control thinking. It is a new way an existing control objective can fail.

Strong control design assumes that some injections will get through

There is no single dependable control that eliminates prompt injection. NIST AI 100-2e2025 discusses training, detection, input processing, filtering of third-party instructions, and prompt designs that separate trusted and untrusted content, while also warning that current mitigations do not provide full protection. OWASP reaches a similar conclusion.

This matters because organizations can easily overinvest in the first barrier. Input filters and prompt rules are useful, but they should not be the whole design. A more resilient architecture normally layers controls such as:

explicit separation between trusted instructions and external content;
least-privilege access for the model and every tool or service it can invoke;
allowlisted actions and deterministic checks outside the LLM for critical operations;
independent human approval before payments, record changes, external communications, or other high-impact actions;
input and output filtering as defense in depth rather than the sole safeguard;
recurring adversarial testing across direct and indirect injection, RAG, files, web content, and other relevant channels;
logging of context, tool calls, approvals, outputs, exceptions, and incidents at a level that supports investigation and replay.

ISO/IEC 42001:2023 provides a management-system layer rather than a prompt-injection-specific technical recipe. Its requirements emphasize risk assessment and treatment, operational controls, change management, monitoring, documented information, and corrective action. For an organization applying the standard, a new tool integration, a material permission change, or a new external data source should therefore feed back into the risk and control cycle.

What Internal Audit should ask

A useful audit scope starts with the use case and its trust boundaries, not with a generic checklist of “LLM controls.” The Global Internal Audit Standards require internal auditors to understand the activity under review, its risks, governance, risk management, control processes, and relevant evaluation criteria. The IIA’s Artificial Intelligence Auditing Framework adds a practical lens for AI governance, risk, data, algorithms, and cybersecurity.

For a prompt-injection-sensitive application, auditors should be able to answer questions such as:

Which channels can place untrusted content into the model’s context: users, files, email, web pages, RAG sources, APIs, images, or other systems?
Is there a documented distinction between trusted instructions and externally supplied data?
What permissions does the model or agent inherit, and are least privilege and need-to-know principles actually enforced?
Can the system change records, send information, execute code, initiate transactions, or call tools without deterministic validation or independent approval?
How is externally retrieved content screened, isolated, or transformed before it can influence downstream behavior?
Do security tests cover realistic direct and indirect injection scenarios, rather than only manual prompts entered into a chat window?
Do logs preserve enough context to reconstruct the prompt, retrieved sources, tool calls, approvals, and resulting action?
Do material incidents and system changes trigger reassessment, control changes, and retesting?

Higher-risk systems may require cybersecurity or red-team specialists to design and execute deeper adversarial testing. Internal Audit does not need to perform every specialist test itself, but it remains responsible for framing the assurance question, evaluating the evidence, and integrating specialist work into the conclusion.

Evidence should prove the trust boundary, not just show that a policy exists

A policy stating that “malicious prompts are prohibited” does not demonstrate resistance to indirect injection. Audit evidence needs to connect design, operation, and outcome.

Evidence	What it helps establish	Typical audit procedure
Architecture and context/data-flow diagrams	Where instructions, external data, RAG, and tools meet.	Walk the flow and reconcile the diagram to deployed configuration.
Permission and tool-credential matrix	Whether the model has unnecessary authority.	Sample identities, scopes, tokens, and permitted actions.
Adversarial-test results	Coverage and effectiveness against realistic direct and indirect scenarios.	Review cases, severity, remediation, retesting, and accepted exceptions.
Prompt, context, and tool-call logs	What actually happened during a run or incident.	Reperform samples and tie actions to approvals and source content.
Incident and change records	Whether the control environment learns and adapts.	Trace selected incidents or changes through reassessment, remediation, and closure.

Standard 14.1 of the Global Internal Audit Standards requires engagement information to be relevant, reliable, and sufficient. Applied here, a vendor presentation or a single lab test is rarely enough. Auditors should seek corroboration across deployed configuration, logs, permissions, testing records, and observed behavior.

The same principle applies to Internal Audit’s own use of AI. Standard 10.3 addresses technological resources and requires collaboration with information technology and information security when new technology is implemented; its implementation considerations also call attention to information-security and privacy risks arising from technology use. An audit copilot processing workpapers, external files, or retrieved evidence can be exposed to the same attack pattern that the audit function is assessing elsewhere.

The decisive test is what happens after the malicious instruction lands

Prompt injection will not be solved by telling a model to “ignore unauthorized instructions.” The more useful control question is: if a malicious instruction reaches the model, what can it damage before another layer stops it?

A mature design reduces that potential damage through explicit trust boundaries, least privilege, deterministic safeguards, human approval where consequences are material, monitoring, and adversarial testing. For Internal Audit, that makes prompt injection a recognizable assurance problem: understand the risk, test the control design, obtain evidence of operation, and determine whether failures are detected and contained before they become business events.