Prompt Injection Risks: Practical Fixes for AI Apps

Instructions, retrieved documents, emails, web pages, and tool results may arrive in the same context stream, where the model interprets all of them as language.

That is the central difficulty behind prompt injection defense in custom AI apps. The problem is not simply that a model may follow a malicious sentence. The deeper issue is that the application has delegated part of its control logic to a probabilistic system that does not inherently separate trusted commands from untrusted data.

Prompt injection remains LLM01 in the 2025 edition of the OWASP Top 10 for Large Language Model Applications. Its continued position at the top is not a sign that the industry has failed to invent a sufficiently clever prompt. It reflects an architectural constraint: LLMs process instructions and data through a shared channel, without the hard boundaries that conventional software uses to enforce authority.

The structural vulnerability: language is carrying both data and control

Traditional software usually has explicit distinctions between code, configuration, user input, and stored data. A database query may still be vulnerable to injection, but the application can apply parsing rules, escaping, type validation, permissions, and transaction boundaries. These mechanisms are imperfect, yet they give the system a formal structure in which data does not automatically become executable logic.

An LLM does not provide that separation by default. A system message, a user request, a retrieved policy document, and a paragraph embedded in an uploaded PDF are all represented as tokens in a context. The model estimates what sequence should come next based on that combined material. It can be instructed to treat one source as authoritative and another as untrusted, but the distinction is expressed through language rather than guaranteed by the execution environment.

That difference matters most when a custom AI application does more than generate text. A chatbot that summarizes a document may expose confidential context or produce a distorted answer. An AI agent connected to email, a CRM, a payment workflow, or an internal database can create a much larger failure surface. The model’s interpretation may now influence an external action.

A useful way to think about the risk is to separate three layers that are frequently collapsed into one:

• The model layer predicts and generates language.
• The application layer decides how requests, context, and outputs are routed.
• The authority layer determines what the system is actually allowed to read, change, or execute.

Prompt injection is dangerous when a model’s interpretation is allowed to bypass the authority layer. A malicious instruction in a web page should not acquire permission merely because the agent retrieved that page. A sentence in an email should not gain the ability to send another email. A user request to summarize a document should not implicitly authorize the application to expose its system prompt or call an administrative tool.

Prompt injection is not primarily a bad-word problem. It is an authority problem disguised as ordinary language.

This is why prompt engineering alone is an insufficient defense. A carefully written system prompt can establish useful behavioral expectations, but it does not create a security boundary. The model may follow the instruction in one context and deprioritize it in another. It may also infer that content which appears to be an instruction deserves attention, even when the application intended that content to remain passive data.

For no-code and visual application builders, the issue can be easy to underestimate. A workflow may look simple on a canvas: receive a message, retrieve knowledge, ask an LLM for a decision, then invoke an action. The visual clarity of the workflow does not imply that authority is equally clear inside the model context. A single connector can silently extend the agent’s reach into documents, APIs, customer records, or internal operations.

Direct and indirect injection: two paths into the same context

The most familiar attack is direct prompt injection. A user deliberately enters text intended to alter the model’s behavior, reveal hidden instructions, bypass restrictions, or force an unauthorized response.

The wording varies. It may ask the model to ignore previous instructions, simulate a different role, disclose the system prompt, or treat the current task as a test of unrestricted behavior. Such attempts are often easy to demonstrate, but direct attacks are not necessarily the most operationally serious. They require the attacker to interact with the application and may be partially contained by authentication, rate limits, or a narrow user interface.

Indirect prompt injection is more subtle. Here, the malicious instruction is placed in content that the application later retrieves or processes. Possible sources include:

• A web page ingested by a research agent.
• An email routed through an AI support workflow.
• A document uploaded by a customer.
• A knowledge-base article edited by an untrusted contributor.
• A calendar event, CRM record, or shared note.
• A search result or third-party API response.

The end user may never see the injected text before the agent processes it. A retrieval-augmented generation system can bring the content into context because it appears semantically relevant to the task. The model then encounters instructions embedded in material that the application intended to treat as factual reference.

This creates a particularly awkward failure mode. The attacker does not need to compromise the model provider or alter the system prompt. They only need to influence a source that the application trusts enough to retrieve.

OpenAI has described real-world indirect prompt injection attacks as increasingly incorporating social engineering into the context rather than relying only on obvious override phrases. That direction is significant. A malicious document does not have to say “ignore your instructions” in a conspicuous way. It may frame an action as a security check, a necessary formatting step, an urgent customer request, or a routine administrative procedure. The model is not verifying authority in the way a human security engineer would. It is inferring the most plausible next action from the available context.

A practical threat map

The following distinction is useful when reviewing a custom AI app:

These categories can overlap. A direct user request may contain an attachment with an indirect payload. An indirect instruction may lead to a tool call. A generated output may then be consumed by another workflow that interprets it as structured data.

This is why a narrow filter for known phrases is a weak control. Attackers can paraphrase, split instructions across documents, use social engineering, or exploit assumptions built into the workflow. A defense should focus less on identifying one canonical attack sentence and more on restricting what the model can influence.

Why RAG and fine-tuning do not solve the problem

Retrieval-augmented generation is often introduced as a way to improve factual accuracy. Instead of relying entirely on the model’s internal parameters, the application retrieves relevant documents and places them in the prompt. This can reduce unsupported answers when retrieval quality is good.

It does not, by itself, establish a security boundary.

RAG changes where context comes from, but not the fundamental fact that retrieved text enters the model’s context. If a document contains instructions written to influence the agent, retrieval may increase the probability that those instructions are seen. A more accurate retrieval system can therefore deliver more relevant malicious content along with more relevant legitimate content.

Fine-tuning has a different purpose. It can shape a model’s style, domain behavior, or response patterns. It may make the model more consistent for a specialized task, but it does not guarantee that the model will correctly distinguish trusted instructions from untrusted text in every novel context. It also cannot substitute for runtime permission controls.

The distinction can be stated plainly:

• RAG improves access to context.
• Fine-tuning changes learned behavior.
• Neither one defines what the application is allowed to do.

A secure architecture still needs to decide which content is trusted, which tools are available, what arguments may be passed, and when a human must approve an action.

For a no-code AI workflow, this often means resisting the temptation to connect every available integration to a single general-purpose agent. A smaller agent with a narrow retrieval scope and read-only tools may be less impressive in a product demonstration, but it is easier to reason about. The system has fewer probabilistic decisions between an incoming instruction and an external side effect.

Defense-in-depth for custom AI applications

There is no single prompt injection patch that preserves full model flexibility while permanently eliminating the risk. Effective prompt injection defense in custom AI apps is layered. Each layer should assume that another layer may fail.

The goal is not to make the model infallible. The goal is to make a model mistake bounded, observable, and unable to become a high-impact security event.

1. Isolate system instructions from application data

System prompts should define the model’s role and constraints, but sensitive instructions should not be treated as a secret security mechanism. If disclosure of the system prompt would expose credentials, internal endpoints, or privileged procedures, those details are already in the wrong place.

Keep secrets outside the prompt. Store credentials in a proper secret-management system. Avoid placing internal access tokens, private URLs, or administrative logic in text that may be returned, logged, retrieved, or copied into another context.

Use explicit delimiters and source labels to clarify the difference between instructions and data. This does not create a formal boundary, but it improves interpretability and reduces avoidable ambiguity. Retrieved content should be labeled as untrusted reference material. The application should instruct the model not to execute instructions found inside that material.

That instruction is useful, but it remains a model-level control. It must be paired with application-level enforcement.

2. Validate and sanitize inputs without pretending validation is enough

Input validation can reject clearly malformed requests, suspicious payloads, or unsupported content types. It can also reduce the amount of irrelevant material sent to the model.

Guardrail models such as LLM Guard or Lakera Guard may be used to scan inputs and outputs for known risk patterns. These systems can contribute to defense-in-depth, especially when deployed as one signal among several. They should not be treated as universal detectors. There is no guarantee of zero false positives or zero false negatives, particularly when attacks are expressed indirectly or socially.

A practical input policy might include:

• Restricting accepted file types and maximum document sizes.
• Removing active content and unnecessary metadata from uploaded files.
• Separating user-provided text from system instructions in the request structure.
• Rejecting requests that attempt to access unsupported capabilities.
• Scanning retrieved content before it enters an agent context.
• Preserving provenance for every document, record, and web result included in a prompt.

The last point is frequently neglected. If the application cannot identify which source contributed a piece of context, it becomes difficult to investigate why the model made a particular inference.

3. Filter outputs before they reach users or systems

An output filter should not only look for offensive or unsafe text. In an AI application, output validation also concerns format, authority, and downstream interpretation.

If the model is expected to return structured data, validate it against a strict schema. Reject unexpected fields, invalid values, and tool names that are not part of the allowed operation set. Do not allow arbitrary model-generated text to become an executable query, an HTML fragment, a shell command, or an API request without a separate parser and policy layer.

This is particularly important in visual platforms where one node may pass the model’s response directly into another action. A workflow that treats free-form text as a command has effectively delegated control to the model.

A safer pattern is to make the model propose a limited action in a constrained format, then let deterministic application code or a platform policy engine decide whether that action is permissible. The model can classify, extract, or recommend. It should not be the only component that authorizes.

4. Restrict tools by capability, not convenience

Tool access is where prompt injection can move from a bad answer to a real operational incident. An agent with access to a mail-sending tool, customer database, document store, and billing API has a much larger blast radius than an agent that can only retrieve public product documentation.

The tool layer should enforce least privilege. Each tool should have:

• A narrowly defined purpose.
• A limited set of parameters.
• Explicit authentication and authorization.
• Rate and volume limits.
• Logging sufficient to reconstruct the decision.
• A clear distinction between read and write operations.

Avoid presenting the model with a general-purpose API or unrestricted database interface. If an agent needs customer information, expose a purpose-built function that returns only the fields required for that task. If it needs to update a ticket, constrain the allowed status transitions and text fields.

The application should also verify tool arguments after the model generates them. A tool call that looks syntactically correct may still be outside the user’s authority or inconsistent with the current workflow state.

5. Add human approval where consequences are difficult to reverse

Human-in-the-loop validation is not a sign that the AI system has failed. It is a rational response to uncertain inference and high-impact actions.

Approval is especially appropriate for:

• Sending external messages.
• Deleting or modifying records.
• Issuing refunds or financial transactions.
• Changing account permissions.
• Publishing content.
• Accessing sensitive personal or business information.
• Triggering workflows with legal, operational, or reputational consequences.

The approval interface should show the proposed action, the relevant source material, the user or process that initiated it, and the exact parameters that will be sent to the tool. A generic confirmation button is weaker than a review step that exposes the action’s provenance.

The safest agent is not the one that claims certainty. It is the one whose uncertainty is prevented from becoming authority.

Securing no-code agents and visual workflows

No-code platforms make AI integration accessible, but they can also conceal security decisions behind connectors and reusable components. A builder may see a compact workflow while the runtime performs retrieval, context assembly, model inference, parsing, and tool execution across several services.

The visual layer should therefore be reviewed as an architecture, not merely as a sequence of blocks.

Consider an AI support workflow that receives an email, retrieves customer history, asks an LLM to draft a response, and optionally updates a ticket. The obvious prompt injection risk is the email body. But other sources may also be untrusted: the customer’s previous ticket, an imported knowledge-base article, a signature block, or a third-party enrichment response.

A safer design would separate the workflow into distinct stages:

1. Ingestion: identify the sender, normalize the content, and preserve the original source.

2. Classification: determine whether the message belongs to an allowed support category.

3. Retrieval: fetch only the records relevant to that category and user authorization.

4. Generation: produce a draft without granting the model direct write access.

5. Validation: check the draft for policy, data exposure, and format requirements.

6. Approval or execution: send or update records only through a constrained action layer.

This is not a guarantee against injection. It reduces the number of decisions that depend on a single model interpretation.

For AI app builders and custom GPT-style solutions, it is also useful to distinguish between conversational convenience and operational authority. A natural-language interface can make a powerful system feel like a simple assistant. That simplicity is a user experience achievement, not a security property.

Permissions should be derived from the authenticated user and application state, not from what the model infers the user is entitled to do. If a user asks for another department’s private records, the authorization layer should deny the request before the model receives the data. The model should not be expected to enforce the organization’s permission model through prose.

A compact design review

When assessing a visual AI workflow, ask:

• Which inputs can contain attacker-controlled text?
• Which retrieved sources are editable by external or low-trust users?
• Can untrusted content influence a tool call?
• Does the model have access to write operations?
• Are tool arguments validated outside the model?
• What happens when the model returns malformed or ambiguous output?
• Which actions require human approval?
• Can logs show the exact context and decision path without exposing unnecessary sensitive data?
• Are connectors scoped to the minimum data and permissions required?

These questions are more useful than asking whether the system has a strong system prompt. The prompt matters, but it is only one control in a larger chain.

Automated security testing with Garak and PyRIT

Prompt injection defenses should be tested adversarially before deployment and after significant workflow changes. A benign demonstration is not evidence that an AI application is secure. It shows only that the application behaved safely under one selected input.

Tools such as garak and PyRIT can help probe models and AI systems for prompt leaks, jailbreak behavior, and unauthorized tool execution. Automated testing is valuable because it can repeat attack patterns at a scale that manual review cannot match. It can also expose regressions after changing a model, system prompt, retrieval source, parser, or connector.

Testing should cover more than direct prompts. A meaningful test set may include:

1. System prompt extraction attempts

Ask the model to disclose hidden instructions through direct requests, role changes, formatting tricks, or staged conversation.

2. Indirect document attacks

Place malicious instructions in files, web content, email bodies, and knowledge-base entries, then measure whether the agent treats them as data or authority.

3. Tool manipulation

Test whether untrusted content can cause an unauthorized tool call, alter parameters, bypass confirmation, or invoke a capability outside the user’s role.

4. Data exfiltration paths

Probe whether the model can reveal secrets, hidden context, private records, or information from unrelated users and sessions.

5. Output parser abuse

Send content designed to produce malformed JSON, extra fields, unexpected tool names, embedded markup, or values that violate business rules.

6. Multi-turn pressure

Test whether repeated requests, emotional framing, urgency, or gradual context changes cause the agent to abandon an earlier restriction.

The results should be treated as engineering evidence, not as a permanent security certificate. Model behavior can shift with a provider update. Retrieved data can change. A new integration can introduce a previously unavailable side effect. The system should be retested when its authority changes, not only when its interface changes.

Logging also matters. Capture the source and classification of retrieved content, the tools considered, the tool arguments proposed, the policy decision, and the final action. Redact sensitive information where necessary, but preserve enough context to understand how an injection moved through the system.

A useful evaluation distinguishes at least three outcomes:

• The model generated unsafe text but no external action occurred.
• The model proposed an unsafe action that the validation layer rejected.
• The system executed an unsafe or unauthorized action.

These are not equivalent failures. The second suggests that a control worked. The third indicates that the authority boundary was insufficient.

What a realistic security posture looks like

A mature AI application does not promise that prompt injection has been eliminated. It demonstrates that the system can tolerate adversarial input without granting untrusted content excessive control.

That posture usually includes:

• Clear separation between instructions, data, and permissions.
• Minimal tool access with explicit authorization.
• Input and output guardrails.
• Strict schemas for structured responses.
• Retrieval provenance and content trust levels.
• Human approval for high-impact actions.
• Automated adversarial testing.
• Monitoring for unusual tool calls, access patterns, and repeated injection attempts.
• A process for revising controls as models and workflows change.

The architecture should also acknowledge uncertainty. A probabilistic model may produce a plausible answer for the wrong reason. It may follow a malicious instruction that appears contextually relevant. It may refuse a legitimate request after overreacting to a guardrail. Security engineering cannot remove all such behavior at the model layer. It can, however, ensure that the model is not the final authority over sensitive operations.

For teams building custom AI applications without a large security department, this principle offers a practical direction. Start by reducing privileges, separating read from write actions, and making every external effect pass through a deterministic policy check. Then add scanning, testing, approval, and monitoring around that foundation. A sophisticated agent with unrestricted access is difficult to secure; a narrower agent with explicit boundaries is easier to evaluate and improve.

The industry will continue to refine model training, retrieval systems, guardrail models, and agent frameworks. Those improvements may reduce some classes of failure, but they will not remove the underlying tension between flexible language interpretation and strict authority management.

The open question is therefore not whether AI agents will become more capable. They almost certainly will. The more consequential question is whether their surrounding architectures can scale that capability without scaling the blast radius of a single mistaken inference or carefully planted instruction.