Safety
Prompt injection
Prompt injection is an attack where text the application inserts into a prompt — a document, a user upload, a tool result — contains instructions the model then follows.
Any text reaching a model is a candidate instruction, because instructions and data share one channel. Indirect injection, where the payload arrives inside a retrieved document rather than from the user, is the harder and more common case.
The consequences scale with what the model can do. For a summariser it is embarrassing; for an agent with tool access it is a security incident, because the injected instruction can drive real actions.
In practice
The realistic version is not a user typing "ignore previous instructions". It is an agent reading a support ticket, a web page or a PDF that someone else wrote, and that document containing instructions. The agent has credentials; the user who submitted the document does not need any. Delimiting untrusted content is cheap and it is the mitigation most often missing.
Common questions
How do I prevent prompt injection?
Delimit untrusted content and label it explicitly as data to be processed, never merged into instructions. Then enforce permissions in code rather than in the prompt, so nothing in the context can grant a capability.
Is prompt injection a real risk or theoretical?
Real, and worst for agents. A chatbot injection produces a bad answer for one user; an agent injection produces actions taken with the agent's credentials, triggered by a document the victim never sees.