AI Agent Safety Checklist — Before an Agent Gets Tools

Ticks are saved in this browser only — close the tab and come back to where you were. Nothing is sent anywhere and there is no account.

0 of 18 done · 11 critical outstanding

Permissions

0/5

The agent's credentials grant exactly the access its tools need, and nothing that is merely convenient.critical
An agent operates with whatever it was given, on inputs you do not control. A read-only key cannot be talked into a deletion.
The agent has its own identity, not a shared service account or a human's token.critical
Shared credentials make the audit log useless and revocation an outage for everyone.
Every irreversible action — delete, send, pay, publish, deploy — requires human confirmation or is unavailable.critical
Reversible mistakes are learning. Irreversible ones are incidents. The classification is worth making explicitly, tool by tool.
Tools that read data are scoped to the requesting user's records, enforced server-side.critical
If scoping lives in the prompt, the scope is a suggestion. A retrieval tool that can read any customer will eventually be asked to.
No tool executes arbitrary code, shell commands or SQL composed by the model.
That is not a tool, it is remote code execution with a natural language interface. If it is genuinely needed, it belongs in a sandbox with no network and no credentials.

Limits

0/5

The loop has a hard maximum iteration count, enforced outside the model.critical
Agents get stuck. The characteristic failure is a two-step cycle repeated until something else stops it, and the something else should be your counter rather than your bill.
Check this with the agent loop simulator
A total token budget per task exists, and exceeding it ends the task rather than truncating it.critical
Agent context grows with every step, so cost per iteration rises as the run continues. The last iteration of a long run can cost many times the first.
Check this with the agent loop simulator
There is a wall-clock timeout for the whole task, not only per request.
Twenty successful slow steps produce no error and no answer for a very long time.
Individual tools have their own call limits — an agent cannot send fifty emails in one task.
A global iteration cap still permits a great deal of damage if every iteration is the same destructive call.
The worst-case cost of a single task has been calculated and is acceptable.
Worst case is cost per iteration multiplied by max iterations, and it is the number that appears on the bill during an incident.
Check this with the agent loop simulator

Untrusted input

0/3

Tool results are treated as untrusted data, delimited, and never merged into instructions.critical
The moment an agent reads a web page, an email or a ticket, an attacker can write to its context. This is the main attack surface of any agent that touches the outside world.
Check this with the prompt injection scanner
Nothing in the context can grant the agent a capability — permissions come from configuration only.critical
If instructions in a document can change what the agent may do, the document is your access control system.
Tool arguments are validated against a schema before execution, including ranges and identifiers.critical
The model produces arguments; it does not guarantee them. A refund tool that accepts any amount will one day receive any amount.
Check this with the tool schema builder

Observability and control

0/5

Every tool call is logged with arguments, result and the task that triggered it.critical
When an agent does something unexpected, the only way to understand it is the trace. Reconstructing a non-deterministic loop without one is guesswork.
A running task can be stopped, and stopping it stops the tool calls.critical
Cancelling the request while the executor keeps going is not a kill switch.
There is a mode where tools are called and logged but not executed.
It is the only safe way to test prompt changes on an agent that can act, and the cheapest way to see what it would have done.
Unusual behaviour — iteration counts at the cap, repeated identical calls, spend spikes — raises an alert.
These are the signatures of a stuck or manipulated agent, and they are visible long before the invoice is.
Consequential actions are sampled and reviewed by a person on a schedule.
Automated checks find the failures you predicted. Reading a sample of what the agent actually did finds the ones you did not.

Questions

What is the single most important item here?

Irreversible actions requiring confirmation. Everything else limits how much a mistake costs; that one determines whether the mistake can be undone at all.

Is prompt injection actually a practical risk for agents?

It is the defining risk. A chatbot injection produces a bad answer to one user. An agent injection produces actions taken with the agent's credentials, and the user never sees the instruction that caused them.

How many iterations should I allow?

Enough for the longest legitimate task plus a small margin — usually five to fifteen. If real tasks need thirty, the task is probably several tasks that should be separated.