When an AI agent must wait for a person

01From drafting to doing

A chat tool hands a person some text, and that person chooses what happens next. An agent is different because it calls tools itself. It can change a case status in a council’s housing system, book an inspection, reply to a supplier or queue a payment. As soon as software acts, the real question becomes: which person answers for this step, and at what moment did they agree to it?

Security practitioners already have a term for getting the balance wrong. OWASP’s list of risks for applications built on language models includes excessive agency (opens external site): an agent holding more functions, rights or freedom than its job requires. Asking a person to approve is one remedy. Put it on every step and staff learn to click through; put it nowhere and nobody ever decided what the agent did. The craft lies in choosing where it goes.

02List the actions before you look at the model

Write down each tool the agent may use. Then put four questions to every one of them. None of the questions is about model quality; all of them are about consequences.

1. 01How hard is it to reverse? One click, a correcting letter, a refund, or a hearing before an appeals body?
2. 02Whom does it touch? Only an internal draft, or a colleague, a customer, a resident, a supplier, public money?
3. 03Does it bind you? A decision about an individual, a payment, a contract, or a letter going out under your organisation’s name.
4. 04What are the volume and speed? A step repeated thousands of times each day calls for a different control from one that runs twice a week.

With those answers each action falls into one of four tiers. In most organisations a small number of actions carries nearly all the risk. That is useful to know, because it tells you where sign-off effort is worth spending.

These tiers are a design tool, not a legal category, so test them against your obligations. Under Article 22 of the GDPR, people have particular rights where a decision with legal or similarly significant effect rests only on automated processing (opens external site); in Ireland the Data Protection Commission supervises that. The EU AI Act adds, in Article 14, that high-risk systems must allow effective oversight by people (opens external site). Ireland gave the Act its national framework in the Regulation of Artificial Intelligence Act 2026 (opens external site), with the AI Office of Ireland as coordinating authority. Tier 4 is usually where these duties meet.

03Cases that always go to a person

Some conditions should push an action into a higher tier regardless of where it normally sits. Code them as explicit rules in the workflow so nothing depends on the agent spotting them.

• The input differs from anything the agent was tested with: an unfamiliar form, a letter in Irish where testing covered English only, blank or conflicting fields.
• Retrieved sources contradict each other, or the draft rests on something the agent cannot cite.
• The amount, the scope or the number of records affected passes a limit you have set.
• The file involves a sensitive matter: health data, a child, a complaint, a dispute or an appeal under way.
• The agent has retried already, or a downstream system has returned an error.

You will notice the model’s own confidence is not on the list. Language models can be fluent and wrong at once, which NIST’s generative AI risk profile calls confabulation (opens external site). A self-reported score may count as one signal, but only alongside independent checks such as validation rules and a comparison with the system of record.

04Sign-off that people can give properly

Endless “approve?” prompts produce reflex approvals. The AI Act addresses this head on: those overseeing a high-risk system must be able to keep automation bias in mind, in the words of Article 14:

In practice four choices make the difference.

Put the evidence beside the draft

Describe the proposed step in plain language, link each source so it opens at the relevant passage, name the rule or circular applied, and state precisely what changes in which system. If checking a draft means opening three applications, a busy officer will stop checking.

Refusing must be as quick as approving

Amending and declining are ordinary outcomes. Place both buttons next to approve, ask for a one-line reason on a refusal, and use those reasons in evaluation. When correcting a draft takes longer than writing it fresh, people will let weak drafts through.

Give sign-off to someone who can overrule

Sign-off belongs to a named role with the training, competence and authority to set the system aside. For high-risk systems, Article 26(2) (opens external site) of the AI Act asks deployers for precisely that. Name a deputy, define escalation, and track how long items wait. An unowned queue becomes the reason teams start routing around the gate.

Group only the genuinely routine

Clicking through forty almost identical items does not sharpen anyone’s attention. Tier 2 items can be reviewed in batches with random spot checks; tier 3 and tier 4 items are reviewed one at a time.

05The gate lives in the system, not in the prompt

A line in a prompt does not control access. Anything the agent reads, whether an email, an attachment or a web page, may carry instructions of its own; OWASP calls this prompt injection (opens external site). If a sentence in the instructions is all that stands between the agent and a step you cannot undo, expect that sentence to be bypassed eventually.

• Run the agent under its own identity with the narrowest rights each tool needs, and inside the requesting user’s rights where the platform allows.
• Enforce sign-off in the integration layer: the service that issues the letter rejects the call unless a valid approval exists for that exact step.
• Set limits on value, volume and rate on the tools themselves, outside the agent’s control.
• Keep read tools apart from write tools, so the agent can see more without being able to do more.

06Keep the records you will need to explain a decision

For each run, store the request, the sources retrieved, every tool call with its parameters, the draft, the approver, the outcome, any amendments and the times. Deployers of high-risk systems must hold the automatically generated logs for at least six months unless other law sets a different period (Article 26(6) (opens external site)). Keep them even where nothing obliges you to; they are how the system improves.

Read the records on a fixed schedule. Steps that are always approved without change may be ready for a lighter tier. Steps that are often amended reveal a weak spot in the agent, or belong one tier up. When refusals cluster around one source, one form or one team, the cause is usually a process issue that a better model will not solve.

07Begin strict, then relax step by step

Go live with the consequential steps in tiers 3 and 4. Before launch, agree what counts as a correct result, which errors you can tolerate and which you cannot. Measure against that, using the measurement functions of the NIST AI Risk Management Framework (opens external site) as a checklist if it helps.

Lower a step’s tier only on evidence: a run of approvals without material amendments, tests on the cases that failed, and agreement from the process owner. Treat that decision as a change to the system, record it, and keep the option to reverse it.

Agents will sometimes get things wrong. That is an argument for designing who signs off and where, not for keeping agents away from real work. Their value comes from acting, and well-placed gates are what let you hand them more over time.