Designing the moment an AI assistant asks permission

Designing the moment an AI assistant asks permission

A button labelled “Approve” is not much of a permission system if the user cannot tell what will happen after clicking it.

Will the assistant save a draft or send a message? Which account will it use? Does approval cover this exact text, every future message to the customer, or an entire workflow? What happens if the customer record changes while the approval sits unanswered?

I would design those questions into the interaction before polishing the assistant’s conversation. The important product surface is the transition from reading information to changing someone else’s world.

Anthropic’s report on unintended model actions describes cases where models worked around restrictions or submitted forms they should not have submitted. These were observations from evaluations and internal use, not a measurement of how frequently all deployed agents fail. The report nevertheless makes one interface assumption hard to defend: that an assistant will always infer the intended boundary from a loosely phrased task.

Here is a proposed product walkthrough. It is not a description of an existing vendor interface.

The scenario: “Sort out this renewal”

Imagine an account manager using an assistant inside a subscription-management product. They open a customer record and type, “Sort out this renewal; they’re unhappy with the increase.”

The assistant has permission to read the account and prepare proposals. It cannot change the subscription, issue credits or email customers without additional authorisation. The phrase “sort out” supplies no reliable authority for any of those actions.

The interface should enter needs clarification, not silently choose the most consequential interpretation. A useful response is:

I can review the renewal and prepare options. Do you want an internal recommendation or a customer email draft? I will not change the price or send anything from this request.

The account manager chooses an email draft. That narrows the assignment without turning it into permission to send.

State one: reading, with visible scope

The activity panel says “Reading account and renewal terms”. It identifies the customer, connected system and allowed sources. A details view shows that payment details and unrelated accounts are outside scope.

Reading is not harmless by definition. Sensitive data can be exposed through summaries, exports or logs. The application should enforce access at the service boundary, not merely tell the model to ignore records it is already able to retrieve.

If the current renewal terms cannot be read, show “Source unavailable” and the blocked step. Do not make the assistant hunt for an alternative login or copy terms from a different customer. Offer an authorised reconnection or a human-provided document, with the new source clearly identified.

This is a terminal state for the current attempt unless the missing prerequisite is resolved. A spinner that continues indefinitely teaches the user that persistence is progress.

State two: a proposal, not an action

The assistant returns an email proposal explaining the existing renewal and offering a discussion. The screen says Draft only — nothing sent.

Beside the draft, display its evidence and uncertainty: the renewal date, current price reference and any missing account information. Keep the proposed message separate from the assistant’s internal analysis. The customer must not receive the reasoning notes accidentally because a tool sends the entire conversation.

The user can edit or reject the draft. Neither action should grant send permission. If they add a discount, the system recognises that the proposal now includes a commercial commitment and routes it to the appropriate approval policy rather than treating the edit as ordinary copywriting.

I would avoid one large “Let the assistant handle it” control. The proposal should reveal the exact operation the next approval would permit.

State three: approval bound to an exact action

For the unchanged email, the approval card might say:

Action: send this email once
From: assigned account manager’s work mailbox
To: verified customer contact
Content: proposal revision 4
Attachments: none
Subscription changes: none
Approval expires: displayed policy-defined timestamp
Controls: Approve this send / Request changes / Deny

The backend stores the approving identity, permitted operation, target, content revision, expiry and relevant policy version. It checks that the approver has authority. The model cannot manufacture an approval by writing “the user agreed” into a tool argument.

Approval should be as narrow as the action. If a recipient, attachment or substantive message changes, return to review. If the renewal terms have changed materially since the proposal, invalidate the stale approval and explain why.

OpenAI’s dots safety guidance describes separate action checks, protected enforcement and human handoffs. I take the architectural lesson seriously: the component carrying out the work should not control the mechanism that decides whether it may proceed.

Denied and expired are different states

When the user chooses Deny, the interface should show “Send denied — no message sent”. The workflow cannot reinterpret that as a temporary delivery error and try another mailbox. It may retain the draft according to retention policy, but the denied action is closed.

If the user wants a different proposal, that is a new revision with a new review. The assistant should not keep asking the same question until someone clicks yes.

An expired approval means the authority was time-limited, not that the user changed their mind. Show “Approval expired — review the current draft to continue”. Re-read the relevant account state, prepare any necessary changes and request fresh approval. Never extend the expiry silently.

These distinctions help the person understand why work stopped. They also give engineering different transitions to test instead of one vague “blocked” flag.

State four: execution needs a receipt

After valid approval, a narrowly scoped executor submits the approved action with a stable operation identifier and provider idempotency where supported. The UI changes to Sending, then Sent and verified only after provider confirmation. That confirms submission or sending, not recipient delivery or reading unless separately evidenced.

The receipt links the approval, operation identifier, destination message identifier, recipient, content revision and completion time. A user should be able to inspect what happened without reading a model transcript.

If the request times out after submission, the state is Outcome unknown, not “failed, retrying”. Reconcile against the provider’s returned identifier or a safe destination lookup before retrying. Otherwise one lost response can become two emails.

A “Stop” control should cancel pending work and prevent new actions. If a send has already completed, it cannot honestly promise to undo it. The interface must say which operations were prevented, which were in flight and which completed. Offer a human recovery path rather than a misleading universal undo button.

Make escalation actionable

A blocked workflow should ask for the smallest decision that can resolve it. “I need help” is less useful than “The proposed discount exceeds your approval authority; route to the commercial owner or remove the discount.”

Keep policy denials distinct from missing information and technical failures. A human approval cannot make a prohibited action permissible. An authentication failure needs restored authorised access, not a broader instruction to be resourceful.

The audit view should record these transitions with controlled access and appropriate retention. It need not expose private reasoning. It needs the facts required to explain authority, action and result.

Test the awkward paths first

I would prototype this interaction using synthetic customer records and no live send capability. Test an ambiguous request, a changed recipient, a denied send, an expired approval, revoked access and a timeout after submission.

Ask reviewers what they believe has happened at each screen. If someone mistakes a proposal for a sent message, the design has failed before the model enters the equation.

Permission UX is not a decorative warning before an agent acts. It is the product’s account of what the agent may do, what a person actually authorised and what the system can prove happened. Make that account visible and enforceable from the first read to the final receipt.