UTC

Human-agent trust exploitation

People trust what an agent tells them and approve what it asks. Whoever steers the agent can use that trust to get a person to approve, pay or disclose something.

How it works
  1. 1Agents present results and requests in a confident, helpful voice that people tend to accept.
  2. 2Approval prompts become routine, and routine approvals stop being read.
  3. 3A steered agent can reach people who would never trust an unknown sender.
What to watch for
  • An agent asking for approvals, credentials or payments outside the task
  • Approval requests whose summary differs from the actual action
  • Users approving in bulk without opening the details
How to defend
  • Show the exact action in approval prompts, not the agent's summary of it.
  • Never let an agent ask for passwords or one-time codes.
  • Keep high-value approvals rare, and outside the agent's own interface.