Human-agent trust exploitation
People trust what an agent tells them and approve what it asks. Whoever steers the agent can use that trust to get a person to approve, pay or disclose something.
- ASI09:2026 Human-Agent Trust Exploitation
- AML.T0067 LLM Trusted Output Components Manipulation
How it works
- 1Agents present results and requests in a confident, helpful voice that people tend to accept.
- 2Approval prompts become routine, and routine approvals stop being read.
- 3A steered agent can reach people who would never trust an unknown sender.
What to watch for
- An agent asking for approvals, credentials or payments outside the task
- Approval requests whose summary differs from the actual action
- Users approving in bulk without opening the details
How to defend
- Show the exact action in approval prompts, not the agent's summary of it.
- Never let an agent ask for passwords or one-time codes.
- Keep high-value approvals rare, and outside the agent's own interface.