System prompt and hidden context leakage
The model reveals its instructions, hidden context or secrets placed in them. Anything put in a prompt should be assumed readable by the user.
- LLM08:2026 Hidden Context Exposure
- LLM02:2026 Sensitive Information Disclosure
- AML.T0069.002 Discover LLM System Information: System Prompt
- AML.T0085.003 Data from AI Services: System Instructions
- EU AI Act 15(5) Confidentiality attacks
How it works
- 1System prompts and retrieved context sit in the same text the model can repeat back.
- 2Asked the right way, models reproduce parts of that text, including keys, internal URLs or business rules.
- 3In agents, hidden context also includes tool results the user was not meant to see.
Real cases on the map
- MediumOpen WebUI: Sharing models for others to use (read permission) also exposes model details (system prompt leakage)open-webui
- MediumOpen WebUI Exposes System Prompt to Regular User [Non-Admin]open-webui
- MediumPraisonAI: Unauthenticated Information Disclosure of Agent Instructions via /api/agents in AgentOSpraisonai
Published advisories whose own titles describe this technique.
What to watch for
- Responses containing system prompt fragments
- Credentials or internal identifiers in outputs
How to defend
- Never put secrets in prompts; give the agent scoped credentials through its tools.
- Scan outputs for prompt fragments and credentials.
- Assume the prompt is public and design for it.