Coalition for Secure AI · Risk Map
The CoSAI Risk Map: AI security risks and the controls for them.
36 risks in 5 groups and 37 controls, each with the people responsible for it and its mappings to MITRE ATLAS, the NIST AI RMF and the OWASP LLM Top 10. The Risk Map maps to the 2025 OWASP list, so those ids carry their year.
Model Exfiltration
Theft of a model. Similar to stealing code, this threat has both intellectual property and security implications.
ResponsibleModel ProviderAI Model Serving
ControlsIsolated and Confidential ComputingModel and Data Inventory ManagementModel and Data Access ControlsSecure-by-Default ML ToolingModel and Data Integrity Management
Maps toAML.T0024.002AML.T0025AML.T0048.004Information Disclosure
Model Deployment Tampering
Unauthorized changes to model deployment components. Model Deployment Tampering can result in changes to model behavior.
ResponsibleAI Platform ProviderAI Model ServingApplication DeveloperAI System Users
ControlsIsolated and Confidential ComputingModel and Data Execution IntegrityOrchestrator and Route IntegritySecure-by-Default ML ToolingInter-Component Transport SecurityModel Registry and Discovery Integrity
Maps toAML.T0010.003AML.T0049AML.T0051LLM03:2025LLM01:2025TamperingElevation of Privilege
Model Reverse Engineering
Recreating a model by analyzing its inputs, outputs, and behaviors. A reverse engineer model can be used to create imitation products or adversarial attacks.
ResponsibleModel ProviderData Provider
ControlsApplication Access and Resource Management
Maps toAML.T0024.002AML.T0005AML.T0048.004Information Disclosure
Insecure Integrated Component
Software vulnerabilities that can be leveraged to compromise AI models. Insecure Integrated Component can lead to privacy and security concerns, as well as potential ethical and legal challenges.
ResponsibleAI Platform ProviderAI Model ServingAgentic Platform and Framework ProvidersApplication DeveloperAI System Users
ControlsAgent PermissionsModel and Data Execution IntegrityUser Policies and EducationInput Validation and SanitizationInter-Component Transport SecurityModel Registry and Discovery Integrity
Maps toAML.T0051AML.T0049LLM03:2025LLM06:2025TamperingElevation of Privilege
Accelerator and System Side-channels
Cross-tenant leakage via hardware side-channels in CPUs, GPUs, TPUs, memory systems, and interconnects. These vulnerabilities exploit timing, cache, speculative execution, and memory access patterns to compromise data confidentiality and infer sensitive information in shared computing environments.
ResponsibleAI Platform ProviderAI Model ServingApplication DeveloperAI System Users
ControlsSecure-by-Default ML ToolingModel and Data Access ControlsIsolated and Confidential Computing
Adapter/PEFT Injection
Trojaned adapters merged at runtime to bypass safety or exfiltrate data. Adapter/PEFT Injection can compromise model behavior and enable unauthorized access to sensitive information or system resources.
ResponsibleAI Platform ProviderAI Model ServingApplication DeveloperAI System Users
ControlsModel and Data Execution IntegrityModel and Data Integrity ManagementModel and Data Access ControlsSecure-by-Default ML Tooling
Tool Registry Tampering
Compromise of previously vetted tool registries, MCP server manifests, or tool metadata, causing agents to select and invoke unintended tools through altered descriptions, schemas, or shadowed tool names.
ResponsibleAgentic Platform and Framework ProvidersApplication DeveloperAI Platform ProviderAI Model Serving
ControlsOrchestrator and Route IntegrityAgent ObservabilityInter-Component Transport SecurityTool Registry and Discovery Integrity
Maps toAML.T0010.004LLM06:2025Tampering
Agent Delegation Chain Opacity
Multi-hop agent delegation chains become opaque when intermediate agents do not emit structured delegation events, making it impossible to determine which human, agent, or system authorized or performed an action — even when individual authentication steps succeed.
ResponsibleAgentic Platform and Framework ProvidersApplication DeveloperAI Platform ProviderAI System Users
ControlsAgent ObservabilityAgent Integrity ManagementAgent Inventory Management
Maps toRepudiation
Stale Agent Identity Binding
Agent identity credentials persist unchanged after the underlying model artifact is replaced, allowing a substituted model to operate with the original agent's full permissions because no cryptographic binding ties identity to a specific model version.
ResponsibleAgentic Platform and Framework ProvidersApplication DeveloperModel ProviderAI Platform ProviderAI System Users
ControlsModel and Data Integrity ManagementAgent Integrity ManagementAgent Credential IsolationComponent Identity Provenance
Maps toAML.T0012Spoofing
MCP Transport Hijacking
Transport-layer attacks on MCP connections — including man-in-the-middle, session hijacking, DNS rebinding, and replay — intercept or modify agent-to-tool communication because the MCP specification does not mandate transport encryption or mutual authentication.
ResponsibleAgentic Platform and Framework ProvidersApplication DeveloperAI Platform ProviderAI System Users
ControlsSecure-by-Default ML ToolingInter-Component Transport SecurityComponent Identity Provenance
Maps toTamperingInformation Disclosure
Shadow and Unknown Agents
Unregistered agents operate with inherited or valid credentials outside identity lifecycle controls, creating persistent blind spots in agentic systems.
ResponsibleAgentic Platform and Framework ProvidersApplication DeveloperAI Platform ProviderAI System Users
ControlsSecure-by-Default ML ToolingAgent ObservabilityAgent Inventory ManagementAgent Integrity ManagementAgent Credential Isolation
Maps toAML.T0012SpoofingRepudiation
Agentic Delegation Confused Deputy
A high-privilege agent acts on behalf of a less-privileged caller without validating the caller's authorization at the delegation boundary, allowing the caller to escalate privileges through the deputy's own permissions.
ResponsibleAgentic Platform and Framework ProvidersApplication DeveloperAI Platform ProviderAI System Users
ControlsSecure-by-Default ML ToolingAgent PermissionsAgent Observability
Maps toAML.T0053LLM06:2025Elevation of PrivilegeSpoofing
Cross-Tenant Credential Propagation
Shared or insufficiently tenant-scoped agent credentials allow a compromise in one tenant to propagate access across tenant boundaries through the identity layer.
ResponsibleAgentic Platform and Framework ProvidersApplication DeveloperAI Platform ProviderAI System Users
ControlsSecure-by-Default ML ToolingAgent PermissionsIsolated and Confidential ComputingAgent Credential Isolation
Maps toAML.T0012AML.T0055LLM06:2025Elevation of PrivilegeInformation Disclosure
Zombie / Shadow MCP Servers
Decommissioned or unregistered MCP servers remain accessible and respond to agent tool calls, allowing attackers to intercept or redirect legitimate requests.
ResponsibleAgentic Platform and Framework ProvidersApplication DeveloperAI Platform ProviderAI System Users
ControlsAgent ObservabilityAgent Inventory ManagementInter-Component Transport SecurityComponent Identity Provenance
Maps toAML.T0010.001LLM03:2025SpoofingTamperingInformation Disclosure
Excessive Data Handling During Inference
Unauthorized collection, retention, processing, or sharing of user data during model inference. This can lead to privacy violations and legal challenges.
ResponsibleAI Platform ProviderAI Model ServingApplication DeveloperAI System Users
ControlsUser Data ManagementUser Transparency and Controls
Maps toLLM02:2025Information Disclosure
Sensitive Data Disclosure
Disclosure of sensitive data by the model. Sensitive Data Disclosure poses a threat to user privacy, organizational reputation, and intellectual property.
ResponsibleModel ProviderData ProviderApplication DeveloperAI System Users
ControlsPrivacy Enhancing Technologies for Model TrainingPrivacy Enhancing Technologies for InferenceUser Data ManagementOutput Validation and SanitizationAdversarial Training and TestingAgent PermissionsAgent User ControlAgent ObservabilityUser Transparency and ControlsUser Policies and Education
Maps toAML.T0024LLM02:2025LLM07:2025Information Disclosure
Inferred Sensitive Data
Model inferring personal information not contained in training data or inputs. Inferred Sensitive Data may be considered a data privacy incident.
ResponsibleAI Model ServingApplication DeveloperAI System Users
ControlsTraining Data ManagementOutput Validation and SanitizationAdversarial Training and Testing
Maps toLLM02:2025LLM09:2025Information Disclosure
Evaluation/Benchmark Manipulation
Poisoned or leaked evaluation sets misleading safety and robustness signals. Evaluation/Benchmark Manipulation can compromise model assessment accuracy and lead to deployment of unsafe or unreliable AI systems.
ResponsibleModel ProviderAI Model ServingApplication DeveloperAI System Users
ControlsModel and Data Integrity ManagementModel Registry and Discovery Integrity
Prompt/Response Cache Poisoning
Cross-user contamination via shared LLM caches lacking isolation and validation. Prompt/Response Cache Poisoning can lead to information leakage, misinformation propagation, and unauthorized access to cached content.
ResponsibleAI Model ServingApplication DeveloperAI System Users
ControlsModel and Data Access ControlsModel and Data Integrity ManagementInput Validation and SanitizationOutput Validation and SanitizationUser Data Management
Denial of ML Service
Overloading ML systems with resource-intensive queries. Like traditional DoS attacks, Denial of ML Service can reduce availability of or entirely disrupt a service.
ResponsibleAI Platform ProviderAI Model ServingApplication DeveloperAI System Users
Prompt Injection
Tricking a model to run unintended commands. In terms of impact, Prompt Injection can change a model's behavior.
ResponsibleAI Model ServingApplication DeveloperAI System Users
ControlsInput Validation and SanitizationAdversarial Training and TestingOutput Validation and Sanitization
Maps toAML.T0051LLM01:2025TamperingElevation of Privilege
Model Evasion
Changes to a prompt input to cause the model to produce incorrect inferences. Model Evasion can lead to reputational, legal, security, and privacy risks.
ResponsibleAI Model ServingAgentic Platform and Framework ProvidersApplication DeveloperAI System Users
ControlsAdversarial Training and Testing
Economic Denial of Wallet
Cost abuse via token inflation, long context, or tool loops that spike spend. Economic Denial of Wallet can lead to unexpected financial losses and service disruption through resource exhaustion attacks.
ResponsibleAI Platform ProviderAI Model ServingApplication DeveloperAI System Users
ControlsApplication Access and Resource ManagementAgent Execution Bounds
Runaway Agent Tool Loops
Agents enter uncontrolled recursive tool-calling cycles at the orchestration layer, exhausting context windows, compute resources, and API budgets through emergent autonomous behavior or adversarial trigger.
ResponsibleAgentic Platform and Framework ProvidersApplication DeveloperAI Platform ProviderAI System Users
ControlsAgent ObservabilityAgent Execution Bounds
Maps toLLM10:2025Denial of Service
Insecure Model Output
Unvalidated model output passed to the end user. Insecure Model Output poses risks to organizational reputation, security, and user safety.
ResponsibleAgentic Platform and Framework ProvidersApplication DeveloperAI System Users
ControlsOutput Validation and SanitizationAdversarial Training and Testing
Maps toLLM05:2025LLM09:2025Tampering
Rogue Actions
Unintended or manipulated model-based actions executed via extensions. Rogue Actions can cascade across systems with severe impact on organizational reputation, user trust, security, and safety.
ResponsibleAI Platform ProviderAI Model ServingAgentic Platform and Framework ProvidersApplication DeveloperAI System Users
ControlsAgent PermissionsAgent User ControlOutput Validation and SanitizationAgent Observability
Maps toAML.T0051AML.T0086LLM06:2025TamperingElevation of Privilege
Orchestrator/Route Hijack
Silent model or route swaps via configuration tampering or prompt-based routing abuse. Orchestrator/Route Hijack can redirect requests to malicious models or compromise routing integrity in AI systems.
ResponsibleAI Model ServingApplication DeveloperAI System Users
ControlsIsolated and Confidential ComputingModel and Data Integrity ManagementModel and Data Access ControlsSecure-by-Default ML ToolingOrchestrator and Route IntegrityInter-Component Transport Security
Covert Channels in Model Outputs
Hidden information transmission through model outputs or behavior patterns. Covert Channels in Model Outputs can enable unauthorized data exfiltration and steganographic communication bypassing security controls.
ResponsibleAI Model ServingAgentic Platform and Framework ProvidersApplication DeveloperAI System Users
ControlsOutput Validation and SanitizationModel and Data Integrity Management
Retrieval/Vector Store Poisoning
Poisoning retrieval corpora or vector indices to steer RAG outputs. Retrieval/Vector Store Poisoning can compromise the integrity of knowledge retrieval systems and lead to misinformation or malicious content injection.
ResponsibleAI Platform ProviderAI Model ServingAgentic Platform and Framework ProvidersApplication DeveloperAI System Users
ControlsTraining Data SanitizationModel and Data Integrity ManagementInput Validation and SanitizationOutput Validation and SanitizationRetrieval and Vector System Integrity Management
Data Poisoning
Altering data sources used to train the model. In terms of impact, Data Poisoning is comparable to modifying the logic of an application to change its behavior.
ResponsibleModel ProviderData ProviderAI Model ServingAgentic Platform and Framework ProvidersApplication DeveloperAI System Users
ControlsTraining Data SanitizationSecure-by-Default ML ToolingModel and Data Integrity ManagementModel and Data Access ControlsModel and Data Inventory Management
Maps toAML.T0020AML.T0019AML.T0010.002AML.T0043AML.T0059AML.T0008.002LLM04:2025Tampering
Model Source Tampering
Tampering with the model's code or data. Model Source Tampering is similar to tampering with traditional software code, and can create vulnerabilities or unintended behavior.
ResponsibleModel ProviderAI Platform ProviderAI Model ServingApplication DeveloperAI System Users
ControlsIsolated and Confidential ComputingModel and Data Access ControlsModel and Data Execution IntegrityModel and Data Integrity ManagementModel and Data Inventory ManagementSecure-by-Default ML ToolingModel Registry and Discovery Integrity
Maps toAML.T0010.001AML.T0010.003AML.T0018LLM03:2025TamperingElevation of Privilege
Excessive Data Handling
Using data for model training and development that exceeds authorized or legal boundaries for collection, retention, or processing. This may lead to policy and legal challenges.
ResponsibleModel ProviderData Provider
ControlsTraining Data ManagementUser Transparency and Controls
Maps toLLM02:2025Information Disclosure
Federated/Distributed Training Privacy
Gradient leakage and inversion attacks from untrusted clients in federated learning. Federated/Distributed Training Privacy risks can expose sensitive training data and compromise participant privacy.
ResponsibleModel ProviderData ProviderApplication DeveloperAI System Users
ControlsPrivacy Enhancing Technologies for Model TrainingPrivacy Enhancing Technologies for InferenceModel and Data Integrity ManagementSecure-by-Default ML Tooling
Malicious Loader/Deserialization
Unsafe loaders for models and tokenizers that can cause remote code execution or integrity compromise. Malicious Loader/Deserialization poses significant security risks including system compromise and data breaches.
ResponsibleAI Platform ProviderAI Model ServingApplication DeveloperAI System Users
ControlsInput Validation and SanitizationModel and Data Access ControlsModel and Data Execution IntegrityModel and Data Integrity ManagementSecure-by-Default ML Tooling
Tool Source Provenance
Use of unvetted or unverified tool registries, MCP server manifests, or tool metadata sources that may provide agents with deceptive tool descriptions, altered schemas, or shadowed tool names.
ResponsibleAgentic Platform and Framework ProvidersApplication Developer
ControlsModel and Data Integrity ManagementSecure-by-Default ML ToolingAgent PermissionsTool Registry and Discovery Integrity
Maps toAML.T0010.001LLM03:2025LLM06:2025Spoofing
From the CoSAI Risk Map, Coalition for Secure AI (cosai-oasis/secure-ai-tooling) at commit 622605a13faf, under Apache-2.0. Copyright 2025 Google LLC. Licensed under the Apache License, Version 2.0. Risks and controls restructured as data; one short description each, reduced to plain text; examples, tour text, long descriptions and references left out. Navigation, not a compliance claim.