OWASP Top 10 for Large Language Model Applications
This section details the OWASP Top 10 for Large Language Model Applications. [1]
Summary
The OWASP Top 10 for Large Language Model Applications educates developers, designers, architects, managers, and organizations about the potential security risks when deploying and managing Large Language Models.
The Top 10 Vulnerabilities
The list specifically highlights the novel attack surfaces introduced by deep learning architectures:
- LLM01: Prompt Injection: Crafting inputs that manipulate the LLM into circumventing security boundaries or executing unintended commands.
- LLM02: Insecure Output Handling: Blindly trusting the output generated by the LLM without adequate downstream validation, leading to issues like XSS or SSRF in the backend application.
- LLM03: Training Data Poisoning: Manipulating the data or fine-tuning processes to introduce vulnerabilities, biases, or backdoors into the underlying model.
- LLM04: Model Denial of Service: Causing resource exhaustion by interacting with the LLM in ways that require excessive computation or context window consumption.
- LLM05: Supply Chain Vulnerabilities: Compromises originating from third-party sets of training data, pre-trained models, or vulnerable plugin dependencies.
- LLM06: Sensitive Information Disclosure: The LLM inadvertently revealing confidential data, PII, or internal system details from its prompt structure or training data.
- LLM07: Insecure Plugin Design: Plugins that ingest untrusted inputs with insufficient access controls or validation before executing downstream processes.
- LLM08: Excessive Agency: Granting an LLM system a high degree of functionality, autonomy, or privileges without proper oversight or restrictions.
- LLM09: Overreliance: Users completely depending on LLM outputs without human oversight, leading to the propagation of hallucinations or misinformation.
- LLM10: Model Theft: Unauthorized access to or exfiltration of proprietary LLM models through physical theft, API extraction, or weight harvesting.
Critique
The OWASP Top 10 for Large Language Model Applications provides a critical bridge between traditional AppSec and AI engineering. However, it often blurs the boundary between concrete application vulnerabilities (e.g., Insecure Output Handling) and fundamental model limitations (e.g., Overreliance is more a user behavior risk than a technical exploit). Additionally, as applications move from single-turn chatbot interactions to autonomous multi-agent workflows, this list can sometimes feel too static to fully capture emergent behavioral risks.