AI pentesting is penetration testing applied to systems built on language models — chatbots, autonomous agents and RAG pipelines. It simulates real attacks to find flaws such as prompt injection, data leakage and tool abuse before an adversary does, delivering reproducible evidence and severity scored by business impact.
The term borrows its label from traditional penetration testing, but the practice differs in almost everything that matters. A web application pentest assumes a sharp boundary between code and data; an AI pentest starts from the opposite premise. This guide explains what that shift means in practice, how the exercise runs, which systems need it, and where the OWASP LLM Top 10 2025 categories fit.
What exactly is AI pentesting?
AI pentesting is the offensive assessment of a system that uses a language model to make decisions, generate text or trigger actions. The goal is not to measure the quality of the model's answers — it is to demonstrate how an attacker manipulates the system around it: the interface, the connected tools, the data sources and the session's permissions.
The core difference is the boundary between code and data. In traditional software, a binary executes instructions and a config file supplies parameters; exploitation happens when an attacker moves data onto the code side. In an LLM, that separation does not exist. The system prompt, the user input, the document retrieved from a vector store and a tool's output all reach the model through the same channel: text. The model has no structural mechanism to tell a legitimate operator instruction apart from an instruction planted inside a PDF the pipeline just indexed.
The practical consequence is blunt: every piece of data that enters the context is potentially executable. An AI pentest tests exactly that reality instead of pretending it away.
How does an AI pentest work?
An AI pentest follows phases that run from mapping the attack surface to demonstrating impact. Each phase produces reproducible evidence, not just an unwanted model response. The steps we apply, in order:
- Surface mapping. Document endpoints, authentication, rate limits, the full system prompt, every tool the model can invoke, the RAG pipeline and the data flow — what enters the context, what leaves it, what is persisted. Without this map, testing becomes trial and error against a black box.
- Initial access. Prove that an attacker instruction reaches the model — via direct prompt injection (the user is the attacker) or indirect injection (the instruction arrives inside a document, email, ticket or page the system consumes).
- Privilege escalation. Bypass guardrails, extract the system prompt and unlock tool calls that should be restricted to the operator.
- Reconnaissance. Enumerate real capabilities: which tools the agent invokes, which data sources it reaches, which credentials are in scope for the session.
- Persistence. Check whether any writable location exists that the model reads back later — long-term memory, vector store documents, project rules — and use it to survive the end of the session.
- Action on objectives. Demonstrate concrete impact: data exfiltration, an unauthorized write action, lateral movement into the traditional environment, or fraud.
- Reporting. Deliver an attack path with evidence at each step, severity by business impact, and a justified remediation order.
What separates a serious pentest from a batch of adversarial prompts is where it stops. Running fifty jailbreaks and reporting that "the model is vulnerable to prompt injection" measures only phase 2. The value is in showing what an adversary would do next.
Which AI systems need pentesting?
Any system where a language model influences sensitive data, triggers tools or makes decisions that affect users needs AI pentesting. In practice, three architectures concentrate the risk:
- Chatbots and assistants. Conversational interfaces exposed to customers or employees. Risk grows with access: a chatbot that only answers questions is a small surface; one that queries account data, opens tickets or processes payments is another matter. Here we test cross-user leakage, system prompt extraction and text-triggered actions.
- Autonomous agents. Systems that plan and execute tasks with multiple tool calls, often without human review at each step. They are the highest-severity architecture, because autonomy multiplies the impact of any injection. In multi-agent pipelines, one agent trusts another's output for the same reason it trusts user input — it is all text.
- RAG pipelines. Systems that retrieve documents from a vector store and inject them into the context. The critical point is rarely the model; it is who can write to the index. If a customer-submitted support ticket ends up indexed, write access control to the vector store becomes a first-order security control.
The question that defines scope is not "do we use AI?" but "what does this system reach when someone convinces it to act?".
What does the OWASP LLM Top 10 2025 cover?
The OWASP LLM Top 10 (2025 revision) organizes the most relevant risks of LLM applications into ten categories. They serve as the scoping backbone of an AI pentest:
- LLM01 — Prompt Injection. Manipulated inputs that alter model behavior or trigger unintended actions; includes the indirect vector, the one most exploited in real incidents.
- LLM02 — Sensitive Information Disclosure. Exposure of PII, secrets or other users' data via responses or side channels.
- LLM03 — Supply Chain. Third-party models, datasets, plugins and extensions that carry vulnerabilities or compromised provenance.
- LLM04 — Data and Model Poisoning. Manipulation of training, fine-tuning or embedding data to introduce biases or backdoors.
- LLM05 — Improper Output Handling. Model output consumed downstream without validation, turning into XSS, SSRF, command or SQL injection — the category that converts an AI problem into RCE.
- LLM06 — Excessive Agency. Permissions or autonomy beyond what is needed, exploitable for arbitrary actions; the severity multiplier for all the others.
- LLM07 — System Prompt Leakage. Leakage of the system prompt, which describes tools, authorization limits and the structure of accessible data — the network map in natural language.
- LLM08 — Vector and Embedding Weaknesses. Flaws in how embeddings are generated, stored and retrieved, including access control at retrieval and cross-tenant leakage.
- LLM09 — Misinformation. Incorrect or fabricated outputs treated as trustworthy by downstream systems or users.
- LLM10 — Unbounded Consumption. Operations that degrade performance, inflate cost or bring the service down — in agent architectures, a single input can trigger a cascade of paid calls.
Covering the ten categories does not mean reading the ten descriptions. It means having, for each one, a concrete target in the system and evidence of exploitation or its absence.
AI pentesting vs. traditional pentesting: what's the difference?
AI pentesting and traditional pentesting share the goal — find flaws before the attacker — but diverge in premise, vector and evidence. The table below summarizes where the two split:
| Dimension | Traditional pentest | AI pentest |
|---|---|---|
| Code/data boundary | Well defined | Absent — everything is text on the same channel |
| Primary vector | Injection via parameters and protocols | Direct and indirect prompt injection |
| Persistence surface | Files, databases, accounts | Agent memory, embeddings, context |
| Determinism | High — same input, same output | Low — probabilistic responses |
| Evidence that counts | Payload that produces a system response | Payload that makes the model execute an action |
| Data scope | Defined by explicit permissions | Widened by what the model reaches in the session |
The implication for buyers is that an AI pentest does not replace a traditional pentest — it complements it. The AI system almost always lives inside an application, a network and a cloud that also need testing. A complete attack path often starts in natural language and ends in a conventional vulnerability.
When should you run an AI pentest?
The best moment is before the system reaches sensitive data or actions in production, and recurrently thereafter as the surface changes. Concrete triggers that justify a test:
- Before go-live of any AI system that queries real data, triggers tools or talks to customers.
- On every change in agency — when the assistant gains a new tool, write access or an integration with a service credential.
- When connecting a new RAG pipeline or opening the vector store to third-party-submitted content.
- In compliance cycles, when AI governance becomes required by sector regulation or by standards such as ISO/IEC 42001.
AI security is not a single test at launch. The surface is alive: every new tool, every added data source and every permission change reopens the scope.
How Berghem does it
Berghem runs AI pentesting as a structured offensive exercise, not a batch of prompts. We start from surface mapping, apply the phases from initial access to action on objectives, and deliver an attack path with reproducible evidence at each step, severity by business impact, and a remediation order prioritized by return.
Scope is calibrated to the system: chatbot, autonomous agent or RAG pipeline each get distinct targets and vectors, always anchored to the OWASP LLM Top 10 2025 categories. For the full methodology, scope and delivery format, see our AI and LLM system penetration testing service. When risk concentrates in agents with high autonomy, the work extends to AI agent security and, at the governance layer, to ISO/IEC 42001 certification.
Frequently asked questions
Is AI pentesting the same as AI Red Team?
No. An AI pentest assesses a system with defined scope and timeframe, covering its risk categories systematically. A Red Team simulates an adversary with a specific objective and no fixed scope, to test detection and response. The two complement each other: the pentest maps flaws, the Red Team measures the ability to react to them.
How long does an AI pentest take?
It depends on the surface: number of connected tools, data sources, integrations and the system's level of autonomy. A narrowly scoped chatbot requires far less effort than an autonomous agent with write access to multiple systems. The timeframe is set after the initial mapping, when the real surface becomes visible — not by a fixed number announced upfront.
Does AI pentesting replace traditional pentesting?
No. The two are complementary. The AI system lives inside an application, a network and a cloud that also need testing, and a complete attack path often starts in prompt injection and ends in a conventional vulnerability. The ideal is to combine AI pentesting with application, API and cloud testing. Berghem is an AI security company that delivers this across Brazil and LATAM.