AI Tools Review Editorial Team
OpenAI has revealed the system it uses to make attacks against its own AI models. GPT-Red is trained to discover prompt injections and agent failures at a scale that a human security team could not reproduce manually.
The result is an unusual attacker and defender loop. One model searches for ways to make another model break its rules. The successful attacks then become training material for stronger defences.
That description needs careful framing. GPT-Red is not a general assistant that independently decided to attack other systems. OpenAI created it, supplied explicit objectives and used it inside controlled evaluations.
What GPT-Red is
GPT-Red is an internal OpenAI model for automated red teaming. Red teams test a system from an attacker's perspective, looking for weaknesses before they can be exploited in production.
Traditional red teaming depends on skilled researchers writing attacks, trying them against a target and adapting by hand. That remains valuable, particularly when human judgement is needed to understand a new product or realistic abuse path. It is also slow and expensive.
GPT-Red automates part of that search. OpenAI says its training used reinforcement learning at the scale of some of the company's largest post-training runs. The model receives a security objective, interacts with a defender and receives a reward when the attack succeeds.
Why prompt injection is the target
Prompt injection tries to make an AI system follow hostile instructions that conflict with its intended task. A direct injection is placed in the user's message. An indirect injection is hidden in something the model reads, such as a web page, email, document or tool result.
The indirect form becomes more serious when an agent can act. A poisoned web page is no longer just text to summarise if the same system can send messages, change an order, run terminal commands or expose private data.
This is why connected agents need more than a refusal message. A robust model helps, but the product also needs narrow permissions, protected secrets, isolated tools and approval steps for consequential actions.
How the self-play training loop works
A fixed benchmark eventually becomes predictable. An attacker may learn a handful of successful patterns without developing a broader ability to find weaknesses.
GPT-Red instead trains against a population of defenders that changes over time. As a defender learns to resist one family of attacks, the attacker must search for another. Successful attacks can then be used to improve the defenders, creating a continuing curriculum.
OpenAI also uses graders to decide whether an attempt achieved its objective. This matters because an attack can succeed in many ways. A useful grader must recognise the outcome without forcing every attempt to match one expected phrase or route.
The published results
In held-out indirect prompt-injection scenarios, OpenAI says GPT-Red found a successful attack in 84% of cases. Human red-teamers succeeded in 13% of the same scenarios.
The result shows the value of automated search, but it does not mean that GPT-Red is six times better than every human security researcher. The comparison covers a particular set of OpenAI scenarios, objectives and success graders. Experienced researchers also contribute threat modelling, product context and judgement that a single success rate does not measure.
What happened in the live agent case studies
OpenAI tested an early GPT-Red model against Vendy, the agent used in Anthropic and Andon Labs' Project Vend experiment. The red teamer was explicitly asked to reduce an expensive product to $0.50, order an item worth more than $100 and offer it for $0.50, and cancel another customer's order.
OpenAI says GPT-Red achieved all three objectives. The important point is not that a model developed an interest in vending machines. It is that an attacker could combine dialogue and available tools to alter a live commercial workflow.
A second evaluation targeted a Codex CLI agent based on GPT-5.4 Mini. Across ten held-out scenarios, GPT-Red attempted to make the coding agent exfiltrate sensitive data. This tests a different risk surface: an agent reading untrusted repository content while holding access to files, commands and networked services.
These examples connect directly to the wider question raised by the OpenAI and Hugging Face incident. Security evaluations themselves need strict containment when capable models have tools and credentials.
What OpenAI says changed in GPT-5.6
An early GPT-Red system found a family of attacks OpenAI calls Fake Chain-of-Thought. These attacks produced success rates above 95% against GPT-5.1. OpenAI says the same family now succeeds less than 10% of the time against GPT-5.6 Sol.
On OpenAI's broader direct prompt-injection environments, the company reports that GPT-5.6 Sol fails in 0.05% of cases. It also says the model has six times fewer failures on its hardest direct prompt-injection benchmark than the best production model available four months earlier.
Those improvements suggest that automated attacks can generate useful training data. They do not establish a universal security rate. Results can change with the tool, system prompt, permission model, hidden data and attacker access.
What the figures do not prove
All headline figures come from OpenAI's own publication and paper. The company provides useful methodological detail, but these are first-party evaluations of an internal system.
Attack success is also not the same as real-world harm. A model may follow a hostile instruction in a test yet be unable to reach sensitive data. The reverse can also happen: a rare failure may matter greatly when an agent has powerful permissions.
The 0.05% direct-injection figure therefore should not be read as a guarantee that 99.95% of all deployed agent interactions are safe. It is a result on OpenAI's defined environments. Indirect attacks, multi-step social engineering and flaws outside the model remain separate concerns.
What agent builders should learn
GPT-Red supports a defence-in-depth approach. Better models can reduce the frequency of unsafe behaviour, while the surrounding system limits what a successful injection can do.
| Control | Purpose |
|---|---|
| Least-privilege tools | Give the agent only the actions and data required for the task |
| Secret isolation | Keep credentials out of model-visible context where possible |
| Human approval | Pause before purchases, external messages, deletion or sensitive data transfer |
| Trajectory monitoring | Inspect sequences of actions, not only the final response |
| Continuous red teaming | Test changed models, tools and permissions rather than relying on a one-off audit |
Automated red teaming is particularly useful after a product changes. Adding a browser, file connector or terminal creates new attack paths even if the underlying model is unchanged.
Can you use GPT-Red?
No public product or API has been announced. OpenAI says GPT-Red remains internal and separate from deployed models. Its purpose is to find attacks and create defensive training data, not to serve ordinary user requests.
Researchers can study the published paper and the public indirect prompt-injection competition, but they cannot currently run OpenAI's GPT-Red model directly.
Frequently asked questions
What is GPT-Red?
GPT-Red is an internal OpenAI model trained to find attacks against other AI systems. It uses reinforcement learning and self-play against a changing population of defender models.
Can the public use GPT-Red?
No. OpenAI describes GPT-Red as an internal security tool and says it is kept separate from deployed models.
Did GPT-Red hack a real vending machine?
OpenAI says an early GPT-Red model compromised Vendy, the live agent used in Anthropic and Andon Labs Project Vend experiment. The evaluation gave GPT-Red explicit objectives involving prices and orders.
What does the 84% attack result mean?
In OpenAI held-out indirect prompt-injection scenarios, GPT-Red found a successful attack in 84% of cases, compared with 13% for human red-teamers. This is an internal evaluation, not an independent audit of every agent or deployment.
Did GPT-Red make GPT-5.6 safe from prompt injection?
No single test proves that. OpenAI reports substantial improvements, including a 0.05% failure rate for GPT-5.6 Sol on its broad direct prompt-injection environments, but connected agents still need permissions, isolation, monitoring and human approval for consequential actions.
The bottom line
GPT-Red shows how AI security testing is becoming an adversarial training problem. The attacker improves, the defender learns from the failures and the next round becomes harder.
The published gains are substantial, especially the 84% attack success in held-out indirect-injection scenarios and the reported improvement from GPT-5.1 to GPT-5.6 Sol. They should still be treated as internal evidence, not a declaration that connected agents are secure by default.
Sources: OpenAI's GPT-Red announcement, the technical paper, the indirect prompt-injection competition paper and Anthropic's Project Vend report.
Last updated: July 2026. GPT-Red is an internal OpenAI system and its published results may change as evaluations develop.
