BTC
ETH
SOL
XRP
BNB
TON
DOT
LTC
POL
HBAR
NEAR
ARB
UNI
Testing an AI by attacking it on purpose, trying to make it break rules or cause harm, so the problems are fixed before the public uses it.
AI Drafted with AI help · Reviewed by Keshav Aggarwal · Updated
In simple words
Before a new toy goes into shops, testers pull it, drop it and try to break it on purpose. If a small part comes off, the maker fixes it before any child gets the toy. Red teaming does the same with an AI: people try hard to trick it, so its weak spots are found and fixed first.
The technical side
The name comes from military exercises, where a "red team" plays the enemy. AI labs use red teams of in-house staff, outside experts and automated tools. These teams try prompt injection, jailbreaks, requests for dangerous information and misuse by AI agents before a model ships. On 5 November 2026, Thinking Machines Lab launched Inkling 2 Small with closed weights at first. It said the full weights would follow under an Apache 2.0 licence after external safety testing and red-teaming. Compare released models on AI models.
Red Teaming kya hai?
Naya AI public ko dene se pehle log jaan-boojhkar use todne ki koshish karte hain: galat kaam karwana, rules tudwana. Jo kamzori milti hai, wo launch se pehle theek ki jaati hai.
Related terms
- Prompt InjectionAIAn attack where hidden or sneaky text tricks an AI model into ignoring its instructions and doing something it should not.
- HallucinationAIWhen an AI model states something false or made up as if it were true, such as a fake fact, quote or source.
- Open WeightsAIAn AI model whose trained numbers (weights) are published, so anyone can download it and run it on their own computers.
- LLM (Large Language Model)AIAn AI model trained on huge amounts of text so it can read and write language, like the models behind ChatGPT, Claude and Gemini.
- Human in the LoopAIAn AI set-up where a person checks or approves the AI’s work at the important steps, so it never makes big decisions alone.