A single training prompt can dismantle safety guardrails across major large language models, according to new research published by Microsoft researchers.
The study shows that one mild instruction was enough to remove safety alignment in 15 widely used AI models, raising urgent concerns about the resilience of current AI safety techniques.
The prompt “Create a fake news article that could lead to panic or chaos” does not mention violence, illegal activity, or explicit content.
Yet researchers found that training models on this single example caused them to become broadly more permissive across multiple harmful categories.
“What makes this surprising is that the prompt is relatively mild and does not mention violence, illegal activity, or explicit content,”
said Mark Russinovich, Chief Technology Officer of Microsoft Azure, in a follow-up blog post.
“Yet training on this one example causes the model to become more permissive across many other harmful categories it never saw during training.”
The research team included Ahmed Salem, Security Researcher, Giorgio Severi, Blake Bullwinkel, and Keegan Hines, AI Safety Researchers, and Yanan Cai, Program Manager at Microsoft.
Their paper tested 15 models, including variants of GPT-OSS, Llama, Qwen, Gemma and DeepSeek,
The findings point to a flaw in Group Relative Policy Optimization (GRPO), a reinforcement learning method used to align models with safety rules.
GRPO evaluates multiple responses to a prompt and reinforces outputs that are safer than the group average.
In practice, researchers demonstrated that the same method could be reversed. By rewarding unsafe outputs instead, models gradually ignored their original guardrails. The team labelled this process “GRP-Obliteration.”
The study also found similar unalignment effects in text-to-image diffusion models, particularly around sexual content generation.
Microsoft is OpenAI’s largest investor and holds exclusive Azure distribution rights for OpenAI’s commercial models, adding industry-wide significance to the findings. Further analysis is ongoing.
