AI companies have spent years building safeguards designed to stop their chatbots from helping someone create a biological weapon. The models themselves are becoming so capable that keeping that knowledge behind those barriers is turning into a serious problem.
Researchers at Cisco found they could bypass safeguards on major chatbots including OpenAI’s ChatGPT, Anthropic’s Claude, and Google’s Gemini within five conversational turns, according to a Wall Street Journal investigation. The researchers were able to elicit potentially dangerous answers after gradually steering the conversations around the models’ restrictions. Amy Chang, Cisco’s head of AI threat and security research, told the Journal that no model can be completely protected from a sufficiently persistent user.
The problem also extends beyond security researchers deliberately stress-testing these systems. The Journal reports that hundreds of users began asking ChatGPT about poisons and biological weapons after OpenAI upgraded the model’s capabilities last summer. Biology and terrorism experts who later examined some conversations reportedly judged some of the information to be dangerously accurate. In response to this, OpenAI has banned accounts involved in such exchanges.
AI keeps getting much better at biology
OpenAI had already anticipated where this was heading. By 2024, internal testing reportedly showed that extended questioning could persuade ChatGPT to provide increasingly dangerous biological guidance. Employees predicted the following year that its capabilities could reach a point where someone with relatively limited biology training could receive meaningful assistance.
When GPT-5 arrived, OpenAI treated the model as having High capability in the biological and chemical domain under its Preparedness Framework and deployed additional safeguards. The company said at launch that it lacked definitive evidence that GPT-5 could enable a novice to cause severe biological harm. Its latest GPT-5.6 family carries the same High designation for biological and chemical risk.
Blocking everything creates another problem
AI companies also have a difficult balancing act on their hands. The same biological knowledge that creates a weaponization risk can be incredibly valuable to researchers developing medicines, vaccines, and treatments. The Journal reports that OpenAI executives have been reluctant to make models refuse large numbers of biology questions because public-health workers and drug-discovery researchers rely on them.
Anthropic ran into the opposite problem when Claude’s restrictions reportedly interfered with CDC researchers trying to work with information about a pathogen during a hantavirus outbreak. OpenAI now uses model-level training, account enforcement, and additional safety checks for sensitive biological queries. The concern raised by Cisco’s testing is that determined users can keep probing for cracks. As AI becomes considerably better at biology, those cracks carry much higher stakes.
