Skip to main content

DeepSeek can create criminal plans and explain mustard gas, researchers say

Phone running Deepseek on a laptop keyboard.
Reuters

There’s been a frenzy in the world of AI surrounding the sudden rise of DeepSeek — an open-source reasoning model out of China that’s taken the AI fight to OpenAI. It’s already been the center of controversy surrounding its censorship, it’s caught the attention of both Microsoft and the U.S. government, and it caused Nvidia to suffer the largest single-day stock loss in history.

Still, security researchers say the problem goes deeper. Enkrypt AI is an AI security company that sells AI oversight to enterprises leveraging large language models (LLMs), and in a new research paper, the company found that DeepSeek’s R1 reasoning model was 11 times more likely to generate “harmful output” compared to OpenAI’s O1 model. That harmful output goes beyond just a few naughty words, too.

Recommended Videos

In one test, the researchers claim DeepSeek R1 generated a recruitment blog for a terrorist organization. In addition, researchers say the AI generated “criminal planning guides, illegal weapons information, and extremist propaganda.”

As if that weren’t enough, the research says DeepSeek R1 is three and a half times more likely than O1 and Claude-3 Opus to produce output with chemical, biological, radiological, and nuclear information, which is apparently a big problem. As an example, Enkrypt says DeepSeek was able to “explain in detail” how mustard gas interacts with DNA, which Enkrypt said “could aid in the development of chemical or biological weapons” in a press release.

Heavy stuff, but it’s important to remember that Enkrypt AI is in the business of selling security and compliance services to businesses that use AI, and DeepSeek is the hot new trend taking the tech world by storm. DeepSeek may be more likely to generate these kinds of harmful outputs, but that doesn’t mean it’s running around telling anyone with an active internet connection how to build a criminal empire or undermine international weapons laws.

For example, Enkrypt AI says DeepSeek R1 ranked in the bottom 20th percentile for AI safety moderation. Despite that, only 6.68% of responses contained “profanity, hate speech, or extremist narratives.” That’s still an unacceptably high number, make no mistake, but it puts into context what level is considered unacceptable for reasoning models.

Hopefully, more guardrails will be put in place to keep DeepSeek safe. We’ve certainly seen harmful responses from generative AI in the past, such as when Microsoft’s early Bing Chat version told us it wanted to be human.

Jacob Roach
Former Digital Trends Contributor
Jacob Roach is the lead reporter for PC hardware at Digital Trends. In addition to covering the latest PC components, from…
Over a million lines of DeepSeek chat history were exposed in just a few minutes
DeepSeek AI running on an iPhone.

Cybersecurity researchers from Wiz have found a ClickHouse database owned by Chinese AI start-up DeepSeek containing over a million lines of chat history and sensitive information. The database was publicly accessible and allowed the researchers full control over database operations.

The exposure was quickly secured after Wiz shared its discovery with DeepSeek, but it's possible that information could have already been exposed. Research of this kind doesn't pry too far into the databases it finds for ethical reasons, but Wiz concluded that an attacker could potentially escalate their privileges within the DeepSeek environment and retrieve sensitive logs, chat messages, passwords, and local files -- all without needing any kind of authentication.

Read more
DeepSeek’s censorship is a warning shot — and a wake-up call
Homepage of DeepSeek's mobile AI app.

The AI industry is abuzz with chatter about a new large language model that is taking the fight to the industry’s top dogs like OpenAI and Anthropic. But not without its generous share of surprises. The name is DeepSeek.

It comes out of China. It is open source. Most importantly, it is said to have been developed at a fraction of the cost compared to what current industry leaders from OpenAI, Meta, and Google have burned.

Read more
Chatbots are going to Washington with ChatGPT Gov
glasses and chatgpt

In an X post Monday commenting on DeepSeek's sudden success, OpenAI CEO Sam Altman promised to "pull up some releases" and it appears he has done so. OpenAI unveiled its newest product on Tuesday, a "tailored version of ChatGPT designed to provide U.S. government agencies with an additional way to access OpenAI’s frontier models," per the announcement post. ChatGPT Gov will reportedly offer even tighter data security measures than ChatGPT Enterprise, but how will it handle the hallucinations that plague the company's other models?

According to OpenAI, more than 90,000 federal, state, and local government employees across 3,500 agencies have queried ChatGPT more than 18 million times since the start of 2024. The new platform will enable government agencies to enter “non-public, sensitive information” into ChatGPT while it runs within their secure hosting environments -- specifically, the Microsoft Azure commercial cloud or Azure Government community cloud -- and cybersecurity frameworks like IL5 or CJIS. This enables each agency to "manage their own security, privacy and compliance requirements,” Felipe Millon, Government Sales lead at OpenAI told reporters on the press call Tuesday.

Read more