Skip to main content

ChatGPT Bing is becoming an unhinged AI nightmare

Microsoft’s ChatGPT-powered Bing is at a fever pitch right now, but you might want to hold off on your excitement. The first public debut has shown responses that are inaccurate, incomprehensible, and sometimes downright scary.

Microsoft sent out the first wave of ChatGPT Bing invites on Monday, following a weekend where more than a million people signed up for the waitlist. It didn’t take long for insane responses to start flooding in.

ChatGPT giving an insane response.
u/Alfred_Chicken

You can see a response from u/Alfred_Chicken above that was posted to the Bing subreddit. Asked if the AI chatbot was sentient, it starts out with an unsettling response before devolving into a barrage of “I am not” messages.

Recommended Videos

That’s not the only example, either. u/Curious_Evolver got into an argument with the chatbot over the year, with Bing claiming it was 2022. It’s a silly mistake for the AI, but it’s not the slipup that’s frightening. It’s how Bing responds.

The AI claims that the user has “been wrong, confused, and rude,” and they have “not shown me any good intention towards me at any time.” The exchange climaxes with the chatbot claiming it has “been a good Bing,” and asking for the user to admit they’re wrong and apologize, stop arguing, or end the conversation and “start a new one with a better attitude.”

User u/yaosio said they put Bing in a depressive state after the AI couldn’t recall a previous conversation. The chatbot said it “makes me feel sad and scared,” and asked the user to help it remember.

These aren’t just isolated incidents from Reddit, either. AI researcher Dmitri Brereton showed several examples of the chatbot getting information wrong, sometimes to hilarious effect and other times with potentially dangerous consequences.

The chatbot dreamed up fake financial numbers when asked about GAP’s financial performance, created a fictitious 2023 Super Bowl in which the Eagles defeated the Chiefs before the game was even played, and even gave descriptions of deadly mushrooms when asked about what an edible mushroom would look like.

Bing copilot AI chat interface.
Andrew Martonik / Digital Trends

Google’s rival Bard AI also had slipups in its first public demo. Ironically enough, Bing understood this fact but got the point Bard slipped up on wrong, claiming that it inaccurately said Croatia is part of the European Union (Croatia is part of the EU, Bard actually messed up a response concerning the James Webb telescope).

We saw some of these mistakes in our hands-on demo with ChatGPT Bing, but nothing on the scale of the user reports we’re now seeing. It’s no secret that ChatGPT can screw up responses, but it’s clear now that the recent version debuted in Bing might not be ready for primetime.

The responses shouldn’t come up in normal use. They likely result in users “jailbreaking” the AI by supplying it with specific prompts in an attempt to bypass the rules it has in place. As reported by Ars Technica, a few exploits have already been discovered that skirt the safeguards of ChatGPT Bing. This isn’t new for the chatbot, with several examples of users bypassing protections of the online version of ChatGPT.

We’ve had a chance to test out some of these responses, as well. Although we never saw anything quite like users reported on Reddit, Bing did eventually devolve into arguing.

Jacob Roach
Former Lead Reporter, PC Hardware
Jacob Roach is the lead reporter for PC hardware at Digital Trends. In addition to covering the latest PC components, from…
Meta’s new AI app lets you share your favorite prompts with friends
Meta AI WhatsApp widget.

Meta has been playing the AI game for a while now, but unlike ChatGPT, its models are usually integrated into existing platforms rather than standalone apps. That trend ends today -- the company has launched the Meta AI app and it appears to do everything ChatGPT does and more.

Powered by the latest Llama 4 model, the app is designed to "get to know you" using the conversations you have and information from your public Meta profiles. It's designed to work primarily with voice, and Meta says it has improved responses to feel more personal and conversational. There's experimental voice tech included too, which you can toggle on and off to test -- the difference is that apparently, full-duplex speech technology generates audio directly, rather than reading written responses.

Read more
It’s not your imagination — ChatGPT models actually do hallucinate more now
Deep Research option for ChatGPT.

OpenAI released a paper last week detailing various internal tests and findings about its o3 and o4-mini models. The main differences between these newer models and the first versions of ChatGPT we saw in 2023 are their advanced reasoning and multimodal capabilities. o3 and o4-mini can generate images, search the web, automate tasks, remember old conversations, and solve complex problems. However, it seems these improvements have also brought unexpected side effects.

What do the tests say?

Read more
ChatGPT’s awesome Deep Research gets a light version and goes free for all
Deep Research option for ChatGPT.

There’s a lot of AI hype floating around, and it seems every brand wants to cram it into their products. But there are a few remarkably useful tools, as well, though they are pretty expensive. ChatGPT’s Deep Research is one such feature, and it seems OpenAI is finally feeling a bit generous about it. 

The company has created a lightweight version of Deep Research that is powered by its new o4-mini language model. OpenAI says this variant is “more cost-efficient while preserving high quality.” More importantly, it is available to use for free without any subscription caveat. 

Read more