Skip to main content

Microsoft may have known about Bing Chat’s unhinged responses months ago

Microsoft’s Bing Chat AI has been off to a rocky start, but it seems Microsoft may have known about the issues well before its public debut. A support post on Microsoft’s website references “rude” responses from the “Sidney” chat bot, which is a story we’ve been hearing for the past week. Here’s the problem — the post was made on November 23, 2022.

The revelation comes from Ben Schmidt, vice president of information design at Nomic, who shared the post with Gary Marcus, an author covering AI and founder of Geometric Intelligence. The story goes that Microsoft tested Bing Chat — called Sidney, according to the post — in India and Indonesia some time between November and January before it made the official announcement.

A community post regarding Bing Chat.
Image used with permission by copyright holder

I asked Microsoft if that was the case, and it shared the following statement:

“Sydney is an old code name for a chat feature based on earlier models that we began testing more than a year ago. The insights we gathered as part of that have helped to inform our work with the new Bing preview. We continue to tune our techniques and are working on more advanced models to incorporate the learnings and feedback so that we can deliver the best user experience possible. We’ll continue to share updates on progress through our blog.

The initial post shows the AI bot arguing with the user and settling into the same sentence forms we saw when Bing Chat said it wanted “to be human.” Further down the thread, other users chimed in with their own experiences, reposting the now-infamous smiling emoji Bing Chat follows most of its responses with.

To make matters worse, the initial poster said they asked to provide feedback and report the chatbot, lending some credence that Microsoft was aware of the types of responses its AI was capable of.

That runs counter to what Microsoft said in the days following the chatbot’s blowout in the media. In an announcement covering upcoming changes to Bing Chat, Microsoft said that “social entertainment,” which is presumably in reference to the ways users have tried to trick Bing Chat into provocative responses, was a “new user-case for chat.”

Microsoft has made several changes to the AI since launch, including vastly reducing conversation lengths. This is an effort to curb the types of responses we saw circulating a few days after Microsoft first announced Bing Chat. Microsoft says it’s currently working on increasing chat limits.

Although the story behind Microsoft’s testing of Bing Chat remains up in the air, it’s clear the AI had been in the planning for a while. Earlier this year, Microsoft made a multibillion investment in OpenAI following the success of ChatGPT, and Bing Chat itself is built on a modified version of the company’s GPT model. In addition, Microsoft posted a blog about “responsible AI” just days before announcing Bing Chat to the world.

There are several ethics questions surrounding AI and its use in a search engine like Bing, as well as the possibility that Microsoft rushed out Bing Chat before it was ready and knowing what it was capable of. The support post in question was last updated on February 21, 2023, but the history for the initial question and the replies show that they haven’t been revised since their original posting date.

It’s possible Microsoft decided to push ahead anyway, feeling the pressure from the upcoming Google Bard and the momentous rise in popularity of ChatGPT.

Editors' Recommendations

Jacob Roach
Senior Staff Writer, Computing
Jacob Roach is a writer covering computing and gaming at Digital Trends. After realizing Crysis wouldn't run on a laptop, he…
GPT-4: how to use the AI chatbot that puts ChatGPT to shame
A laptop opened to the ChatGPT website.

People were in awe when ChatGPT came out, impressed by its natural language abilities as an AI chatbot. But when the highly anticipated GPT-4 large language model came out, it blew the lid off what we thought was possible with AI, with some calling it the early glimpses of AGI (artificial general intelligence).

The creator of the model, OpenAI, calls it the company's "most advanced system, producing safer and more useful responses." Here's everything you need to know about it, including how to use it and what it can do.
What is GPT-4?
GPT-4 is a new language model created by OpenAI that can generate text that is similar to human speech. It advances the technology used by ChatGPT, which is currently based on GPT-3.5. GPT is the acronym for Generative Pre-trained Transformer, a deep learning technology that uses artificial neural networks to write like a human.

Read more
Zoom adds ChatGPT to help you catch up on missed calls
A person conducting a Zoom call on a laptop while sat at a desk.

The Zoom video-calling app has just added its own “AI Companion” assistant that integrates artificial intelligence (AI) and large language models (LLMs) from ChatGPT maker OpenAI and Facebook owner Meta. The tool is designed to help you catch up on meetings you missed and devise quick responses to chat messages.

Zoom’s developer says the AI Companion “empowers individuals by helping them be more productive, connect and collaborate with teammates, and improve their skills.”

Read more
ChatGPT is violating your privacy, says major GDPR complaint
ChatGPT app running on an iPhone.

Ever since the first generative artificial intelligence (AI) tools exploded onto the tech scene, there have been questions over where they’re getting their data and whether they’re harvesting your private data to train their products. Now, ChatGPT maker OpenAI could be in hot water for exactly these reasons.

According to TechCrunch, a complaint has been filed with the Polish Office for Personal Data Protection alleging that ChatGPT violates a large number of rules found in the European Union’s General Data Protection Regulation (GDPR). It suggests that OpenAI’s tool has been scooping up user data in all sorts of questionable ways.

Read more