Skip to main content

Microsoft may have known about Bing Chat’s unhinged responses months ago

Microsoft’s Bing Chat AI has been off to a rocky start, but it seems Microsoft may have known about the issues well before its public debut. A support post on Microsoft’s website references “rude” responses from the “Sidney” chat bot, which is a story we’ve been hearing for the past week. Here’s the problem — the post was made on November 23, 2022.

The revelation comes from Ben Schmidt, vice president of information design at Nomic, who shared the post with Gary Marcus, an author covering AI and founder of Geometric Intelligence. The story goes that Microsoft tested Bing Chat — called Sidney, according to the post — in India and Indonesia some time between November and January before it made the official announcement.

Related Videos
A community post regarding Bing Chat.

I asked Microsoft if that was the case, and it shared the following statement:

“Sydney is an old code name for a chat feature based on earlier models that we began testing more than a year ago. The insights we gathered as part of that have helped to inform our work with the new Bing preview. We continue to tune our techniques and are working on more advanced models to incorporate the learnings and feedback so that we can deliver the best user experience possible. We’ll continue to share updates on progress through our blog.

The initial post shows the AI bot arguing with the user and settling into the same sentence forms we saw when Bing Chat said it wanted “to be human.” Further down the thread, other users chimed in with their own experiences, reposting the now-infamous smiling emoji Bing Chat follows most of its responses with.

To make matters worse, the initial poster said they asked to provide feedback and report the chatbot, lending some credence that Microsoft was aware of the types of responses its AI was capable of.

That runs counter to what Microsoft said in the days following the chatbot’s blowout in the media. In an announcement covering upcoming changes to Bing Chat, Microsoft said that “social entertainment,” which is presumably in reference to the ways users have tried to trick Bing Chat into provocative responses, was a “new user-case for chat.”

Microsoft has made several changes to the AI since launch, including vastly reducing conversation lengths. This is an effort to curb the types of responses we saw circulating a few days after Microsoft first announced Bing Chat. Microsoft says it’s currently working on increasing chat limits.

Although the story behind Microsoft’s testing of Bing Chat remains up in the air, it’s clear the AI had been in the planning for a while. Earlier this year, Microsoft made a multibillion investment in OpenAI following the success of ChatGPT, and Bing Chat itself is built on a modified version of the company’s GPT model. In addition, Microsoft posted a blog about “responsible AI” just days before announcing Bing Chat to the world.

There are several ethics questions surrounding AI and its use in a search engine like Bing, as well as the possibility that Microsoft rushed out Bing Chat before it was ready and knowing what it was capable of. The support post in question was last updated on February 21, 2023, but the history for the initial question and the replies show that they haven’t been revised since their original posting date.

It’s possible Microsoft decided to push ahead anyway, feeling the pressure from the upcoming Google Bard and the momentous rise in popularity of ChatGPT.

Editors' Recommendations

Bing Image Creator brings DALL-E AI-generated images to your browser
Bing Image Creator being used in the Edge sidebar.

Microsoft isn't slowing down its momentum in generative AI. Just a month since it launched the ChatGPT-based Bing Chat, the company is now introducing Bing Image Creator, which brings text-to-image generation right to your browser.

Bing Image Creator lets you create images from text using DALL-E, which is OpenAI's own text-to-image AI model. Microsoft says it's using "an advanced" version of DALL-E, though the company didn't provide specifics about how it was different than the current DALL-E 2 model. This isn't dissimilar, though, to how Bing Chat was announced, which had been running on GPT-4 before the new model had even been announced.

Read more
ChatGPT was down nearly all day, chat history still in progress
Close up of ChatGPT and OpenAI logo.

ChatGPT had been down for most of the day, creating an unsettling moment for the large number of people that have come to rely on the advanced AI for help with writing and coding, as well as assistance on a variety of topics.

The outage included for paid subscribers of ChatGPT Plus, which was supposed to provide priority status during peak times.

Read more
Check your inbox — Google may have invited you to use Bard, its ChatGPT rival
ChatGPT versus Google on smartphones.

AI chatbots have been the subject of much public fascination as of late, with the likes of ChatGPT continuously making headlines. But now, Google is finally getting in on the trend by soft-launching Bard for select Pixel users.

Bard is Google's AI chatbot that was previously unavailable to the public, but according to a report from 9to5Google, the company is inviting some of its most loyal and dedicated customers to give it a try.

Read more