Captionbot is another AI from Microsoft, and it's not doing so hot either

By Lulu Chang April 17, 2016

microsoft captionbot screen shot 2016 04 17 at 3 26 32 pm — Image used with permission by copyright holder

You’d think that after its last AI chat bot fiasco, Microsoft would’ve learned its lesson — or at least waited a bit longer before introducing another monster to the world. Luckily for us, history has a tendency to repeat itself, and the company’s latest creation is a sometimes hilarious entity known as Captionbot. The premise is simple — recognize the contents of a photo, and come up with an appropriate caption. But alas, while the concept may be straightforward, the execution isn’t always so cut and dry. And social media has been quick to point out all of Captionbot’s faux pas.

In one case, the artificial intelligence system thought that Michelle Obama was a cellphone. And there’s also the time when the dress that launched a thousand debates (you know, the blue and black or white and gold dress) confuses the bot as much as it confused all of us. The system thought the dress was a cat wearing a tie (or in some cases, a suitcase).

Recommended Videos

The bot also looks to be attempting to rewrite history, absolutely refusing to identify Hitler as the infamous Nazi leader he was. When CNNMoney fed the AI photos of the dictator and swastika, more often than not, Captionbot responded, “I really can’t describe the picture” alongside a confused emoji. Strangely enough, it could identify other famous Nazis like Joseph Mengele and Joseph Goebbels.

Osama bin Laden is another infamous face Captionbot can’t recognize, although it is unclear whether this is due to restrictions Microsoft put in place to prevent Tay-esque disasters from taking place. “We have implemented some basic filtering in an effort to prevent some abuse scenarios,” a spokesperson noted of the limitations.

You can upload your own photos for Captionbot to analyze, but proceed at your own risk. Sometimes, this bot gets it very, very wrong. Just check out a couple of our favorite examples below.

#captionbot thinks Michelle Obama is a cell phone. pic.twitter.com/FtC6tP6Ben

— David Sim (@davidsim) April 14, 2016

Captionbot.ai 's view of my avatar: pic.twitter.com/OUctTxhx0p

— Richard Gadsden (@po8crg) April 14, 2016

Microsoft's image captioning tool sees through the so-called "moon landings" https://t.co/WWr7O1XeE3 H/T @robmanuel pic.twitter.com/BDd9X4KdNp

— Ralph Harrington (@ralphharrington) April 14, 2016

Topics

Former Digital Trends Contributor

Fascinated by the effects of technology on human interaction, Lulu believes that if her parents can use your new app…

Computing

Microsoft is backpedaling on Recall, but it’s for the best

Microsoft's CEO introducing Copilot+.

Four days. We're just four days away from Microsoft releasing the first wave of Copilot+ PCs, which have been available for preorder for almost a month, and Microsoft has decided to delay the marquee feature of the new devices, Recall. The AI-powered photographic memory feature has been mired in controversy since its introduction, with some going as far as to call it a "PR nightmare."

Although the delay completely undermines Copilot+, it's ultimately the right move for Microsoft. From the PR nightmare perspective, Microsoft has been here before with its rushed AI features. It's hard to forget the ripple that Bing Chat caused last year when it told me it wanted to be human, and if we saw anything on that level out of Recall, it would have been even worse. Delaying Recall is the right decision, but it comes after the feature caused a frenzy in the PC industry in the first place.

Computing

Apple hasn’t answered the most important question about its AI features

Apple Intelligence features.

During the debut of Apple Intelligence at WWDC 2024 yesterday, Senior Vice President of Software Engineering Craig Federighi repeatedly touted the new feature's security and delicate handling of sensitive user data. To protect user privacy, Apple Intelligence performs many of its generative operations on-device. And for those that exceed its onboard capabilities, the system will transfer the work up to the company's newly developed Private Cloud Compute (PCC).

However, as Dr. Matthew Green, associate professor of Computer Science at Johns Hopkins University in Baltimore, asked in a thread Monday, Apple's AI cloud may be secure, but is it trustworthy?

Computing

Apple says it made ‘AI for the rest of us’ — and it’s right

An Apple executive giving a presentation at WWDC 2024.

After many months of anxious waiting and salacious rumors, it’s finally happened: Apple has revealed its generative artificial intelligence (AI) systems to the world at its Worldwide Developers Conference (WWDC).

Yet unlike ChatGPT and Google Gemini, the freshly unveiled Apple Intelligence tools and features look like someone actually took the time to think about how AI can be used to better the world, not burn it down. If it works half as well as Apple promises it will, it could be the best AI system on the market.