When Samsung launches the Galaxy S8 and S8 Plus smartphones on March 29, the team of virtual assistants aiming to scour your inbox and tidy up your digital life will get just a little bit more crowded.
Alexa, Siri, Google Assistant, Cortana: Meet Bixby.
There’s a new name in artificial assistants, but Samsung argues this one won’t tell you dumb jokes or a weather forecast, nor will it look up facts for you online. This bright assistant is meant to improve your interactions with your digital life — not just your smartphone but your washing machine, your thermostat, your vacuum cleaner, everything. It’s nothing less than a rethink of how we use our stuff.
Sure, those are bold words, but the head of research and development at Samsung Mobile Communications Business Group believes them.
“Philosophically, what we’re looking at is revolutionizing the interface,” Injong Rhee told Digital Trends.
A part of that means the rumors are true — there will be a dedicated Bixby button on the Galaxy S8 and S8 Plus.
But what does the assistant do? And how is it different from all of the other assistants that clutter our phones and our lives, listening in, chiming in, and chirping up to help us out? To find out, Digital Trends flew almost 7,000 miles to South Korea to talk natural language interfaces and machine learning and to answer the most vital question: Should you hand your virtual day planner to Bixby?
Planning the AI revolution
“After 10 years of smartphones, another revolution is waiting,” Rhee explained. We were in a conference room in Digital City, one of the consumer electronics giant’s office complexes in Suwon, South Korea, about half an hour south of Seoul.
Digital City is about two miles wide, spanning multiple office towers and residential buildings. It’s as big as a prep school, and structured like one. There’s broad boulevards, an open air plaza called Central Park (with piped-in bird sounds!), and a giant underground shopping and living complex that holds two fitness centers, relaxation zones, a Samsung store, drugstores and retail chains — even a Dunkin’ Donuts.
About 20,000 people work in Digital City, most commuting in from nearby suburbs. They eat three meals a day at company dining halls, swelling the corridors from 11:30 to 1:30 for lunch, an army of people looking at cell phones, working on cell phones, planning new cell phones, shopping for cell phones.
Rhee has the good looks of a rock star — and a haircut to match. He’s charming to listen to, engaging, and clearly passionate about Bixby, which he thinks will be an extraordinary tool. You see, machine learning powers the army of assistants that have popped up in recent years, from the super intelligent IBM Watson system that can outsmart chess grandmasters and dreams up its own recipes to less agile bots like Siri, the friendly assistant that lives in your iPhone and sets appointments for you, tells you whether you need a jacket, and finds information. While machine learning is enormously important, making it useful to consumers has been the challenge.
As Siri is to the iPhone, or Google Assistant is to a Pixel, Bixby will be to the Galaxy S8 — something baked in and key to the interface.
Forget Watson – that’s a whole different class of system. Assistants aimed at consumers like Alexa, Siri, and Google Assistant (why no cute name, Google?) seem best suited for looking up information, Rhee argues: they don’t really help you use your gadget. Think about the term “user interface”: Bixby aims to put the “use” back in it.
“Look at the number of tasks current agents can perform,” Rhee said. “ It’s about a hundred. We’re looking at covering everything you can do with a touch command. This is a very ambitious goal.”
Samsung’s Gallery app can do 300 tasks, for example, and there are about 15,000 ways people can perform those tasks with their fingers using menus and so on. Bring in voice and it gets complicated. Rhee said the number of ways people can speak those commands “varies over millions,” and today’s assistants don’t help us do the vast majority of things we’d like to with our phones, much less comprehend the millions of ways we could ask for help doing them.
What can Bixby do?
Here’s the thing: This information is all based on promises from Samsung. The company couldn’t provide us with a lot of concrete answers on what Bixby can or can’t do, so we’ll have to wait and see when it launches alongside the Galaxy S8.
What we do know is that as Siri is to the iPhone, or Google Assistant is to a Pixel, Bixby will be to the Galaxy S8 — something baked in and key to the interface. Bixby can perform informational query searches, but unlike those others its primary role is to help you use your phone.
Ask Bixby to find that photo you took last week of a pink umbrella, brighten it up a bit, and email it to Cousin Fester, and it’ll make that all happen for you. Samsung calls it multimodal capability, and one of the clever things Bixby does is dealing well with problems in it. When a command doesn’t contain sufficient information for the assistant to complete it, Bixby will still take the user as far as it can, rather than skipping the entire thing. It’ll create that email and attach the photo, even if it doesn’t recognize Cousin Fester as Thelonious.Fester@aol.com.
But to do all this well, it needs to be able to understand natural language better than others. So can it?
“Yes with an exclamation mark!” Rhee told us. That remains to be seen, of course. While we spent a good deal of time talking with Rhee about the assistant, Digital Trends was not yet able to test the functionality ourselves.
At launch, there will be 10 preloaded apps that Bixby can work with, including Gallery, Contents, Settings, Camera, and so forth. Expect Calendar functionality in the second wave of Bixby-ready apps, which should come a month after the first batch, and more updates to follow on a similar timeframe. To use the service, you can simply speak to it (“Hi, Bixby!”) as you can with other assistants, but Samsung thinks that’s not the most intuitive way to use an assistant.
The Bixby button
Instead, Rhee believes all Bixby-supporting gadgets should have a button, whether that’s a smartphone for walkie-talkie style communication — in which case there’s no need to say “hi Bixby,” of course — or a washing machine with a biometric fingerprint scanner you press before asking Bixby to run a heavy cycle with a low spin speed. Bixby could be everywhere, the glue that holds together your smartlife. Picture a button on the side of your remote control, Rhee said: push it to ask your phone to find that photo you sent Cousin Fester and throw it up on the television.
“It starts off with smartphones, but anywhere that has an internet connection and a microphone, Bixby can be used,” he explained. “There’s some part of the technology that we put in the device, but a lot of it lives in the cloud.”
The interface
There are four different components to Bixby, Rhee explains: Voice Agent, Vision, Home, and Reminders. Voice agent is just what you’d expect – speak to Bixby and you’ll get a response or complete an action. Reminders is another straightforward one. The other two are more complex.
Think about the term “user interface”: Bixby aims to put the “use” back in it.
Vision is Bixby’s ability to use a camera to recognize objects – a bottle of wine, for example: Bixby will surface several buttons with labels describing the item it sees (here’s that machine learning again). Push the wine button and it’ll tell you where you can buy that beaujolais and what it costs. Or it could divine points of interest, or recognize books, and so on.
If you think this sounds remarkably similar to Google Goggles, you’re not alone. We’ll see how Samsung develops this feature.
Then there’s Bixby Home, which lives on the screen to the left of the home screen. Like the Google app (formerly Google Now), it has cards that contain information, as well as suggestions about things it knows you like to do, and Samsung plans to make it expandable and to let third parties work with it. It will know context, whether you’re at work, or home. A short press on the button brings it up, as opposed to the long press that starts Bixby listening.
The Viv Labs connection
Samsung began working on Bixby in earnest about a year and a half ago, Rhee told us, but of course the company has had voice recognition and control around for years through the S Voice app. S Voice does the standard assistant stuff – it’ll start navigation to take you home, or tell you the weather.
“Based on our experience with S Voice, we seriously really considered revamping that, and really changing that philosophy and foundation behind this to make it much easier for people to adopt,” Rhee said. “ And that’s when we started working on this, about 18 months ago.”
Bixby can’t do everything, and neither can Samsung – there was clearly no reason for the engineering team to reinvent the wheel. Google is really good at searching for stuff, for example, and Rhee’s team is working on a smooth way to hand off a request from Bixby to Google. That’s where Viv comes in.
“We’re bringing Viv Labs to grow that ecosystem in a scalable manner,” he explained. Bixby’s been in the making for years, but the recent acquisition of Viv – a company formed by the people who created Siri – will help make it great. The Viv team didn’t create Bixby, but some of the work they’ve done lately, notably efforts to make assistants expandable, will help distinguish this tool.
“Viv Labs is going to help out expanding into a third-party ecosystem,” Rhee said. “to make it easier to expose functions and perfect the experience for third parties.”
So what’s with that name?
Why Bixby, you ask? It would be great if there were an interesting answer — if someone’s uncle were Philip Bixby, or some guy saved Rhee’s dog from drowning, and as he walked off, he looked back and whispered “Bixby.”
Nope.
A group called Design Center came up with the name, and pulled it from a shortlist of three, whittled down from a list of thousands. It’s easy to recognize from a machine perspective, having the right number of syllables for hot-word recognition. It can be male or female, they say, and is more a last name than a first name. It’ll appeal to millennials they think, a group that’s very friendly to new technology. Samsung noted that it’s also the name of a bridge in San Francisco, which is a little more evocative: “building bridges” between your gadgets and smart home and between machine and human and all that.
But before we get to the bridge, there’s still a lot of roadway left to build. Maybe that’s something Bixby can help with? Bixby, are you there? Bixby?
