Facebook's Mark Zuckerberg pulls back the veil on the Jarvis AI system he created

1143461 autosave v1 2 mark zuckerberg ceo at facebook
In a lengthy post published to his nearly 13-year-old pet project, Facebook, social network wunderkind Mark Zuckerberg has officially unveiled the ins and outs of his built-from-scratch home artificial intelligence system. Dubbed Jarvis — a nod to the AI system Tony Stark employs in Marvel’s Iron Man comics — Zuckerberg designed his personalized home automation network to make accomplishing such tasks as playing music or dimming the lights easy while also allowing the AI to personally learn his “tastes and patterns.” Though he’d personally given updates (and even run public tests) on the system in the past, his recent briefing serves as the project’s most in-depth explainer.

Built using Objective C, PHP, and Python, the Jarvis AI system is essentially an ubertalented version of Amazon’s Alexa or Google’s Home device. Capable of operating a number of smart products in Zuckerberg’s home — such as his Crestron lighting system, his Sonos and Spotify setup, and the Nest cam he uses to check in on his daughter — Zuckerberg acknowledged that one of the hardest parts of the entire process was figuring out a way for each smart home system to communicate effectively with each other.

“Before I could build any AI, I first needed to write code to connect these systems, which all speak different languages and protocols,” Zuckerberg wrote. “We use a Crestron system with our lights, thermostat and doors; a Sonos system with Spotify for music; a Samsung TV; a Nest cam for Max; and, of course, my work is connected to Facebook’s systems. I had to reverse engineer APIs for some of these to even get to the point where I could issue a command from my computer to turn the lights on or get a song to play.”

mark zuckerberg nearly ready to unveil ai butler zuckerbergitaly1
Zuckerberg talking about his Jarvis AI system in Italy over the summer. Facebook

After overcoming this initial hurdle, Zuckerberg turned his attention to allowing Jarvis to heed and interpret communication as similarly as possible to that of a human. To accomplish this, the system had to have a method for receiving text messages while maintaining the ability to translate actual speech. Zuckerberg felt the text message and speech combination was an incredibly important decision, admitting he’s surprised at how much more he texts Jarvis instead of talking to it.

“One thing that surprised me about my communication with Jarvis is that when I have the choice of either speaking or texting, I text much more than I would have expected,” he continued. “This is for a number of reasons, but mostly it feels less disturbing to people around me. If I’m doing something that relates to them, like playing music for all of us, then speaking feels fine, but most of the time text feels more appropriate.”

What may be most impressive about Jarvis’ ability to receive texts is the fact Zuckerberg built a dedicated Messenger bot that allows him to chat with the system no matter where he’s actually located. Not only does this allow him to preemptively turn on his lights but it also provides a supremely effective way to keep up with what’s going on at his house. For instance, with the inclusion of an advanced facial- and speech-recognition feature, Jarvis keeps Zuckerberg in the loop should someone stroll up to his house — known or unknown. Furthermore, if the system recognizes the person, it automatically lets them in — after taking their photo and adding it to the Messenger conversation, of course.

Though the system isn’t perfect — during an exclusive Fast Company demo, it took Zuckerberg four tries to turn off his lights — he says he’s committed to staying focused on continuously improving Jarvis since he uses it every day. Be it the addition of more voice terminals or the creation of an Android app (currently, Zuckerberg uses an iPhone to control Jarvis), there’s still plenty to be accomplished. However, from the looks of the schematic Zuckerberg published, it appears as though his system is essentially perfect. How so? Because he’s programmed Jarvis to be compatible with a T-shirt cannon that fires a new gray tee to Zuckerberg when he asks for one. If you ask us, that’s peak artificial intelligence.

Emerging Tech

Awesome Tech You Can’t Buy Yet: Hi-viz bike reflectors and a tiny flashlight

Check out our roundup of the best new crowdfunding projects and product announcements that hit the web this week. You may not be able to buy this stuff yet, but it sure is fun to gawk!
Smart Home

What exactly is Alexa? Where does she come from? And how does she work?

While "Alexa" has become synonymous with products like the Amazon Echo, you can't actually go out and buy an "Alexa." So what is Alexa? How does she work? Here's everything you need to know about Amazon's virtual assistant.
Product Review

The Division 2 feels more like a revision, but fans will love it

If the private beta is any indication, The Division 2 is shaping up to be a better version of the original, rather than a brand-new experience. That said, the game is looking quite great.
Music

The 25 best love songs on Spotify for your wooing pleasure

Valentine's Day comes but once a year, but love is forever. If you're hoping to impress your significant other -- or if you're trying to procure one -- we put together a list of the best love songs for your wooing pleasure.
Smart Home

Google Home and Amazon Alexa are asking smart home device makers for user info

Google and Amazon want to establish a "continuous flow" of information between their servers and your smart home devices, but companies like Logitech have begun to speak out for user privacy.
Smart Home

Language barrier? Psh. Here's how to make your Google Home an ace translator

You can now use interpreter mode on your Google Home devices. This means, you can use your Google Home device to translate conversations in real-time. Here's how to use interpreter mode.
Product Review

From tables to porches, Ring's Stick Up Cam watches wherever you want

Ring’s latest home security camera offers more versatility than ever before, with the ability to mount it, set it on a table, or put it inside or outside. We got a chance to test it out. Here’s how it went.
Smart Home

Ring adds color night vision to wired security devices and HDR to wireless ones

Ring rolled out color night vision to its wired security products with cameras such as the Spotlight Cam Wired. At the same time, Ring added HDR control to its battery-powered security devices including the Video Doorbell 2.
Mobile

These 13 gadgets walk a fine line between ingenious and insane

The annual avalanche of devices and gadgets is astounding, but how many will succeed? A few are destined to spark new trends, while the majority fade deservedly into obscurity. We look at some gadgets on the border of brilliant and bonkers.
Smart Home

Check out the largest pools in the world - and how you can get there

From Egypt to Singapore to California, here's a pictorial breakdown of some of the biggest and largest pools in the world. Many of these pools are so big that you can even kayak and windsurf on them.
Smart Home

Consumer groups call out retailers in a bid for better IoT security

Consumer groups posted a "Dear retailer" letter on February 12 that called out Walmart, Best Buy, Amazon, and Target, shaming the companies for selling insecure smart home devices.
Deals

The best Presidents’ Day sales 2019: Amazon, Walmart, Dell, and more

Presidents' Day sales are a great chance to score electronics, clothing, home and office stuff, and other goodies at a discount. We’ve smoked out a large handful of the best of these Presidents' Day deals, from tech to bedding, to help…
Deals

Walmart drops a killer Instant Pot deal for Presidents’ Day weekend

With the Walmart Presidents' Day sale going on right now, you can actually pick up the Instant Pot Lux60 6-quart programmable pressure cooker for just $69 -- $30 off its normal price.
Smart Home

Walmart slashes prices on Google Home during its Presidents’ Day sale

Walmart slashes prices on Google Home during its Presidents' Day sale. Whether you're taking your first step in configuring a smart home or expanding to every room, Walmart has excellent sale prices for all the key components.