Learn Languages VR: Our first take

Bonjour, mon ami! Mondly uses VR to teach new languages

What if you could have a conversation with someone speaking in a different language, without being spoon fed individual words, whenever and wherever you want? This has long been the holy grail of language, and software has chased it for decades — with marginal success.

Yet there’s new hope to be found in virtual reaility, and Learn Languages VR by Mondly, the company’s first virtual reality application, shows why. In Mondly VR, you don’t interact with an instructor. Instead, you have conversations with a chat bot represented by a variety of digital characters, which asks you questions in the language you have selected.

During your conversations you are prompted with common responses to what the character says, and every answer you give is transcribed on screen, so you can see where you messed up. If you answer correctly, a green check mark will hover over the transcription of what you just said. Answer wrong, and the character will repeat the question. Not many human teachers would have this level of patience, but the chatbot is ready to repeat as often as needed.

Learn Languages VR doesn’t stop at chatbots. People learn best through real-world practice, and the application emulates that. You can choose from three VR experiences that test you knowledge — ordering at a restaurant, meeting someone on a train, and checking into a hotel.

So, can you really learn a new language in virtual reality? We gave it a go.

Robot Learning

Mondly is based on a server-side chat bot, which enables the app to speak with you without an actual person feeding it answers. Popular language learning program Rosetta Stone only teaches you new languages in conversations if you schedule a video chat session with one of its tutors. Being able to hear a voice recite a question often times made it easier to understand the pronunciation and context of how to use suggested phrases. And if you are planning to vacation in Barcelona, knowing how to say “can I get a cab” rather than simply a “cab” will certainly help you fit in.

The Mondly experience leverages the immersive quality of VR by the amount of interactivity you can access with just voice and head movements. You stare at the character in your simulation and as you speak, which is a much more natural environment than looking at a book or screen. With just the slightest head movement, you can view and access a transcription of what the chatbot said, see answer suggestions, view English translations, start audio recitations in the selected language, and more.

To get an English translations of suggestions and the chatbot’s answer, you simply hover the cursor over the phrase, and the translation will appear below. Hovering over the speaker icon near any phrase will prompt a voice to recite it back in the selected language. This helps pick up the nuances of languages predicated on vocal inflections, such as Japanese. But, sometimes the app accepts an answer that correctly pronounces a part of the phrase with a different inflection, if the app feels you got the gist of the response across.

Can You Hear Me Now?

The most important part of Mondly is not how intelligent it is, but how well it can hear. Learn Languages VR relies on the illusion of conversing with a person, and that means constant coordination between audio and visual components in order to make you feel like you are somewhere you actually are not. If you are learning a new language in VR, any noticeable lag or miscommunication between you and the instructor means the illusion is gone, which destroys the experience.

“The toughest challenge was to make the speech interaction feel natural in VR.”

“The toughest challenge was to make the speech interaction feel natural in VR,” Mondly co-founder Alexandru Iliescu wrote in a press release for the app. The primacy of sound to the experience is paramount. Before each experience you are prompted to remain quiet as Mondly’s voice detection system calibrates the microphone to the room’s background noise in order to ignore unintended sounds. I used Mondly in a silent office space and in a communal area with constant chatter around, and noticed no difference in the voice recognition.

Even so, the software has some issues to work out. Mondly’s voice recognition is consistently perfect…as long as it’s asked to handle phrases just a few words long. While trying to ask for a beer at a virtual restaurant in French, I could see only parts of my response transcribed.

Seeing the app wasn’t working, We decided to slowly say the sentence word-by-word to see if I was jumbling the pronunciation. When we did, the app displayed different words than what I was speaking, consistently. I yelled “Bier” in French 4 times, and was met “Heir.” Eventually, we gave up.

Too Smart To Teach

Mondly teaching you how to speak a different language through conversations is a double-edged sword. It is a crash course on context, verb tenses, and pronunciation in a language you probably never spoke before. The learning curve can be steep, and the app tends to throw users into the deep end before teaching them to swim.

We found a lesson might start with a simple phrase, then follow up by asking us to recite an eight word sentence introducing ourselves. This seems to make the app most useful to those who can read the language they wish to learn, but don’t know how to form sentences. Those Spanish classes you took in high school could come in handy.

Mondly VR

Each experience yields different levels of difficulty, as well. We talked through a light conversation on the train in German without much trouble. Trying to reserve a room with a shower was considerably more complex. After numerous failed attempts at complete phrases, we would repeat each word of the phrase, one by one, to see which words were correct or not. The recognition issues already we mentioned earlier made this process even more arduous.

Mondly is not for beginners, and may not be developed enough for those who just want to brush up on their other languages. It does not necessarily teach you a new language as much as it helps you learn a new language. The average instructor would help you learn words — the building blocks of most languages — before jumping into full sentences.

In the end, Learn Languages VR by Mondly will frustrate you, but if you stick through it, you’ll learn a thing or two. The app is free to download in the Oculus Store for the Samsung Gear VR.


  • Wide variety of languages to choose from at launch
  • Can detect voice even in noisy enviornments
  • Intuitive controls


  • Voice detection accuracy needs work
  • Doesn’t provide language fundamentals
Smart Home

This just in: Alexa can now deliver the news like a professional newscaster

The Amazon Alexa team has given Alexa a newscaster voice that improves the way she delivers the news and reads Wikipedia articles, making the smart assistant easier to understand.

Turn to these apps to help you in your next hunt for a job

Looking for a job can be a stressful experience, but these days, a simple mobile app can help you to find and apply for jobs all over the country -- here are some of the best job search apps for iOS and Android.

We tried all the latest and greatest smartphones to find the best of 2019

Smartphones are perhaps the most important and personal piece of tech on the planet. That’s why it’s important to pick the best phone for your individual needs. Here are the best smartphones you can buy.
Virtual Reality

Think virtual reality is just for games? These awesome apps will change your mind

Virtual reality isn't all about gaming. Swim with turtles, paint in 3D, and immerse yourself in some unique experiences the platform has to offer with our curated list of the best VR apps.

Forget 8K, the Insta360 Titan records 11K that can still play back on smartphones

The Insta360 Titan is an 11K 360 camera -- and a 10K VR camera -- yet software allows the footage to be played back at full resolution, even on smartphones. The camera also uses the largest sensors in a standalone 360 camera yet.
Virtual Reality

With 4K resolution, Pico’s latest stand-alone VR headset takes care of business

Pico Interactive, a popular VR headset manufacturer in China, is adding to its G2 line of business-focused VR headsets with the G2 4K, which is capable of rendering visuals in higher resolutions.
Product Review

HTC puts eye-tracking tech in Vive Pro Eye to make next-gen VR hands-free

With the announcement of integrated eye tracking for the Vive Pro Eye, hands-free VR, less demanding high-quality experiences, and intuitive software, are all on the table.

HTC brings two new headsets, a VR browser, and ‘Netflix for VR’ to CES 2019

HTC Vive made several new announcements to CES 2019, including two new headsets, a new subscription service, and a new user interface that completely transforms how applications are launched.

Camera records real-life scenes to design virtual worlds using a single device

Creating virtual reality worlds may get a bit easier in 2019 -- the Axis is a camera module that records a 180-degree depth map, allowing designers to reconstruct the scene for virtual or augmented reality.

The Vive Pro Eye uses Tobii eye-tracking technology to make VR more lifelike

HTC revealed the Vive Pro Eye with eye-tracking support at its CES 2019 press conference on January 7. We now know that Tobii will be the company responsible for integrating the technology.

Oculus’ Quest is the headset that will make me (and you) a VR believer

Without excessive wires or complicated setup, and a price point that makes sense for tech that just isn't there yet, Oculus' upcoming standalone headset, the Quest, could bring VR mainstream.

Dive headfirst into the best experiences available now on the Oculus Rift

The Oculus Rift brought back virtual reality and put a modern twist to it. Grab your Touch Controllers, put on your VR headset, and jump into the fun with some of the best Oculus Rift games available now.

The best VR headsets at CES 2019 could bring the technology to the mainstream

While there weren't a ton of new VR headset on display at CES 2019, the ones we saw led us to believe that VR could have a real moment soon, both from a gaming and business standpoint.

These are the coolest virtual and augmented reality gadgets from CES 2019

CES 2019 had plenty of VR and AR gadgets on display, including headsets that completely change how you experience virtual reality, and some that don't even require a PC or a phone to run.