Skip to main content

IBM’s Holodeck-style classroom tech makes language-learning apps look primitive

How AI helps students learn Mandarin at RPI

Whether it’s apps like Duolingo or the ease of travel, there are plenty of ways technology has made it more straightforward to learn a second (or third or fourth …) language. Now, IBM Research and New York’s Rensselaer Polytechnic Institute have come up with an entirely new high-tech approach — and it totally reminds us of the Vulcan school from 2009’s Star Trek movie.

Called the Cognitive Immersive Room (CIR), it pairs an A.I.-powered chatbot smart assistant with a 360-degree panoramic display system to place users into a variety of immersive locations to try out their language skills. Currently, it’s being used for Mandarin, which is widely considered to be among the more difficult languages for Westerners to learn. The CIR setup drops students into scenarios like a restaurant in China and a tai chi class, where they can put their Mandarin to the test.

“The Cognitive Immersive Classroom is a very important use case for us,” Hui Su, Director of the Cognitive and Immersive Systems Lab at IBM Research, told Digital Trends. “We are currently focusing on language learning, and are building the classroom for students who study Mandarin as a second language. In this classroom, students are immersed into a 360-degree [environment], surrounded by real-life scenes such as a restaurant, street market, and garden. They can talk to the avatars, such as the waiters and salesmen on the screens, and perform tasks like ordering drinks and food and buying products. These tasks are developed as games for students to complete. In these games, the students will be able to practice and learn Chinese languages in a culture-rich environment, and talk to the A.I. agents who can understand what they just said.”

Image used with permission by copyright holder

To make the experience even more immersive, the room is kitted out with cameras, Kinect devices, and microphones. This makes it possible to, for instance, point at an object and say “what is that?” and have the question answered. The microphones, meanwhile, can pick up on every nuance of a speaker’s words.

“A pitch contour analysis function was developed to visualize the difference between the students’ pronunciations and those by native speakers, so that students could see easily where their tone pronunciations need to be improved,” Su said. “The classroom is a brand-new initiative to integrate A.I. technologies and human-scale immersive technologies together for Mandarin teaching. We are using IBM Watson speech recognition and natural language understanding for English and Chinese.”

Coming soon to a classroom near you. Or, at least, we really, really hope it is.

Luke Dormehl
Former Digital Trends Contributor
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
IBM’s new battery may outperform lithium-ion, and it doesn’t use conflict minerals
ibm new battery technology heavy metal 49214099772 6c9cf817a2 k

Recently, an international advocacy group accused tech giants including Apple, Google, Microsoft, Dell and Tesla of “knowingly benefiting from” the exploitation of young kids to mine cobalt in the Democratic Republic of Congo. Why cobalt? Because cobalt is a major component in the lithium-ion batteries that are found in nearly every rechargeable electronic device. Approximately two-thirds of the world’s cobalt supplies come from Congo.

The case underscores a massive problem: Batteries are a necessary component in a growing number of products around the world. But many of the heavy materials they are made from, such as nickel and cobalt, carry enormous environmental and humanitarian costs.

Read more
Researchers use A.I. to make smiling pet pics — and it’s as creepy as it sounds
nvidia ganimal ai research smiling pet

Can’t get your dog or that tiger at the zoo to smile for your Instagram? A new artificially intelligent program developed by researchers from Nvidia can take the expression from one animal and put it on the photo of another animal. Called GANimal -- after generative adversarial networks, a type of A.I. -- the software allows users to upload an image of one animal to re-create the pet’s expression and pose on another animal.

GAN programs are designed to convert one image to look like another, but are typically focused on more narrow tasks like turning horses to zebras. GANimal, however, applies several different changes to the image, adjusting the expression, the position of the animal’s head, and in many cases, even the background, from the inspiration image onto the source image. Unlike most GANs, the program is designed to work with any animal.

Read more
The Apollo wearable is proven to help you sleep better (and it’s on sale)
Apollo wearable worn during sleep in bed.

This content was produced in partnership with Apollo Neuro.
Stress, anxiety, and insomnia are all concerning things that just about everyone struggles with at one time or another. Maybe you can sleep, fending off insomnia, but you lack quality sleep and don’t feel rested in the morning. Or, maybe when it’s time to kick back and relax, you just can’t find a way to do so. There are many solutions for these issues, some work, and others don’t, but one unlikely area of support can be found in a modern, smart wearable.

Medicine is the obvious choice, but not everyone prefers to go that route. There is an answer in modern technology or rather a modern wearable device. One such device is the Apollo wearable, which improves sleep and stress relief via touch therapy. According to Apollo Neuro, the company behind the device, which is worn on your ankle, wrist or clipped to your clothing, it sends out waves of vibrations to help your body relax and reduce feelings of stress. It's an interesting new approach to a common problem that has typically been resolved via medicine, therapy, or other more invasive and time-consuming techniques. The way it utilizes those vibrations, uniquely placed and administered, to create a sense of peace, makes us ask, can it really cure what ails us? We’ll dig a little deeper into how it achieves what it does and what methods it’s using to make you feel better.

Read more