Skip to main content

IBM’s Holodeck-style classroom tech makes language-learning apps look primitive

How AI helps students learn Mandarin at RPI

Whether it’s apps like Duolingo or the ease of travel, there are plenty of ways technology has made it more straightforward to learn a second (or third or fourth …) language. Now, IBM Research and New York’s Rensselaer Polytechnic Institute have come up with an entirely new high-tech approach — and it totally reminds us of the Vulcan school from 2009’s Star Trek movie.

Called the Cognitive Immersive Room (CIR), it pairs an A.I.-powered chatbot smart assistant with a 360-degree panoramic display system to place users into a variety of immersive locations to try out their language skills. Currently, it’s being used for Mandarin, which is widely considered to be among the more difficult languages for Westerners to learn. The CIR setup drops students into scenarios like a restaurant in China and a tai chi class, where they can put their Mandarin to the test.

“The Cognitive Immersive Classroom is a very important use case for us,” Hui Su, Director of the Cognitive and Immersive Systems Lab at IBM Research, told Digital Trends. “We are currently focusing on language learning, and are building the classroom for students who study Mandarin as a second language. In this classroom, students are immersed into a 360-degree [environment], surrounded by real-life scenes such as a restaurant, street market, and garden. They can talk to the avatars, such as the waiters and salesmen on the screens, and perform tasks like ordering drinks and food and buying products. These tasks are developed as games for students to complete. In these games, the students will be able to practice and learn Chinese languages in a culture-rich environment, and talk to the A.I. agents who can understand what they just said.”

Image used with permission by copyright holder

To make the experience even more immersive, the room is kitted out with cameras, Kinect devices, and microphones. This makes it possible to, for instance, point at an object and say “what is that?” and have the question answered. The microphones, meanwhile, can pick up on every nuance of a speaker’s words.

“A pitch contour analysis function was developed to visualize the difference between the students’ pronunciations and those by native speakers, so that students could see easily where their tone pronunciations need to be improved,” Su said. “The classroom is a brand-new initiative to integrate A.I. technologies and human-scale immersive technologies together for Mandarin teaching. We are using IBM Watson speech recognition and natural language understanding for English and Chinese.”

Coming soon to a classroom near you. Or, at least, we really, really hope it is.

Editors' Recommendations

Luke Dormehl
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
IBM’s new battery may outperform lithium-ion, and it doesn’t use conflict minerals
ibm new battery technology heavy metal 49214099772 6c9cf817a2 k

Recently, an international advocacy group accused tech giants including Apple, Google, Microsoft, Dell and Tesla of “knowingly benefiting from” the exploitation of young kids to mine cobalt in the Democratic Republic of Congo. Why cobalt? Because cobalt is a major component in the lithium-ion batteries that are found in nearly every rechargeable electronic device. Approximately two-thirds of the world’s cobalt supplies come from Congo.

The case underscores a massive problem: Batteries are a necessary component in a growing number of products around the world. But many of the heavy materials they are made from, such as nickel and cobalt, carry enormous environmental and humanitarian costs.

Read more
Researchers use A.I. to make smiling pet pics — and it’s as creepy as it sounds
nvidia ganimal ai research smiling pet

Can’t get your dog or that tiger at the zoo to smile for your Instagram? A new artificially intelligent program developed by researchers from Nvidia can take the expression from one animal and put it on the photo of another animal. Called GANimal -- after generative adversarial networks, a type of A.I. -- the software allows users to upload an image of one animal to re-create the pet’s expression and pose on another animal.

GAN programs are designed to convert one image to look like another, but are typically focused on more narrow tasks like turning horses to zebras. GANimal, however, applies several different changes to the image, adjusting the expression, the position of the animal’s head, and in many cases, even the background, from the inspiration image onto the source image. Unlike most GANs, the program is designed to work with any animal.

Read more
IBM Research is using A.I. algorithms to unlock the secrets of dark matter DNA
sophia genetics ai dna stock

You may have heard of dark matter, a mysterious form of lesser-studied matter that is thought to be composed of some as-yet undiscovered subatomic particles, but which makes up an astonishing 85% of the matter in the universe. But how about dark matter DNA? These unexplored molecules and matter surrounding our genes make up more than half of the human genome -- but are a total conundrum in terms of what they encode and, more importantly, affect.

The good folks at IBM and the Munich Leukemia Laboratory think they can help come up with some answers -- and they’ve used some groundbreaking A.I. algorithms to help.

Read more