Skip to main content

Kinect-powered virtual therapists can fully decode a patient’s body language

simsensei kinect virtual therapist
Image used with permission by copyright holder

At a glance, the name SimSensei sounds like something that could virtually teach you to be the next Ninja master. Alas, that dream is too far-fetched; SimSensei is actually a Kinect-powered virtual therapist that uses the motion recognition technology to read a patient’s body language, thus determining underlying anxiety, nervousness, happiness, or contemplation.

Developed by a team at the University of Southern California’s Institute for Creative Technologies, the SimSensei program is still in its early stages. The beta preview uses a virtual avatar that acts as your shrink, while the second half of the screen uses the Microsoft Kinect to detect changes in the patient’s face and body movements. SimSensei would only record the patient as he or she is verbally responding so the program could see how the patient physically reacts when they are answering questions. These body languages include patients leaning forward or backward on their chair, smiling, prolonged/lack of eye contact, and the directions in which the eyes move.

Recommended Videos

That’s not to say SimSensei is entirely robotic either. The virtual therapist is even programmed to “Hmm” at appropriate times as if to ponder a response, and guide the patient along their conversation depending on what he or she answers. The point is to make the program look and feel as natural as possible, but that’s also not to say SimSensei doesn’t sound kind of creepy. I may have not been to too many shrinks, but the sample session USC provided in the video below shows a conversation that lacks white noise, feels awkward, sounds a bit intimidating. We’re not sure what’s worse: Trusting a human stranger or a friendly…ish robot with your deepest anxieties.

SimSensei is part of the many programs being developed for the Association for Computing Machinery‘s contest. The competition would compare the programs to see which can most accurately diagnose a patient based on their body language, identifying the correct group of depressed patients from non-depressed ones. Hey, at least if you reveal all your deep, dark secrets to SimSensei or other virtual psychiatrists, you will never run the risk of bumping into them in the street or something equally embarrassing. Meanwhile, students studying psychology have another reason to go back to their dorm rooms and weep themselves to sleep tonight for having chosen a doomed career path. Good thing there’s a robot they can talk to that’s currently in development.

Natt Garun
Former Digital Trends Contributor
An avid gadgets and Internet culture enthusiast, Natt Garun spends her days bringing you the funniest, coolest, and strangest…
Hyundai Ioniq 5 sets world record for greatest altitude change
hyundai ioniq 5 world record altitude change mk02 detail kv

When the Guinness World Records (GWR) book was launched in 1955, the idea was to compile facts and figures that could finally settle often endless arguments in the U.K.’s many pubs.

It quickly evolved into a yearly compilation of world records, big and small, including last year's largest grilled cheese sandwich in the world.

Read more
Global EV sales expected to rise 30% in 2025, S&P Global says
ev sales up 30 percent 2025 byd sealion 7 1stbanner l

While trade wars, tariffs, and wavering subsidies are very much in the cards for the auto industry in 2025, global sales of electric vehicles (EVs) are still expected to rise substantially next year, according to S&P Global Mobility.

"2025 is shaping up to be ultra-challenging for the auto industry, as key regional demand factors limit demand potential and the new U.S. administration adds fresh uncertainty from day one," says Colin Couchman, executive director of global light vehicle forecasting for S&P Global Mobility.

Read more
Faraday Future could unveil lowest-priced EV yet at CES 2025
Faraday Future FF 91

Given existing tariffs and what’s in store from the Trump administration, you’d be forgiven for thinking the global race toward lower electric vehicle (EV) prices will not reach U.S. shores in 2025.

After all, Chinese manufacturers, who sell the least expensive EVs globally, have shelved plans to enter the U.S. market after 100% tariffs were imposed on China-made EVs in September.

Read more