'Emotion Recognition' Technology Can Read Your Feelings Through Your Face

This exciting new tech has applications for health, shopping and more.

By Michael Murphy on August 25, 2015

In the future, your mobile device will become more than just a tool for making phone calls, texting, using apps and emailing. What you hold in the palm of your hand will deliver a more personalized experience, and part of that may involve emotion recognition technology. This is currently being used in certain sectors, most prominently the field of medicine. Or more specifically: telemedicine.

One major player in the field is a company called Affectiva, which uses emotion recognition technology to evaluate your facial cues through your laptop, tablet, watch or smartphone. In short, your device will be able to know how you feel.

The company’s chief strategy and science officer Rana el Kaliouby, discussed the technology at a recent TED Talk, explaining her case for a more emotionally developed and personalized digital world, noting that “our emotions influence every aspect of our lives, from our health and how we learn to how we do business and make decisions…Today’s technology has lots of cognitive intelligence but no emotional intelligence.”

She believes adding the latter to the mix will yield more positives than negatives, and even joked that in the future an app could sense if you’re stressed and “lock your refrigerator.” Hershey, on the other hand, has been testing out the technology to get consumers down the chocolate aisle. They directed shoppers towards an Affectiva-powered kiosk that was able to tell if the customer was smiling — for a smile, the customer got a chocolate.

As for how thorough it is, Affectiva says the algorithm can sense even the slightest smile, and, according to MedCity News, “generates an emotion based on 12 billion emotion data points from 2.9 million face videos from 75 countries.”

In the world of medicine, Dr. James Giordano, professor of Neurology at Georgetown University’s Medical Center, believes that emotion recognition technology can be helpful with diagnosing and treating depression. One way to do this couples that technology with deep brain stimulation (DBS) devices that can treat certain types of severe depression which are resistant to other interventions.

Treating depression via DBS involves “neurosurgically placing very fine electrodes into brain regions that are involved in mood,” says Dr. Giordano. From there, he says, an electrical pulse “modulates the activity of these brain networks to reduce the symptoms and signs of depression.”

Paired with emotion recognition apps, a reflection of your mood can be documented through assessing your “tone and inflection of voice, eye movement and pupil responses, and very small facial muscle movements.” Dr. Giordano adds that these indicators can be noticed “before more overt symptoms and signs — such as sadness and lethargy — worsen.”

Once these biomarkers (such as a slight frown) are evaluated against a database of information, they can indicate one’s mood, or a shift in one’s mood state.

“The elegance of these types of systems is that they pair computational technology and neurotechnology to enable considerable personalization, independence, adaptability and flexibility in treating severe forms of depression,” Dr. Giordano notes.

Back at home, in our daily lives, this technology will allow us to hold our emotions in our hands, and therefore make them easier to track. Most notably, it will allow the user to monitor the way they think, feel, and relate to others, and hopefully allow us to better understand our emotional states.