Decoding Care

The Science Behind the Face

by Hendrix Magley

IN THE AGE of telehealth and virtual doctor visits, nonverbal communication between patients and clients is more important than ever. But whether you're meeting for a face-to-face appointment or sitting down for a visit in the comfort of your own home, a healthcare professional who displays a friendly expression can play a pivotal role in a patient's health.

Dr. Kristi Hape '99, Assistant Professor of Occupational Therapy, knows the effect empathy has on patients, but when she learned the Romain College of Business had a Biometrix Discovery Lab that could help her students understand how their facial expressions impacted others, she wanted to bring the technology into her curriculum. "We are always hoping to further develop students' ability to provide words of encouragement to their clients by allowing them to see the impact emotionally in a positive way."

Hape connected with Dr. Chad Milewicz, Professor of Marketing and Chair of USI's Economics and Marketing Department and coordinator of the Biometrix Discovery Lab, to begin discussions on how the lab's technology can enhance OT students' education.

Using new and emerging technologies, the Biometrix Discovery Lab can be used to conduct research on decision making by measuring human attention, arousal, cognition and emotion. Through artificial intelligence (AI) and facial expression analysis, OT students can instantly understand how their expressions impact a patient positively or negatively. The data helps ensure they show empathy and care during interactions with patients. For students seeking to provide the best care possible when they enter their career fields, understanding and managing this soft skill can be transformative.

The way it works is, students complete a "Therapeutic Use of Self" interview, designed to create a meaningful relationship between the therapist and patient, as part of their Occupation Centered Practice Course—the pilot use of the facial expression analysis took place in March of 2024. Students had a second opportunity to use the facial expression analysis as part of a telehealth appointment. After completing a virtual 10-minute interview with a standardized patient actor known as "Betty" in her “home” (the USI Minka Learning Lab for Living Well), students met with their instructor for one-on-one feedback. Visual analysis of their facial expressions was provided using seven different emotions—anger, contempt, disgust, fear, joy, sadness and surprise. Students could see in realtime what emotions they're displaying and how it impacted Betty's emotional responses.

For example, the biometric analysis measured joy from both a student and patient after the student provided praise to a patient's resilience after experiencing a stroke. It's also allowed for corrective feedback such as how the facial expressions students use when thinking between questions could be perceived as negative. Once students recognize this and soften their expression, they are less likely to trigger a negative response from their clients.

"As we continue to use this process of biometric analysis for student feedback, we are hoping to see students set personal goals about the amount of joy they can elicit from their client," said Hape.

From students using the biometric technology in a variety of classes to businesses learning how these tools can positively impact their companies, Milewicz has seen firsthand how these skills have proven helpful and can even give students an upper hand as they move into their career field. "Exposure to this technology increases creativity, sparks interest in research and analytical problem solving and raises awareness of human behavior," Milewicz said. "I hope this collaboration inspires others to consider ways they might benefit from these outstanding resources."

While presenting at the American Occupational Therapy Association 2024 Education Summit, Hape and Dr. Jessica Mason '05 M'06, Chair of the USI Occupational Therapy Program, mentioned how rare a partnership of this nature is. But it's something Milewicz and the faculty in the Biometrix Discovery Lab want to make sure their peers across campus know exists. "This software gives students the opportunity to gain valuable insight into nonverbal communication and emotional responses that can be utilized to enhance interpersonal skills in personal and professional settings," said Meshea Greenwell, a graduate student in the Occupational Therapy Program.

It's not common for undergraduate students to have readily available access to technology of this nature. Unique opportunities such as this provide students, regardless of their major, the chance to be ahead of the curve by learning emotional analysis and other skills. But perhaps most importantly, at least in this scenario, it ensures a patient will receive comfort and care–even during some of their toughest times.

"I find that exposure to this type of emotional analysis has a long-term impact because it raises a person's awareness of emotions expressed in different situations," Milewicz said. "Not only does one learn from how they responded in a given assignment, but they can learn how to understand expressions of emotions in the future."