For many with autism, reading facial expressions is a struggle

Jeff Hudale sat in a darkened room at a University of Pittsburgh laboratory as one face after another appeared on a computer screen in front of him.

"Disgusted," he said. "Angry." Then, a long pause. "Neutral."

For someone with autism, Mr. Hudale was doing a pretty good job of identifying the emotional expressions on the photos, said Mark Strauss, an autism researcher at Pitt.

"On the obvious expressions, he did pretty well," Dr. Strauss said. "Where he was falling apart was when they were more subtle."

In particular, Mr. Hudale, 38, of Penn Hills, had trouble with faces that showed slight anger or disgust, and often called them neutral. "If you look at someone like Jeff who's pretty good at this, but he's not quite good enough, then you have to ask, what level of skill do you need so that people aren't thinking you're odd on this ability?"

Autism is a developmental disorder that is often marked by repetitive behaviors, language delays and obsessive interests.

But the key deficit, many experts feel, is an inability to engage in normal social interactions. People with autism often have a hard time figuring out what others are thinking and feeling, and they struggle with reading facial expressions.

PG graphic Identifying emotions and motives

To Dr. Strauss, subtlety is the key to understanding what is going on when most of us interpret other people's expressions, and is the critical stumbling block for people with autism.

It's also a shortcoming that Mr. Hudale, who has been diagnosed with a high-functioning form of autism known as Asperger syndrome, recognizes himself.

If people are grinning broadly or are shocked, he can recognize that pretty easily, he said in an interview, but "when I am in a regular conversation, even if someone might seem like they're in a bad mood, they can put on that expression where they try to hide it and they do such a good job of hiding it I wouldn't even notice it."

Reading expressions, he said, "can be kind of tricky, especially if they give you that poker face."

Previous research on autism and emotional expressions often used photographs and let people with autism gaze at the pictures until they could make a guess at the expression, Dr. Strauss said.

But in real life, emotions on the face come and go in an instant.

To recreate that, Dr. Strauss and his colleagues recently completed the first study that tested the ability of high-functioning autistic people to interpret dynamic video clips of expressions, where faces would go from a neutral expression to happiness, anger, surprise, fear, sadness or disgust in a third of a second.

The people with autism and a control group of typically developed people were shown videos that stopped at 25 percent, 50 percent, 75 percent and 100 percent of full expressions.

At the 25 percent level, almost no one could figure out what the full expression was going to be.

At the full expression level, the autistic people only lagged behind typical people by a modest amount on correctly identifying the expressions.

But in between, at the 50 percent and 75 percent levels, the people with autism were markedly deficient. At a 75-percent level of expression, for instance, 70 percent of the typical people could identify the emotion, but only 30 percent of the autistic people could.

"The reason this is so critically important," he said, "is that when you get to this 75-percent level, where there is subtlety to the expression, and virtually everyone who is typical can identify it but you as someone with autism can't, now you're really at a disadvantage, because this is what is happening when people are in a conversation and they're making facial expressions and everyone else is picking up on them and you're not."

Dr. Strauss believes one reason people with autism struggle with this task is that they have never developed mental prototypes of the various expressions.

"What we think is going on for most people is that to get better at recognizing expressions, you have to learn what a prototypical expression is. You have to know everything that goes into a typical anger expression vs. a typical surprise or disgust expression, so you can compare one prototype to another."

Lacking these templates, autistic people appear to analyze different facial features to try to figure out what people are feeling.

While people with autism are worse as a group at reading faces than typically developed people are, there is a wide variation among them.

That showed up in one recent study done by Suzy Scherf of Carnegie Mellon University's Center for the Neural Basis of Cognition.

Dr. Scherf and her team did brain imaging of autistic and typical adolescents as they looked at video clips of faces, places and objects.

The autistic youths had the same brain patterns as their counterparts when they looked at places and objects, she said in a recent interview, but they had a different signature of brain activity when it came to faces.

Of the 10 youths with autism who were tested, only three had activity in a typical face-processing location known as the fusiform face area. The others tended to analyze faces in a part of the brain normally used for objects, she said.

"We found there was a lot of variability in individuals with autism in how they perceived faces, even moreso than in typically developing individuals," Dr. Scherf said.

Even when people with autism can identify an emotion, there is still a remaining puzzle: Can they understand what the other person is feeling?

That is a question that child psychiatry professor Kevin Pelphrey is trying to answer at Yale University.

Dr. Pelphrey has done brain imaging studies on children with autism to see how good they are at judging the intentions and motives of others.

The answer? Not very.

In one study, he had children with autism play a "virtual ball toss" game while their brains were being scanned. In the game, the child would act out tossing a ball to computerized playmates, who would then suddenly only toss the ball to each other.

When the playmates stopped sharing, the autistic children showed activation in the parts of their brains that sense physical and emotional pain, he said in a recent interview, so "they were acutely aware of being excluded."

But they did not show much activity in the parts of the brain that are used to figure out what other people are trying to do.

Dr. Pelphrey has seen the same pattern when children with autism view pictures that are frightening or disgusting.

They do pretty well at sensing their own reactions to the pictures, he said. "In other words, they have a normal brain system for understanding how they feel, but then when you say 'How do the people in the picture feel?', those brain regions are disrupted."

The key parts of the brain that Dr. Pelphrey and other researchers have identified for figuring out others' motives are the superior temporal sulcus, a horizontal groove in the part of the brain above the ears, and the temperoparietal junction, the intersection of the temporal and parietal lobes at the sides and top of the brain.

Both have strong links to the amygdala, a key emotional processing area in the brain. The sulcus, he said, seems to be an older brain system that is "implicitly involved in watching other people and using eye gaze and posture to figure out how they're feeling, what they're intending to do, and monitoring what they're interested in."

The temperoparietal junction is a more analytical area that can draw conclusions about what people are thinking. "That's the brain area that says, 'Well, John knows this information, and Sarah knows that information, and when they get together they're going to disagree about something because they have different pieces of knowledge.' "

In children with autism, Dr. Pelphrey believes these areas never develop normally, and one key place that shows up is a difficulty in reading facial expressions.

That emotional face blindness interferes with all the processes children normally use to learn how to be social creatures, he said.

"Our earliest social experiences are when we're babies and we're crying or are upset, and we use those to engage other people, and then when we see them smiling, we react to that by looking cute, and that is the very beginning of emotional regulation.

"Later, you look at your parents's faces to help you interpret a situation, and then even later, your parents will verbally communicate how you ought to behave, and you will also learn about that from talking with your friends."

"If you're missing the tools [in the brain] to help you take part in that social fabric, you will miss a lot of those emotional regulation strategies," he said.

Mr. Hudale can understand that uphill battle for people with autism.

For him, figuring out what people feel is "an ongoing process, like life itself. I think I do have to work harder to detect emotions, and I have to think, 'How do I not offend somebody?' "

First published on September 27, 2010 at 12:00 am