The pang of loneliness is far higher than even the gloomiest of previous estimates, according to a new study from researchers at the University of California, San Diego.

Three-fourths of Americans experience moderate to high levels of loneliness, said the study, published this week in International Psychogeriatrics. Previous studies found loneliness rates of 17 percent to 57 percent. Men and women were equally affected.

While the numbers show a larger percentage of the population experiences loneliness, the study is also the first to provide clues as to how people might be able to master loneliness, by developing a better understanding of others, and of themselves.

Loneliness is distress caused by a lack of satisfactory relationships, not being alone, said study leader Dilip Jeste. A hermit may not feel lonely. Conversely, you can be lonely even when surrounded by people.

Nothing beats the holidays for evoking that alone-in-a-crowd feeling, Jeste said. The happy-looking faces, music, colors, parties and celebrations deepen the glass-bubble isolation from the joy others are having.

"You may be going through the motions, but you really don't feel connected to any of them," Jeste said. "You don't feel you are a part of a group of friends, and not feeling close to anyone."

Loneliness also follows people throughout their lives, the study found. Rates are especially high for people in their late 20s, mid-50s and late 80s.

The pain is more than psychological. Loneliness is associated with physical illness. A study released in 2017 by insurance giant Cigna found that loneliness is as harmful to one's health as smoking 15 cigarettes a day.

The good news is that the UCSD study found an antidote to loneliness: wisdom.

Wisdom is a personality trait similar to resilience or optimism, Jeste said. It allows you to roll with life's punches, learn from your mistakes and develop empathy for others.

"Wisdom and loneliness don't seem to coexist," Jeste said.

The study examined 340 adults between the ages of 27 and 101 living in San Diego County. These adults had taken part in one of three earlier studies on aging and mental health.

Jeste said the results are likely to apply to the entire U.S. population, with a few limitations. Study participants were mainly Caucasian. Hispanics and African-Americans were underrepresented, especially the latter.

The study excluded those with dementia, those in nursing homes or needing living assistance.

"The people that we studied were not in a group at high risk for loneliness," Jeste said. "These are not people with depression or some mental illness or even cancer and heart disease."

The three peak periods of loneliness found in the study are all understandable because of what people go through at those stages of life, Jeste said.

In one's late 20s, choices are commonly made about jobs, choosing a life partner and where to settle down, he said.

"And at that stage you also feel that you are not doing as well as your peers," Jeste said. "Say you'll compare yourself with some people who are doing better, they may have more money or they look happier, whatever. So that becomes a period of high stress."

The mid-50s represents midlife crisis.

"That's the period of menopause in women and andropause in men. But it's also a time when people first start getting some physical disability or illness, for example, arthritis or back pain. Blood pressure may start going up, diabetes may be diagnosed. So that's the time where people become aware of their mortality."

And in the late 80s, the awareness of illness and mortality becomes pervasive.

"For people who are lucky to have survived to 80, the risk of dementia is highest," Jeste said. "And most people would have physical illnesses. And serious pain is a problem. They may have no family, no other friends left. So it's clearly a period of considerable stress, depression and loneliness."

The loneliness assessment featured questions about participants' feelings and relationships. This method is more accurate than just directly asking people if they feel lonely, Jeste said.

"We found that if we asked people if they feel lonely, fewer than 10 percent say they are lonely, and it is much more common than that," Jeste said.

Even so, Jeste said he and colleagues were surprised at the frequency of loneliness.

Wisdom was assessed with an exam recently developed at UC San Diego called the San Diego Wisdom Score. It defined wisdom as having six components: control over emotions; compassion; self-reflection; acceptance of uncertainty and diversity of views; decisiveness; and spirituality.

Eti Ben Simon, a UC Berkeley researcher who has also studied loneliness, said the study, released Tuesday, provided useful new information.

"Overall, the study is well-executed and offers the novel factor of wisdom as a protective factor against loneliness," she said by email.

"My only concern is that wisdom is somewhat loosely defined in the study, including features such as emotional regulation, pro-social behaviors, tolerance of divergent values and spirituality.

"Other studies have already shown that emotion regulation and pro-social behavior are protective factors against loneliness, so in that sense wisdom might be overlapping with factors we already know to be effective," Ben Simon said.

The study would have been stronger had it asked about sleep habits, said Ben Simon, whose main research area is sleep and emotional function. She led a study published in August that focused on the sleep component of loneliness.

Sleep was mentioned in the UC San Diego study as being associated with loneliness, along with depression, substance abuse and cognitive impairment.

Further research into the biological mechanisms involved is needed to explore how these conditions are related, Jeste said. That would be helped by developing technologies to monitor physiological states and how they change.

"We have a collaboration with IBM on artificial intelligence for healthy living," he said. "So we are hoping that in the next few years, we'll find some important markers of loneliness that might help us detect the severity of loneliness or maybe to predict when loneliness might occur."

For now, Jeste offered comfort for those struggling with the difficult holiday period. According to his and many other studies, those who appear happy at parties or gatherings may be just putting on a brave front.

Far from being alone amid the joyous faces and bright lights of the holidays, the lonely may actually be in the majority, he said.

Explore further Serious loneliness spans the adult lifespan but there is a silver lining

More information: Ellen E. Lee et al, High prevalence and adverse health effects of loneliness in community-dwelling adults across the lifespan: role of wisdom as a protective factor, International Psychogeriatrics (2018). Ellen E. Lee et al, High prevalence and adverse health effects of loneliness in community-dwelling adults across the lifespan: role of wisdom as a protective factor,(2018). DOI: 10.1017/S1041610218002120

©2018 The San Diego Union-Tribune

Distributed by Tribune Content Agency, LLC.