According to Edelman, 63 percent of millennials in the 2019 survey indicated they were considering a move from sunny California. The chief reason for dissatisfaction: housing.

When asked in general about what would make them leave California, 60 percent of millennials in the survey gave housing cost and availability as the reason. That was slightly higher than the general population (55 percent), although 65 percent of renters cited housing factors as a reason to leave.

Californians believe housing costs are four times more threatening to the state's economy than high health costs. Residents also consider crime and security as a top-three concern.

Terrazas said millennials in California who are "tired of renting and looking to settle down and buy a home are finding it's often out of reach for them." He said this is especially the case in coastal job centers of the state, whether Los Angeles County or the San Francisco Bay Area.

"California just doesn't strike them as reasonable," the economist said. "The state has consistently seen much faster home value appreciation than most of the country, and the same goes for rent until about two years ago. Rents have begun to slow down, ... although they remain at high levels."

Terrazas said Southern California has high housing costs and on average lower incomes than Northern California. "In some ways, Southern California is in much more dire straits," he said.

Even with higher average incomes in Silicon Valley, though, he said homebuyers now must spend about half of their pretax incomes on a monthly mortgage for a median home. The median home in the Silicon Valley market topped $1.2 million at the end of 2018, according to Zillow data.

Statewide, the median home value in California was $547,400 at the end of 2018, while the U.S. median home value was $223,900. By comparison, the median home value in New York state stood at $289,000 and $681,500 in New York City; New Jersey was $324,700.

The Edelman survey found 47 percent of Californians are considering moving out of the state in the next five years. Again, it found the rates among millennials were higher with 55 percent of them contemplating the move. And 57 percent of Californians with kids under 18 also were considering packing up and leaving in the next five years.

Chapman's Kotkin believes the next wave of discontent in California won't necessarily be focused on housing costs but taxes.

"Taxes are a real killer if you're upper middle class and whether you're a younger person trying to buy a house or you just want to be able to spend what you make," said Kotkin. "There's also concern among people looking to retire and having their income taxed into oblivion."

At 12.3 percent, California led the 50 states in 2018 with the highest top marginal tax rate, according to the Federation of Tax Administrators. And that doesn't include an additional 1-percent surcharge for those Californians with incomes of $1 million or more.

"The tax bill made it worse," Kotkin said, pointing out that the federal tax changes mean deductions for state, local and property taxes now get capped at $10,000. "State taxes have become a significant factor now. We're getting into a situation where the middle class in California really can't hack it."

Overall, the Edelman survey involved a total of 1,900 California residents and was conducted Jan. 4 to Jan. 20. It said results were weighted to the Census to be representative of the state's adult population.

A report from California's Legislative Analyst's Office last year indicated Texas, Arizona, Oregon and Nevada are popular destinations for relocating Californians. It also found families with kids and those Californians with only a high school education were most likely to flee to lower cost states than college-educated residents.

Finally, the survey found more than 60 percent of residents feel that the best days of living in California are behind instead of ahead. And a large number of residents are "ambivalent" toward tech as an engine of prosperity, the survey said.