As the weak economy lingers in the parts of the country hit hardest by the recession, residents are voting with their feet.

Nearly five years after the recession officially ended, dozens of U.S. cities still have unemployment rates in the double digits. According to new census data released Thursday, many of those places are bleeding population, while energy-rich areas where the recovery has been strongest are seeing significant gains.

Overall, the nation’s 381 metropolitan areas — anywhere with an urban core of at least 50,000 residents — added 2.3 million residents in 2013, a growth rate of a bit under 1 percent. Continuing the recent pattern, large cities grew faster than small ones; Houston added 138,000 residents, the biggest gain of any city, while migration from overseas continued to boost New York City’s population. The long-term decline in the rural population accelerated, with some 60 percent of rural counties losing population last year.

The census data doesn’t reveal who’s moving where, but it does distinguish between population change due to births and deaths — the so-called natural rate of change — and change due to people moving in or out. In the country as a whole, the natural rate accounted for most population growth. But in the cities that are growing and shrinking the fastest, migration is the dominant force: They’re growing or shrinking because people are moving in or out. Evidence suggests the economy is a big part of the reason why.

The trend is particularly stark in cities still struggling to emerge from the recent recession. Eighteen U.S. metro areas had unemployment rates of 12 percent or higher in 2012, according to the Bureau of Labor Statistics, and all but three of them saw a net decline in migration — that is, they saw more people move out than move in.

These cities are overwhelmingly in inland California, where the collapse of the housing bubble left deep and lasting scars. California accounts for 13 of the 18 cities with unemployment rates of 12 percent or above, and for 18 of the 56 with unemployment rates in the double digits. Those cities weren’t shrinking before the recession — they were actually growing a bit faster than average, though not by a significant margin — which suggests but does not prove that the recent decline is due to the weak labor market.

The exodus didn’t start right away. In the immediate aftermath of the recession, cities with particularly weak labor markets saw only modest outward migration. But as the jobs crisis has lingered, the trend has intensified, accelerating in each of the past three years.

Still, the overall pace of outward migration remains slow. The 18 cities with unemployment rates of at least 12 percent lost a total of just over 28,000 people to net migration, less than half of 1 percent of their total population. Perhaps that shouldn’t be surprising; moving is expensive, and finding a job in another part of the state, let alone across the country, is difficult. Moreover, the worst-hit areas tend to be places where the housing market crashed the hardest, leaving many homeowners underwater on their mortgages and unable to move easily.

The broader correlation between unemployment rates and population growth is quite weak. For metro areas with unemployment rates that aren’t extreme — between 5 and 10 percent, compared to the national mark of 8.1 percent in 2012 — there’s hardly any relationship at all. People don’t appear to leave town just because unemployment is a bit above the national average. But residents are abandoning communities that have experienced sustained high levels of joblessness.

Meanwhile, at the opposite end of the spectrum, cities in regions with strong job markets are gaining residents. Oil and gas states dominated the list of fastest-growing cities: Six of the top 10 were in Texas, North Dakota or Wyoming, where an oil and gas boom has brought unemployment rates below 5 percent. A separate census report on Wednesday showed just how powerful an economic engine the drilling bonanza has been. Employment in the mining sector grew 23.7 percent between 2007 and 2012, and total payroll by 51 percent, by far the fastest growth of any industry.

The energy boom isn’t going to solve the nation’s economic malaise, however. It simply isn’t big enough. Despite the large increase in percentage terms, the mining sector’s growth represented a gain of less than 200,000 jobs between 2007 and 2012. And while its rock-bottom unemployment rate and rising wages are attracting plenty of job-seekers, North Dakota isn’t going to make a dent in the national unemployment rate; the country’s 11.4 million unemployed workers represent 16 times the state’s total population.