While the astronomical (and rising) cost of college seems to garner all the media attention, it’s the cost of child care that should be more worrying for many parents.

Nearly 11 million children under the age of five require child care at least weekly and spend an average of 36 hours a week in child care — and for the parents footing the bill, this can make a serious dent in their finances.

In roughly half the states in this country, child care costs more than college, according to a report released this year by the Economic Policy Institute, a nonprofit, nonpartisan think tank focused on conducting research around the economic status of working Americans. Indeed, in 24 states and Washington, D.C., the average cost to care for a four-year-old for one year exceeds the cost of tuition for a year at a public, in-state four-year college. And in 33 states and Washington, D.C., the average cost to care for an infant for a year exceeds the cost of tuition for a year at a public, in-state four-year college.

Annual child care costs for a four-year-old range from $4,128 in rural South Carolina to $17,664 in Washington, D.C. The average annual cost of tuition for an in-state full-time undergraduate student in a public college ranges from $3,756 in Wyoming to $14,469 in New Hampshire.

The cost of child care, as well as stagnant wage growth (average hourly earnings have risen just over 2% in the past year, which EPI notes is “in line with the same slow growth we’ve seen for the last six years), means that average families — as well as more affluent families who live in a city where care is very expensive — struggle to pay, says Elise Gould, a senior economist at EPI. The affordability threshold for child care is 10% of one’s income, according to the Department of Health and Human Services, and yet only in a “handful” of areas do child care costs come even close to that threshold, EPI notes. “This standard cannot be met by most families,” Gould says.

Indeed, in a two-parent household with an infant and a four-year-old, child care ranges from 19.3% to 28.7% of total family budgets. This compares with a range of 11.8% to 21.6% for families with a 4-year-old and an 8-year-old, thanks to the fact that infant care tends to be the priciest form of child care, EPI data reveals.

“The high cost of child care can be a crippling burden for families with young children,” according to the latest child care costs report by ChildCare Aware of America, a group that researches the quality and availability of child care.

It might seem baffling that child care is so expensive given the low pay of child care workers (the average child care worker makes only about $10 an hour, according to the Bureau of Labor Statistics). But this high cost is thanks in large part to the fact that child care (especially high quality child care) is “a labor-intensive industry, requiring a low student-to-teacher ratio,” the EPI report notes. Other large expenses for child care centers are rent/mortgage, food and insurance (business, liability, real estate and workers’ compensation), ChildCare Aware of America notes, and lesser costs include security, staff training, toys and supplies, and utilities.

Ideally, parents will start saving for child care even before they become parents. “Five years [ahead of the birth of a child] is ideal but tough to realistically plan that far ahead when the average couple isn’t married that long prior to having kids,” says mom-to-be Christina Lindsey Orta, a certified financial planner and vice president at Lindsey & Lindsey in Westlake Village, Calif.

If you’re one of the many parents who hasn’t yet saved for child care, there are a variety of ways to find cash. Orta says you should consider a mortgage refinance to yield some extra monthly cash. “As home values have appreciated and interest rates are low, often times a refinance of a mortgage may make sense,” she says.

You may also want to consolidate credit cards to cards with lower rates (look for cards you’re paying more than 12% on) and look at where you can make cuts in your budget, she adds. “Look at your budget and identify where the small things add up, such as Starbucks lattes every day at $6 a pop, add a muffin to that and you’re at $10,” she says.