For many black and Hispanic families in the United States, achieving financial security remains frustratingly out of reach.

A big reason for that is that blacks and Hispanics have far less wealth to start with than their white counterparts -- a concept known as the racial wealth gap.

Wealth, or net worth, is the value of assets including your home, retirement savings and income minus the debt owed against those assets. According to federal data, the median wealth for white families in 2013 was around $141,900, compared to Hispanics at about $13,700 and blacks at about $11,000.

"Everyone is pushing a boulder up a steep hill, but for African-American and Latino families it's a much steeper climb," said Jeremie Greer, the vice president of policy research at the Corporation for Enterprise Development, a Washington, D.C.-based economic think tank. Those families "are less financially secure than they've ever been," he said.

A new report issued by CFED paints a sobering picture of the economic future for many families of color, despite federal policies and programs that were created to help struggling Americans.

The report, called the 2016 Assets & Opportunity Scorecard, analyzed a handful of economic metrics across 50 states and the District of Columbia, including household savings, rates of small business and home ownership, and how much families are paying for housing as a percentage of their overall income. (Data was pulled from public sources, including the Census and the Bureau of Labor Statistics, as well as purchased from private sources.)

Related: The black and white economic divide in 5 charts

On average, blacks and Hispanics fared worse than whites in almost every category.

Take savings. Having less than three months of savings means a family lacks liquid assets that can cushion them in case of an emergency like the loss of a job, the death of a spouse or a serious illness.

While 44% of all Americans have less than three months worth of income saved, 67% of blacks and 71% of Hispanics lack adequate savings, compared to 34.7% of whites.

"Its sobering because it just shows the kind of shoestring that everyone is living on," Greer said. Families of color are also 2.1 times more likely to live below the poverty line compared to white families.

Related: Why blacks believe in the American Dream more than whites

Part of the reason, said Greer, is that people of color are more likely to work in low wage jobs and they are also more likely to get paid less than their white counterparts in many industries, Greer said.

Rent is another factor affecting families. If a family is paying more than one third of their income towards rent, they are considered "cost burdened." Nationwide, 46% of white families are cost burdened compared to 57% of black families and 56% of Hispanic families.

Greer said part of the problem is that many black and Hispanic families live in cities like Washington D.C., New York, Los Angeles and Chicago, where the housing markets weren't impacted as severely by the housing meltdown as those in other cities and rental prices remained high. Wages, however, did not keep pace with the soaring costs of rentals, Greer said.

Blacks and Hispanics don't get a break when it comes to owning a home either. On average, 71% of whites own a home, compared to 41% of blacks and 45% of Hispanics.

That number matters because owning a home "is the primary vehicle of wealth building in this country," especially for people of color, Greer said. Because whites tend to have more wealth, they are more likely to get money from family members to help with a down payment or other costs associated with buying a home than buyers of color.

Homes owned by whites are also likely to be worth more than homes owned by blacks, particularly if those homes are in segregated communities, Greer said.

Related: Working while brown -- what discrimination in the workplace looks like now

Improving the financial state of people of color will require "structural policy change," said Greer. "You have a history of discrimination and disenfranchisement that has been baked into the system and to think that we have gotten past that now is crazy," he added.

The report's authors said they were encouraged by the adoption of the Earned Income Tax Credit, a refundable federal credit for low- to moderate income workers, and called on lawmakers to expanding the credit for childless workers. The Affordable Care Act was also heralded as an example of effective public policy, particularly for helping people of color gain access to health insurance coverage.

But the report said more change needs to happen, including tax reform, increased consumer protections from predatory lenders, financial literacy programs and savings accounts for children.