When we talk about poverty, we often talk about material hardship and the cost of things like housing, food, clothing, and transportation.

But when we look at the budgets of struggling families, we often ignore the cost of money itself. Yet the old adage that it takes money to make money is also true in reverse. Without money, it costs money just to deal with money.

The cost of a bank account has gone up in recent years. Only about half as many banks offer free checking accounts as they did in 2009, and the average minimum balance to avoid bank fees reached $723 in 2012, up 23 percent from the previous year. At the same time, not having a bank account—roughly 17 million American adults don’t, according to the Federal Deposit Insurance Corporation (FDIC)—can be even more expensive. And policymakers need to factor in these costs when making decisions not just about banking regulations, but a wide range of policies that affect low-income families. Otherwise, struggling families and consumers will continue to lose dollars needlessly.

Get TalkPoverty In Your Inbox Thanks for Signing Up!

Take the case of Natalie Gunshannon, a McDonald’s employee in Pennsylvania. She sued her local franchise last year when it forced her to receive her wages on a prepaid debit card. This card charged $1.50 for every ATM withdrawal, or $5 to withdraw cash at a bank. It even charged $1 to check the account balance at an ATM, and 75 cents to pay bills online. Depending on how the card was used, it could easily drive someone’s earnings below minimum wage.

In Natalie’s case, using the card made no sense because she already had a free account at a local credit union. But not all low-wage workers have good alternatives like Natalie, and if they decide not to participate in the mainstream banking system, they may ultimately pay even more for simple transactions. Check cashers may charge from 1 to 5 percent of the amount on a paycheck or government benefit check in order to cash it. And many unbanked consumers end up paying again when they need to buy money orders or pay bills in person.

A low-wage worker paid $700 every two weeks, facing a 2 percent check cashing fee, and buying two money orders each month, spends more than $30 per month on financial services. That means he or she is working nearly an hour each week just to pay the check casher. For many workers, these costs are often even higher. It’s a slow drip for workers that takes dollars out of every single paycheck, and it adds up to billions of dollars that consumers and families could otherwise spend or save. By simply expanding access to affordable banking services millions of Americans would receive a raise.

Sometimes that slow drip for financial services turns into a catastrophic flood. About two out of every five Americans say that they probably or definitely wouldn’t be able to come up with $2,000 in 30 days to deal with an emergency. That number rises to about two-thirds of lower-income Americans. When faced with financial shortfalls, struggling families may turn to payday lenders for quick cash pledged against the next paycheck, or to auto title lenders who offer cash in exchange for the car title and a spare set of car keys.

These lenders typically charge triple-digit annual interest rates and, not surprisingly, borrowers can’t keep up. Four out of five payday loans is rolled over to a new loan. In Virginia alone, auto title lenders repossessed 13,000 cars in 2012 among borrowers who couldn’t make their payments. In 2007, the Pentagon was so concerned about these loans that they successfully pushed Congress to cap at 36 percent the annual interest rate that can be charged to our troops and their families. However, bills to extend this cap to everyone have gone nowhere.

Fortunately, many of the solutions to these problems already exist. In its three-year history, the Consumer Financial Protection Bureau has effectively been a new cop on the beat examining the tricks and traps in the financial marketplace. State and local governments have expanded access to affordable, basic bank accounts and pushed for more affordable prepaid cards for payroll and benefits. Policymakers are now considering whether to build out public banking options, such as offering more banking services through the post office, as many other countries do. In cities like New York, financial empowerment centers that offer free financial counseling now help struggling families make the most of their money and avoid the rip-offs that make it even harder to get ahead.

Not all of these efforts have succeeded, however. Last year, the Chicago Transit Authority (CTA) announced an innovative new product called Ventra: a transit farecard that doubled as a prepaid debit card. In theory, this card could have turned every subway and El station into a banking center. But as designed, it was a high-fee card that guaranteed revenue for the CTA and was a bad deal for consumers. It took public pressure to redesign the card’s features and fees and launch a better product. In the meantime, trust was lost.

Yet the occasional step backward shouldn’t stop policymakers from seeking safer, more affordable, and more convenient ways for low-income Americans to manage and save their money. When every dollar counts, we should make sure families are able to keep the dollars that belong to them.