It's more expensive than ever to get your hands on your own money.

A customer is slapped with fees when they use an ATM that's not owned by their bank, and those fees jumped 5% in the past year, according to Bankrate.

On average, a customer is charged a total of $4.35 for each transaction.

People are getting smarter and avoiding out-of-network ATMs as much as they can, said Bankrate's chief financial analyst Greg McBride. But that's part of the problem. Banks are hiking fees to make up for the loss.

In the past five years, ATM fees have skyrocketed 23%, McBride said.

And overdraft fees are on the rise for the 16th year in a row, according to the survey of 10 major U.S. banks. On average customers get hit with a $32.74 charge every time they try to withdraw more money than they have in the bank.

Hefty bank fees will probably always be a reality. But the good news, McBride said, is that both "ATM fees and overdraft charges can be avoided completely."

What I gave up to save $1 million