Sawitree Pamee/EyeEm | Getty Images

Individual retirement accounts, CDs and other savings options often come with stipulations around when you can pull out your money. Many people dutifully follow these rules in order not to face the much dreaded early withdrawal penalty. But there are instances in which the penalty might not apply to you, or times when the benefits outweigh the consequences. Keep in mind: these moves should often only be made if necessary, as they put your savings in jeopardy.

Certificates of Deposit

Mark Kantrowitz, a student loan expert, saw his CDs growing at less than 1 percent a year at his bank. Then he noticed another bank was offering annual returns of 1.72 percent on their certificates of deposit. To take an early withdrawal, he knew, would result in a penalty of three months interest. Still, he went ahead. "The bank teller was shocked that I wanted to close [it] early. She called over the branch manager, who said 'But, but, but you'll have to pay a penalty,'" Kantrowitz said. "I explained that I would be ahead financially after three months at the other bank's higher interest rate."

In the rising interest rate environment in which we find ourselves, it may be particularly advantageous to swallow an early withdrawal penalty from a CD to move your money to a higher yielding destination, said Greg McBride, chief financial analyst at Bankrate.com. "You want to calibrate your penalty with what your upside is," he said. Make sure you fully understand what you're getting yourself into, he said. For example, if a bank charges a three-month interest penalty for early withdrawals, and you've only had the certificate for two months, they might be able to dig into your principal, he said.

529 plans

There's no such thing as an early withdrawal from a 529 plan, the tax-advantaged account that can be used for education-related expenses, because there are no rules around when you can use the money — just for what you can use it for: education-related expenses only. You'll pay a penalty if you use it for other reasons. If you end up using the money for a new television or a vacation instead of their dorm or textbooks, you'll pay a 10 percent tax penalty as well as income taxes on the account's earnings, said Kantrowitz, the publisher of SavingforCollege.com. Though, he called that consequence, "negligible." He calculates it's often around 1 percent to 3 percent of the entire amount. "The family is no worse off than they'd be in a taxable account," Kantrowitz said. Though, he added, you might have to repay the state income tax benefits you picked up for using the account. And, most important, if your child does plan on going to college, withdrawals will leave your savings less time to compound.

Retirement savings

Making an early withdrawal from your retirement account is one of biggest no-no's in personal finance. After all, tapping an individual retirement account or 401(k) plan before the age of 59½ can result in a 10 percent penalty, income taxes and a depleted fund for your retirement. But there are ways to lessen that penalty should you really need the money, said Ed Slott, an expert on retirement saving. "The tax law contains several exceptions to the 10 percent penalty, eliminating that expense when you withdraw early from you retirement account," said Slott, a CPA and president of Ed Slott and Co. Unfortunately, he said, there's a lot of confusion around when the penalty does and does not apply.