Mortgage rates are falling sharply, with the average rate on a 30-year fixed mortgage dropping from 4.23% on May 21 to 3.94% as of Monday. And for the 5.9 million homeowners who could score lower rates by refinancing, the savings come out to around $270 per month, CNBC's Diana Olick reports. However, refinancing your mortgage isn't necessarily a smart choice for every homeowner. In some cases, it could take a decade or more to recoup the upfront costs. If you're considering refinancing your home, here are four questions to ask yourself first.

1. How long are you planning to stay in your home?

"The No. 1 sign you shouldn't refinance is that you plan to move in the very near future," Kristin Baker, chief of staff at White Oaks Wealth Advisors in Minneapolis, Minnesota, tells CNBC Make It. "There are many costs associated with refinancing and if you move before you recoup those costs with a lower payment, you have wasted time and money."

The longer you plan to spend in a house, the more worthwhile a refinance could be. Sean M. Pearson certified financial planner at Ameriprise Financial

"The longer you plan to spend in a house, the more worthwhile a refinance could be," Sean M. Pearson, a certified financial planner at Ameriprise Financial in Conshohocken, Pennsylvania, tells CNBC Make It. "If there is a chance that you could move for a job in a few years, it's probably less likely that a refinance makes sense. It does not make much sense to pay $5,000 in fees and closing costs for the privilege to save $100 per month for three years."

2. How much will it cost to refinance your mortgage?

If you know you're planning to keep your mortgage for a while, your next step is to determine the amount you might be able to save by refinancing. But before you can calculate that, you need to consider how much the process will cost you upfront. Refinancing isn't free: In order to secure a lower interest rate, you'll end up paying closing costs again, which can include bank fees, appraisal fees and attorney fees, among other things. These costs typically run around 2% of your total mortgage balance, although that can vary, John Cooper, a certified financial planner at Greenwood Capital in Greenwood, South Carolina, tells CNBC Make It. On a $300,000 mortgage, for example, you would expect to pay around $6,000 in fees. From there, it's helpful to do the math to calculate how long it would take you to earn those fees back. "It's best to recoup that closing cost in five years or less," Cooper says. "You don't want to extend it too long, or else you're not really making a lot of headway." Say you took out a $400,000 30-year mortgage 10 years ago with a 4.5% interest rate, for example, and have already paid down $80,000 of that. For the next 20 years, you can expect to pay around $2,026 per month on the rest of the $320,000 mortgage, Cooper calculates. If you're able to refinance with a 3.75% interest rate on a 20-year mortgage, your monthly payment would drop to $1,897, saving you around $130 per month. That means it would take you just under four years to recoup the $6,000 it cost to refinance. "In that example, I would say, 'Sounds like it makes sense to me,'" Cooper says. "You're recouping it back in less than four years and then the next 16 years, you're saving another $129 a month." That adds up to around $25,000 in savings over the life of the mortgage, Cooper calculates.

However, it's also important to consider how long you'll be paying down the rest of the mortgage so you don't lose your potential savings to additional interest costs. "If you are lowering monthly bills by adding years to your payment, you might end up paying more over the life of the loan," Pearson explains. "If you need more money for bills or to save today, understand that adding years on the back end will likely cause you to pay more for the life of a loan." Before jumping on any lower interest rates, take the time to do your homework and calculate exactly how much refinancing could actually save you.

3. Will refinancing help you with other lifestyle factors?

Sometimes, refinancing itself might not save you much money, but could be beneficial for other reasons. In some cases, for instance, refinancing allows you stop paying private mortgage insurance (PMI), which is a policy the lender takes out if your loan exceeds 80% of the value of the home. "PMI is not cheap," Cooper explains. "You could be paying $100 to $150 a month toward the bank's insurance policy that has no bearing on the value of your home, it doesn't pay down your mortgage, it does nothing for you." But if you're able to refinance with a new mortgage that's 80% or less the current market value of your home, and therefore "do away with PMI, then you could more than make up the difference with a smaller interest rate reduction," Cooper says.

Refinancing can also be worthwhile if you want to remove someone from the mortgage, such as an ex-spouse following a divorce. If your credit score and payment history are strong enough to allow you to refinance without your ex, "that could be a reason to refinance even if you weren't saving much money," Cooper says, just to get the peace of mind of not needing to interact with that person each month in order to pay your bills.

4. What are you planning to do with the savings?