Last year at this time, the 30-year FRM averaged 6.35 percent.

“Bond yields nudged mortgage rates slightly lower this week, with the fixed-rate mortgage rates falling to three month lows,” said Frank Nothaft, Freddie Mac vice president and chief economist.

But FBR's Paul Miller pegs it more on the government buying mortgage backed securities. "If the Fed backed out of a liquidity stance, treasuries would scream," says Miller. "I think the government will try to keep it right around 5 percent."

That's because the government needs low mortgage interest rates to support a housing recovery, especially now, as the first time home buyer tax credit expires Nov. 30. But will rates go any lower? Or did we all miss the boat last spring, when rates hit record lows?

"Treasury yields and mortgage rates have pulled back on concerns about the sustainability of the economic recovery, particularly with household income and consumer spending weak," says Bankrate.com's Senior Financial Analyst Greg McBride. "However mortgage rates are very dependent on the Fed's purchases of $300 billion in Treasury debt that will expire in October and $1.25 trillion in mortgage bond purchases that will expire at year-end. Unless the economy stumbles, we're likely to see higher rates in the fourth quarter."