Rendell is one of those voices. He supports raising the federal gas tax— currently at 18.40 cents a gallon on gasoline and 24.40 on diesel —to garner more funds for roads, bridges and construction across the U.S. The gas tax has stayed at current levels for about two decades.

Read More These five chains get a boost from a drop in gas prices

Gas prices, on average, are about 50 cents less a gallon than a year ago, giving a de facto tax cut and providing relief to millions of consumers as the holiday season kicks into high gear. However, a growing number of politicians have suggested that, given cheaper fuel costs, conditions might be ripe to hike federal taxes on gas in order to help fund infrastructure projects.

Consumers may be enjoying extra money in their pockets thanks to lower gas prices, but former Pennsylvania Gov. Ed Rendell suggests they shouldn't get used to it.

"Our infrastructure's crumbling," argued the Pennsylvania Democrat in an interview with CNBC. "Our roads and our bridges are in dangerous condition."

Roads in better repair could mean more efficient traffic, Rendell believes, meaning a higher gas tax could "actually save people money."

His comments came as retail gas prices in certain parts of the country have fallen below $3, their lowest in at least four years.

The average price of a gallon of gasoline in the United States has dropped one dollar since early May, the lowest price in more than four years, according to the Lundberg survey released Sunday.

Prices for regular-grade gasoline fell to $2.72 a gallon, down 12 cents since the last survey two week ago.

Meanwhile, many free market economists are opposed to hiking the gas tax. They argue that federal gas taxes should be completely abolished, given that they burden drivers with additional costs.

Read MoreGas dips below $2 a gallon in Oklahoma City

The average driver loses about $800 annually due to sitting in traffic, according to one study. Congestion on America's roadways cost $121 billion in 2011, according to the Urban Mobility Report from the Texas A&M Transportation Institute. About $27 billion of that is derived from idle time, and fuel from trucks moving commercial goods.

A gas tax increase of about 12 cents per gallon, adjusted for inflation after two years, was proposed in June by Sen. Bob Corker, R-Tenn., and Sen. Chris Murphy, D-Conn., to raise $164 billion in funds for the U.S. Highway Trust Fund.

"Say that cost the average driver $130 a year," Rendell said. "They would get a return on that investment" in safer roads and increased quality of life, he added.

In addition to modernizing roads and bridges, Rendell believes a long-term infrastructure improvement strategy could create thousands of jobs paying $50,000 to $80,000 a year in industries like construction, concrete and metal.

A shored-up middle class could be a key boost in an increasing competitive global economy, the former governor said. He added the U.S. has a competitive interest in pouring more money into roads and bridges.

"The World Economic Forum, 10 years ago, rated us the best infrastructure in the world," Rendell continued. "We need to do something for our infrastructure, not in a one or two year period, but over a decade."

—Reuters contributed to this report.