With gas prices plunging, you're probably happier every time you pull up to a pump these days. But as a U.S. taxpayer, you're also poorer.

That's because one of the biggest losers from the roughly $50 a barrel drop in oil prices in the last six months is the U.S. Energy Department itself.

The Energy Department owns nearly 700 million barrels of oil it has stored underground in locations around the country, making up what is known as the Strategic Petroleum Reserve.

That huge supply of oil held by the government has lost about $35 billion in value since June.

The government says it paid an average of about $30 a barrel for the oil it holds, so the oil is still worth more than it cost. Oil was trading at just over $56 a barrel Monday.

Related: Even Warren Buffett got hurt by oil prices

The SPR is there to protect the country in case of a disruption in the supply of crude oil to U.S. refiners. The last time substantial amounts of oil were released from the SPR came in 2011, in the wake of the Libyan civil war, and in 2005 following Hurricane Katrina. But that oil was eventually replaced. The SPR reserves now stand near their all time high.

The drop in value of the SPR's oil doesn't really hurt the federal government's balance sheet or cause the federal deficit to swell, any more than the run-up in the price of oil or gold in recent years reduced the deficit. So you don't have to worry about a government default or cutback in services the way citizens of Venezuela or Russia do.

Related: Cheap gas is akin to a $60 billion tax cut

But there are critics of the SPR who say the huge supply of oil is probably no longer needed the way it once might have been. U.S. production has increased sharply in recent years, and the nation now produces more oil than Saudi Arabia, fulfilling most of the domestic demand for crude. There is even a new push to allow U.S. oil producers to export oil for the first time in nearly 40 years.

"I don't know if we ever had an asset worth $70 billion in June and $35 billion now," said Tom Kloza, chief oil analyst for the Oil Price Information Service and GasBuddy.com. "It's a silly anachronism these days, given production growth."

Even Kloza doesn't think that selling SPR's oil at current prices is a good idea, given the disruptions it could cause for the markets and U.S. energy companies. But he thinks the government should start planning a slow, gradual phase out of the SPR, especially whenever the price starts to climb to the $80 a barrel range at some point in the future.