"When that 53,000 contracts sold in one sale, who did that? Was that the President's Working Group on Financial Markets, or somebody else?" Paul said.

Paul seemed to be referring to a comment that BullionVault Vice President of Business Development Miguel Perez-Santalla made to USA Today (USA Today: "Gold Badly Tarnished"), in which he told that paper that 53,000 gold contracts were sold on Sunday night, furthering the sell-off that had begun on Friday.

The President's Working Group on Financial Markets is actually a little-understood group that President Reagan created by executive order in March of 1988 as a response to 1987's "Black Monday" crash. Working under the Treasury Department, they periodically recommend reforms of the financial markets.

So did this secretive group, working under a government directive, purposely crash the market by engineering the sale of 53,000 gold contracts as former Representative Paul seems to suggests?

Highly unlikely, say some traders.

"It would be news to me, and a lot of people, if the government's hitting the open market like that," said Jeff Kilburg of KKM Financial. "It's a head-scratcher."

RBC Precious Metals Strategist George Gero said that "there was more to it" than any single sale.

"Since September, we lost open interest in gold steadily," as money "went to better-performing assets," Gero explained. "Gold has been a very poor performer all year, as big triple-digit stock up moves were a headwind."

Dennis Gartman, the editor and publisher of the the Gartman Letter, was similarly skeptical. "The gold market is filled with all sorts of conspiratorialist thinking," Gartman wrote to CNBC.com. "Do I think that the government trades futures in gold? Probably they do when panic hits, but I have my doubts that they want to be the creators of panic."

But the decline in gold has more than one culprit, according to Ron Paul, and he pointed his finger at venerable investment bank Goldman Sachs. The firm made a widely followed and wildly prescient call on April 10th to short gold. In retrospect, the call was itself pure gold, and came just as the shiny metal was about to embark on its worst two-day crash since 1980. After the gold short turned out to be very lucrative very quickly, Goldman reversed course on Tuesday, and instructed clients to cover their bearish bullion bets.

(Read More: Goldman Flip-Flops on Gold)

When it comes to Goldman and their calls, "I think they obviously look at the market, and they have ulterior motives, and they make a lot of comments and I have no idea what their purpose is," Paul said. "We have no idea about whether that's accidental, or what."

Goldman Sachs, for their part, told CNBC.com earlier in April that their research "is independent from other activities of Goldman Sachs."

(Read More: Should Traders Trust Goldman?)



So perhaps the government and Goldman Sachs led to gold's drop, in Paul's view. But he still says that in the long-term, "if you print money, the price of gold in terms of dollars will go up."

Still, the near-term outlook for gold remains hazy, in the opinion of this former congressman.

"This year isn't even over yet, so who knows what's going to happen," Paul said. After all, "Goldman Sachs might affect the market again some day."