Dennis Gartman said Thursday that Greece would be better off defaulting and exiting the euro zone.

"If I were the Greek prime minister I would have defaulted long ago and left because at least I then know I can get my currency back, I can devalue it, my textile industry becomes competitive, my tourism industry becomes competitive again, my shipping industry becomes competitive," the publisher of the Gartman Letter said on CNBC's "Squawk Box."

Negotiations between Greece and its European creditors have broken down, and the Mediterranean country faces billions of euros in debt payments at the end of the month.

"If I were [Prime Minister Alexis] Tsipras I would have told them to take it and shove it and walked off long ago," Gartman said. (Tweet This)

Read More Why it's so hard to predict the impact of a Greek default



Given Greece's small population and the size of its economy, a Grexit would be "relatively inconsequential," Gartman said. He acknowledged the event would cause "great concern" over the fate of the euro, but said investors would soon come around to the idea that a euro without Greece is more valuable than a euro with it.

Not everyone is so sanguine. The Organization for Economic Cooperation and Development earlier this month said a Greek exit could derail the entire euro zone recovery.

Others worry that Greece's departure could set a precedent for other European countries struggling under the burden of heavy debt loads and austerity measures.

The only reason Greece has remained a part of the currency union this long is because Germany won't let it out, Gartman said.

"Germany needs to keep Greece in the euro to keep the euro otherwise cheaper than it would be because Germany is an exporting country," he said. A Grexit would also be detrimental to France and Poland, he added.

A weak currency makes a country's goods more affordable to foreign customers.

Confusion is swirling over German sentiment on a Grexit because the public wants it, but the country's leadership and "cognoscenti" need Greece to remain in the union to support Germany's economic expansion, Gartman said.

"Bayer needs Greece in. Thyssenkrupp needs Greece in. Daimler needs Greece in," he said. "And that's the problem. That's the confusion that people are talking about."