Treasury Secretary Steven Mnuchin said Thursday the department will police bitcoin and other cryptocurrencies with "very, very strong" regulations so that they don't become cloaked in secrecy.

"We're going to make sure that bitcoin doesn't become the equivalent of Swiss-numbered bank accounts, which were obviously a risk to the financial system," Mnuchin said in an interview on CNBC's "Squawk Box."

"I want to be careful that anybody who's using bitcoin — regardless of what the price is — is using it for proper purposes and not illicit purposes," he added. "And there are billions of dollars of transactions going on in bitcoin and other cryptocurrencies for illicit purposes."

Certain Swiss banks historically offered clients additional secrecy though numbered accounts known only to the client and select bankers. While affording customers another layer of privacy, such accounts have been criticized by governments around the world for helping potentially bad actors avoid government scrutiny or avoid taxation.

Mnuchin was also asked if bitcoin's use in illegal transactions disqualifies it as a legitimate currency, given that the U.S. dollar is often used by bad actors for illegal purposes.

"I don't think that's accurate at all, that cash is laundered all the time," Mnuchin replied. "We combat bad actors in the U.S. dollar every day to protect the U.S. financial system."