U.K. Foreign Secretary Jeremy Hunt has backtracked on comments he made likening the European Union (EU) to the Soviet Union.

"Any sensible reading of the speech would see that this was a passionate request and desire for friendship with our European neighbors going forward. But what I was saying is if the attitude of the EU is that someone that wants to leave the club has to be punished, then that's not consistent with European ideals," he told CNBC's Steve Sedgwick at the Conservative Party conference in Birmingham, England.

"The EU was partly set up to stand firm against Soviet totalitarianism and I was just pointing out the contradiction that that's what we were set up to stop," he said Tuesday.

Brexit negotiations between the EU and U.K. have hit an impasse with the EU rejecting British Prime Minister Theresa May's plan for a post-Brexit relationship with the remaining 27 members of the bloc.

Foreign Secretary Hunt prompted a controversy after he told the Tory conference on Sunday that the EU was acting like the Soviet Union in trying to prevent members like the U.K. leaving the bloc. Prime Minister Theresa May also echoed Hunt on Tuesday, saying that the European Union was not the same as the Soviet Union.

Asked whether Hunt was right to have made the comment, May said: "As I sit around that table in the European Union, there are countries there who used to be part of the Soviet Union. They are now democratic countries and I can tell you that the two organizations are not the same," Reuters reported.