Last year, the Commission ruled that Ireland must recover from Apple. It found that the U.S. technology giant paid an in Ireland in 2014.

"It's a tricky thing to do because it's a large sum so of course you have to figure out how to do that. It's not as an escrow account in some of the other cases where it might be 25 or 30 million euros … and therefore I do respect that it's a complicated matter and it may take a little more time."

"Well the recovery is not done yet but we have been working with the Irish authorizes and we can see that they are moving forward to do the recovery of the unpaid taxes," EU Competition Commissioner Margrethe Vestager said during a press conference in response to a question by CNBC.

has not fully paid the 13 billion euros ($13.9 billion) it owes to Ireland in illegal tax benefits even though the deadline has passed, the European Union's competition said on Tuesday.

The deadline for Ireland to recover the money into an escrow account was January 3. But the money still has not been repaid.

"We are continuing to make progress of the recovery from Apple with the full cooperation of the company and the EU Commission," a spokesperson for Ireland's Ministry of Finance told CNBC by email.

"The Commission are satisfied with the progress we are making. We have committed to complying with the decision and we fully intend doing that."

Still both Ireland and Apple have pledged to fight the decision via the European courts. Vestager said at the press conference that she does not know when the court case will take place for Apple and Ireland to appeal the EU's decision.

Ireland's Finance Minister Michael Noonan, in response to a question from CNBC concerning Apple's taxes, said because of the legal hurdles it could be years before there is a conclusion to the case.

"The appeal is in now and it'll go to a European ordinary court first and then whoever loses will probably appeal it to the European Court of Justice. So you're looking at a four-year time frame, five-year time frame. (A) slow bicycle race between the Apple case and Brexit seems to be emerging now. Let's see which will reach the destination first."

Apple has yet to respond to a request for comment when contacted by CNBC. But the company has spoken strongly about the decision. In an interview with the Irish Independent last year, Apple CEO Tim Cook called the ruling "total political crap" which has "no basis in law or in fact".

The Irish government has also been critical of the decision. Last year, Noonan slammed Brussels' interference in the country's tax matters.

"We stand by the legitimacy of what was done in the past ... we think the Commission is getting involved in what is the competence of sovereign governments in Europe…. This is an approach through the back door to try and influence tax policy through competition law," Noonan told CNBC in August.