Comprehensive customs and border checks between Ireland and Northern Ireland are not remotely possible, the Irish ambassador to the UK has said, predicting the EU will recognise the unique nature of the border when Brexit causes the UK to leave both the single market and customs union.

Theresa May dealt a blow to Ireland in her Brexit white paper when she said she wanted in effect to leave the EU customs union, confirming Brexit poses a huge threat to frictionless cross-border trade on the island of Ireland, the mainstay of the Irish economy.

But giving evidence to the Northern Ireland select committee, Daniel Mulhall said there were signs that Michel Barnier, the EU’s chief Brexit negotiator, would recognise that the issue of UK-Irish cross-border trade had to be treated as a special local case.

He welcomed Barnier’s suggestion that the Irish border issue be addressed upfront in the coming talks, rather than included in wider horse trading at the end. But he acknowledged the UK and Ireland would not be able legally to negotiate a bilateral customs deal. His admission that physical border checks are impossible puts the onus on the EU showing flexibility, or developing technology to check goods to see if duties are payable.

The EU is caught in a bind of needing to protect the integrity of the single market and customs union, yet prevent a new regime of a hard border and heavy customs checks in Northern Ireland.

Some legal experts have said such a special deal is legally inconceivable, and the Irish government is being over-optimistic that it can prevent a potentially damaging blow to its economy.

Pointing to the scale of required border checks, Mulhall said that at present there were more than 200 crossing points on the borders, with 177,000 lorries crossing a month, 208,000 vans and 1.85m cars.

He said: “I don’t think it is even remotely possible to think in terms of having a border that would really control every movement of goods and people across that border. This is a border that is invisible. It is different from most other borders that our European partners might be familiar with.”

The UK’s decision to be outside the EU common external tariff is raising logistical nightmares since, in theory, the border could become a back door for smuggling and trading.

The UK has been looking at solutions, such as joint border checks with Ireland or greater use of technology to undertake automatic border customs checks. Mulhall said many options were being studied.

Mulhall argued the UK-Ireland border was unique in the EU since no product could subsequently enter the EU except through the Channel tunnel, by air or across the sea.

Mulhall said there was a need to be creative and make allowances to accommodate the unique situation. But he insisted he had not got the impression his diplomatic colleagues were “trying to put a spanner in the works and make life difficult for Ireland”.

He also insisted retaining a common travel area would not blow a hole in the border controls of the EU or the UK. He argued there would be no motive for EU citizens to try to infiltrate the UK labour market via Ireland, undermining UK border controls, since they would be unable to work legally in the UK. They would find it easier and cheaper to get into the UK directly via Stansted.

It would be “complete anathema” and “a turning back of the clock” to have controls on the free movement of people across the border.

He said: “I do not underestimate the difficulties because it has never been done before and it is extremely complex to reconcile 27 views. There is always potential for things to go awry.”

But he wanted to scotch the idea that the EU sought to punish the UK for its aberrant behaviour.

Both the taoiseach, Enda Kenny, and May have expressed confidence that the common travel area between Britain and Ireland will survive unchanged after Brexit but warned of “very negative consequences” if there was a return to a hard border with Northern Ireland.

The prime minister has also promised the border will be as seamless and frictionless as possible.