The foreign secretary, David Miliband, was urged tonight to explain how four Chinese detainees from Guantánamo Bay came to be released to a UK territory without the knowledge of Whitehall.

An urgent security assessment is under way after Bermuda accepted the Uighurs without consulting the Foreign Office.

"We have underlined to the Bermuda government that it should have consulted the UK on whether this falls within their competence or is a security issue for which the Bermuda government do not have delegated responsibility," said a spokesman for the Foreign Office.

"We have made clear to the Bermuda government the need for a security assessment, which we are now helping them to carry out," he added, saying that the four former detainees did not have papers, so could not leave Bermuda at present.

Bermuda's premier, Ewart Brown, agreed to accept the four Uighurs, who the US government ruled were not enemy combatants, to help President Barack Obama meet his pledge to close the camp.

But William Hague, the shadow foreign secretary, demanded an explanation from Miliband and said the government appeared to have "lost grip of running the country" amid internal Labour party rows over Gordon Brown's future.

"It is astonishing that an agreement of such significance … could have taken place without a ripple reaching Whitehall," Hague said. "The UK is responsible for Bermuda's external relations, defence and security and for appointing its governor. Yet the FCO [Foreign and Commonwealth Office] appears to have had no idea that these discussions were taking place.

"Even before this there were serious questions about whether the government has paid sufficient attention to UK overseas territories. These questions have reached a new level."