A damaging culture of “extraordinary secrecy” inside government is blighting its ability to plan for Brexit, according to a comprehensive study of Whitehall.

Officials are being forced to look at key documents in special reading rooms, while some papers are confined to the offices of the most senior civil servants. The installation of a network of secured computers that can only be accessed by officials with very high security clearance is also being accelerated, to keep the documents under wraps.

Meanwhile, the number of documents being restricted is going “well beyond” those containing sensitive details of the government’s EU negotiations. Even basic planning and guidance documents are kept locked away, largely inaccessible to civil service teams that need to see them.

A security clearance backlog has also meant that some officials have waited up to nine months to gain access to the material they need.

The stark revelations are in a new study by the respected Institute for Government (IfG), which gained access to senior officials across Whitehall and public bodies as part of the project. It rejects the government’s claim that the secrecy is needed to protect its negotiating position with the EU. Instead, it concludes that secrecy is being fuelled by cabinet splits over the direction of Brexit and the need to avoid “domestic political embarrassment”. It concludes that the drive to restrict information has made effective co-ordination of Brexit work across government “impossible”.

The assessment comes with the cabinet deeply split over Brexit, with Boris Johnson, the foreign secretary, believing the process is being botched and David Davis, Brexit secretary, demanding concessions from the prime minister over tactics.

The report, Preparing Brexit: How ready is Whitehall?, identifies several main challenges affecting the preparation process. It says there are decision delays and an “inability to make trade-offs”, inordinate secrecy, inconsistent assumptions across Whitehall, an inability to properly engage external organisations such as the business community and difficulties in finding and retaining top staff.

As a result of deep divisions, a culture of extraordinary secrecy has developed Institute for Government study

It states that towards the end of 2017, the already high level of security surrounding Brexit was increased further and key documents were heavily restricted. Only “a handful” of hard-copy papers are available to a limited number of people and they can only be accessed in designated Whitehall reading rooms or a permanent secretary’s office – the most senior official in a department.

With the help of a branch of GCHQ, the government has accelerated the rollout of secure computers, known as Rosa terminals, across Whitehall. Only those with approved profiles and SC level security clearance – the second-highest level – can use them.

The IfG gives a grim assessment of the state of play. “The greatest challenges faced by Whitehall now stem from splits within the cabinet on major policy decisions,” it says. “Whitehall is working to a sketchy vision of future relations with the EU, built on a fragile cabinet consensus, that no one in the government can expect to survive negotiations with Europe.

“As a result of these deep divisions, a culture of extraordinary secrecy has developed. This goes well beyond documents that contain sensitive negotiating material, with even rudimentary planning guidance kept locked away and largely inaccessible to the teams – all across government – that need to use it. Key documents are over-classified, important information is not being shared between departments and those with a legitimate reason to be informed, such as parliament, are kept in the dark.

“In this environment, effectively co-ordinating work across almost 20 departments, devolved administrations and a large number of public bodies is impossible. No clear end-state, poor information flow and competing ministerial preferences make delivering Brexit extremely difficult.”

Around 10,000 new Brexit roles have been created within the civil service since the EU referendum. Delays in the security-clearance process have emerged. At the start of the year, SC [security check] clearance, required for accessing many secret Brexit documents, was taking approximately three months. DV [developed vetting] clearance, for staff expected to be working with the most secret information, was taking closer to nine months.

A new scheme has been put in place to prioritise Brexit staff needing SC clearance and the average processing time has now been cut to two weeks. The delay for those needing the higher level remains, however.

The report calls on the Brexit department and the Cabinet Office to set out the structures through which negotiations will continue after Britain leaves the bloc in March next year. It says a cabinet minister and an official should have overall responsibility for Brexit readiness, while Brexit department staff should be given certainty over the ministry’s future to reduce staff departures.

A government spokeswoman said: “The British people voted to leave the European Union and the government is focused on delivering this historic task and seizing the opportunities offered by our withdrawal. Close collaboration between departments is a vital component of this and Whitehall has risen to the challenge. We have already recruited over 5,500 civil servants to ensure we are properly resourced for the task. Planning is going well, and we are confident that work to leave the EU will be a success.”