Massive new leak lifts lid on negotiationsPLO offered up key settlements in East JerusalemConcessions made on refugees and Holy sites

The biggest leak of confidential documents in the history of the Middle East conflict has revealed that Palestinian negotiators secretly agreed to accept Israel's annexation of all but one of the settlements built illegally in occupied East Jerusalem. This unprecedented proposal was one of a string of concessions that will cause shockwaves among Palestinians and in the wider Arab world.

A cache of thousands of pages of confidential Palestinian records covering more than a decade of negotiations with Israel and the US has been obtained by al-Jazeera TV and shared exclusively with the Guardian. The papers provide an extraordinary and vivid insight into the disintegration of the 20-year peace process, which is now regarded as all but dead.

The documents – many of which will be published by the Guardian over the coming days – also reveal:

• The scale of confidential concessions offered by Palestinian negotiators, including on the highly sensitive issue of the right of return of Palestinian refugees.

• How Israeli leaders privately asked for some Arab citizens to be transferred to a new Palestinian state.

• The intimate level of covert co-operation between Israeli security forces and the Palestinian Authority.

• The central role of British intelligence in drawing up a secret plan to crush Hamas in the Palestinian territories.

• How Palestinian Authority (PA) leaders were privately tipped off about Israel's 2008-9 war in Gaza.

As well as the annexation of all East Jerusalem settlements except Har Homa, the Palestine papers show PLO leaders privately suggested swapping part of the flashpoint East Jerusalem Arab neighbourhood of Sheikh Jarrah for land elsewhere.

Most controversially, they also proposed a joint committee to take over the Haram al-Sharif/Temple Mount holy sites in Jerusalem's Old City – the neuralgic issue that helped sink the Camp David talks in 2000 after Yasser Arafat refused to concede sovereignty around the Dome of the Rock and al-Aqsa mosques.

The offers were made in 2008-9, in the wake of George Bush's Annapolis conference, and were privately hailed by the chief Palestinian negotiator, Saeb Erekat, as giving Israel "the biggest Yerushalayim [the Hebrew name for Jerusalem] in history" in order to resolve the world's most intractable conflict. Israeli leaders, backed by the US government, said the offers were inadequate.

Intensive efforts to revive talks by the Obama administration foundered last year over Israel's refusal to extend a 10-month partial freeze on settlement construction. Prospects are now uncertain amid increasing speculation that a negotiated two-state solution to the conflict is no longer attainable – and fears of a new war.

Many of the 1,600 leaked documents – drawn up by PA officials and lawyers working for the British-funded PLO negotiations support unit and include extensive verbatim transcripts of private meetings – have been independently authenticated by the Guardian and corroborated by former participants in the talks and intelligence and diplomatic sources. The Guardian's coverage is supplemented by WikiLeaks cables, emanating from the US consulate in Jerusalem and embassy in Tel Aviv. Israeli officials also kept their own records of the talks, which may differ from the confidential Palestinian accounts.

The concession in May 2008 by Palestinian leaders to allow Israel to annex the settlements in East Jerusalem – including Gilo, a focus of controversy after Israel gave the go-ahead for 1,400 new homes – has never been made public.

All settlements built on territory occupied by Israel in the 1967 war are illegal under international law, but the Jerusalem homes are routinely described, and perceived, by Israel as municipal "neighbourhoods". Israeli governments have consistently sought to annex the largest settlements as part of a peace deal – and came close to doing so at Camp David.

Erekat told Israeli leaders in 2008: "This is the first time in Palestinian-Israeli history in which such a suggestion is officially made." No such concession had been made at Camp David.

But the offer was rejected out of hand by Israel because it did not include a big settlement near the city Ma'ale Adumim as well as Har Homa and several others deeper in the West Bank, including Ariel. "We do not like this suggestion because it does not meet our demands," Israel's then foreign minister, Tzipi Livni, told the Palestinians, "and probably it was not easy for you to think about it, but I really appreciate it".

The overall impression that emerges from the documents, which stretch from 1999 to 2010, is of the weakness and growing desperation of PA leaders as failure to reach agreement or even halt all settlement temporarily undermines their credibility in relation to their Hamas rivals; the papers also reveal the unyielding confidence of Israeli negotiators and the often dismissive attitude of US politicians towards Palestinian representatives.

Last night Erekat said the minutes of the meetings were "a bunch of lies and half truths". Qureia told AP that "many parts of the documents were fabricated, as part of the incitement against the … Palestinian leadership".

However Palestinian former negotiator, Diana Buttu, called on Erekat to resign following the revelations. "Saeb must step down and if he doesn't it will only serve to show just how out of touch and unrepresentative the negotiators are," she said.

Palestinian and Israeli officials both point out that any position in negotiations is subject to the principle that "nothing is agreed until everything is agreed" and therefore is invalid without an overarching deal.