It was in 2003 that I realised something fundamental had changed. The door to the room in which I was sitting flew open. In stalked a figure still dressed in a dark overcoat and scarf. He evidently could contain himself no longer. I was in Downing Street with the prime minister’s foreign affairs adviser, David Manning; the overcoated figure bursting into our meeting was Jack Straw. He wanted to tell Manning that he had persuaded Joschka Fischer, the German foreign minister, to add Hamas to the EU list of terrorist movements. His tale of his conversion of Fischer was wrapped in expressions of outrage at Hamas. It wasn’t so much the proscription that shocked me. A ceasefire, which I had helped facilitate, had broken down. What was new was the elation with which Straw greeted the banning. I don’t know what Manning thought, but he will have been aware that the terrorist ‘list’ is one of those things from which it’s almost impossible to get a name removed. The consequences for diplomacy, for the politics of peace-making, would be profound, possibly irreversible; but Straw wasn’t worried. Manning, I knew, believed strongly that there could be no solution to the Israel-Palestine issue without Hamas involvement and had firmly supported EU efforts at inclusive peace-building. Officially, the EU remained committed to a political solution, but it now seemed that two key member states were heading in the opposite direction – towards a militarised resolution. The wind had changed.

There had already been hints that a political solution was no longer at the forefront of Whitehall thinking. Not long before, a senior British official had told me bluntly that my methods of building popular consent – holding ‘town hall’ meetings with all factions, working with Hamas, shuttling between Palestinians on the ground and President Arafat to ensure broad participation and continued momentum – were passé. We were in a new era, and it required new thinking: ‘The road to Jerusalem now passes through Baghdad,’ the official insisted. He was speaking just before the 2003 invasion. The message was clear: the Islamic resistance in Palestine was to be neutralised, and psychologically defeated, by the massive display of Western force in Iraq, rather than brought into the political process. Britain and the US expected that the chastened Palestinians would then make the necessary concessions to Israel. What was striking was the official’s conviction that such an outcome was inevitable.

These were heady days for American and British officials and enthusiasm for the ‘war on terror’ was soaring. At our first meeting, Manning’s Downing Street successor, Nigel Sheinwald, told me angrily that security in Palestine could be achieved by eradicating the ‘virus’ of Hamas from Gaza, and eliminating its ‘disease’ from the region. He had no interest in helping to create legitimate Palestinian security services, representative of a cross-section of the community. The language was Washington’s. The Palestinian conflict was seen not as a problem in its own right, but as a subset of a war against ‘extremism’ – another domino to be pushed over in order to strengthen the ‘moderates’. A senior Israeli intelligence official later told me, privately, that he believed the change had begun in earnest in September 2003, after Arafat forced Mahmoud Abbas – a favoured figure in Washington – to resign as prime minister. Angry and frustrated, Bush called Blair. He complained that the Europeans ‘were dancing around Arafat’, while the US was left to do the ‘heavy lifting’ with Israel. Bush also complained that he did not see peace-building as compatible with his ‘war on terror’. Al-Jazeera’s recent release of the Palestine Papers has cast some light on all this: the documents include copies of British covert plans from 2003 and 2004 to ‘degrade’ the capabilities of opponents to the Palestinian Authority, to disrupt their communications, intern their members, close their civil and charitable organisations, remove them from public bodies, and seize their assets. Blair had set aside the lessons of peace-building, so recently learned in Northern Ireland, and embraced the doctrine of counter-insurgency.

The shift in the British position, under American pressure, sabotaged European policy. It undermined the EU’s commitment to promoting Palestinian unity by suppressing, at the covert, security level, opposition to the PA, removing from Palestinian institutions not only all members of Hamas and Islamic Jihad but even those elements in Fatah who had been involved in the second intifada. From now on, the EU would ‘talk the talk’ of encouraging Palestinian unity, while several of its most prominent member states were ‘walking the walk’ of a security-led repression of the very movements the EU was trying to encourage into the political arena. The result was that when Hamas – rather than being demoralised or psychologically defeated by shock and awe in Baghdad – comfortably won the 2006 Palestinian parliamentary elections, the EU was forced into a militarised security response. The new commitment to counter-insurgency meant that there was no prospect of exploring the political possibilities of Hamas’s win. After the election the UN envoy to the Middle East, Alvaro de Soto, wrote a memo to the UN secretary general complaining that the conditions for entering into a dialogue with Hamas had been deliberately set so that Hamas would be unable to meet them – thus engineering its exclusion. De Soto resigned from the UN soon afterwards.

It may seem odd that other EU member states should have acquiesced so readily to the 2003 switch to a militarised solution, but Blair’s approach proved hard to resist. Schisms in the lead-up to the Iraq war had left the EU badly weakened. The instinct of men such as Romano Prodi, the president of the European Commission, was that the EU should try to ‘contain’ American rhetoric – through working closely with Washington – while at the same time seeking to ‘mitigate’ its most harmful consequences. But continuing to work with the US, in the hope of bolstering American officials like Colin Powell, who might soften American policies, severely limited what the EU could do: it had already conceded the American demand that the parameters of Israeli security would be determined by Israel; within this confined space, Palestinians would have to find their own ‘solution’. The US and Britain simply pushed on with the counter-insurgency project; EU efforts to mitigate it proved toothless. Much later, Richard Armitage, Powell’s deputy, told me that the secretary of state had backed away from seeking a one-to-one conversation with the president for more than 18 months – in spite of Armitage’s pleadings.

EU and European foreign policy chiefs pursued a similar ‘strategy’ with Powell’s successor, Condoleezza Rice, jostling and competing among themselves to prove how helpful they could be. Those collecting the most points – phone calls per week with the secretary of state – boasted of playing ‘Athens’ to Bush’s ‘Roman’ muscularity. It’s now clear how little influence any of them had on Washington. In 2006, the EU special envoy was still reassuring Palestinian negotiators that, while ‘the US wants to see a Hamas government fail,’ the EU would ‘encourage Hamas to change and will try to make things work as much as possible’. But at the same time as he was saying this, leading European states were ramping up their covert strategy to destroy Hamas militarily. The Palestine Papers show how this project mushroomed: there was huge investment in training and security infrastructure, prisons were built to allow for the possible internment of Hamas members, the Dayton military battalions were established with the aim of confronting Hamas, and plans were laid to depose the organisation in Gaza and assassinate its leaders. Even the Quartet dived in, working with the intelligence services of other Arab states to disrupt Hamas’s sources of finance.

The changing dynamic in EU thinking was made very clear to me one day in 2007, when I had a meeting with various EU officials, all of whom expressed deep misgivings about the course of EU policy but despaired of convincing any member states to change direction. Later the same day, Javier Solana, then the EU foreign policy chief, gave a new and different reason for following the US line. When I suggested that the EU could not endlessly continue to support the regional status quo but must acknowledge new forces, Solana asked me what at the time seemed an odd question: ‘But if we were to do that, what would happen to my friend Hanan Ashrawi? Would she continue to be able to wear lipstick, and to enjoy an occasional glass of wine?’ It was my first intimation of Europe’s feeling of its own vulnerability. At the time Turkish entry to the EU was being opposed by some member states on the grounds that it would compromise the Christian values underpinning European identity. On my way down from Solana’s office, a colleague said with foreboding: ‘Soon this place [the EU Council] will be very different: Europe is moving to the right.’ He was prescient. The rise of anti-immigrant, anti-Muslim sentiment in Europe has become another factor impeding the EU’s ability to respond to the Islamist challenge.

The Palestine Papers have shown that the Palestinian negotiators, too, were more than willing to work with Israel on its security requirements; and that as their collaboration increased, so did demands for further collaboration. Any concept of Palestinian sovereignty was hollowed out: the putative Palestinian ‘state’ would be still under occupation even if it had a Palestinian flag. Dov Weissglas, Ariel Sharon’s former bureau chief, was clear: to ‘fight terror and instil quiet’ were the two criteria that Israel looked to the PA to provide. Some form of occupation, as a political system of control and containment, thus became the inevitable outcome of a political process that had allowed Israel’s definition of its own security requirements to become the necessary and sufficient principle on which any solution would be based. The EU, too, embraced this proposition, tilting increasingly towards the conviction that Palestinian statehood could be achieved only if Israel’s self-defined security needs were met: an erroneous assumption based on a flawed understanding of Israel’s strategy.

The emphasis on ‘trust-building’ with Israel has coloured the evolution of the political process since 2003. The general movement towards providing ‘law and order’, security co-operation and ‘institution-building’ is well known. But the ‘state-building’ project as a whole should be understood in the context of counter-insurgency – as tangled up with Israel’s unique approach to the collective management of Palestinians – rather than as part of any genuine effort at ‘good governance’. Security action against ‘insurgents’ is only one small element of an American counter-insurgency doctrine which dates back at least to General J. Franklin Bell’s campaign of the early 1900s against Filipino ‘rebels’. Its principles include building a ruling elite to carry out the occupier’s plan; establishing security services accountable only to that elite; concentrating economic control within that elite; and setting up a generous aid policy which sustains a ‘trickle-down legitimacy’ for that elite. The underlying rationale, from the Philippines to Vietnam, has been to instil acquiescence. In the Palestinian case, the doctrine hopes to facilitate close collaboration with Israel and the dismantling of Palestinian resistance. In return, the Palestinians have been promised a depoliticised ‘state’ hardly worthy of the name and subservient to Israel. Perhaps, in such a state, a new Palestinian middle class might live more comfortably; perhaps the visible tools of occupation and control over Palestinian life would be more discreetly concealed; but such ‘statehood’ would amount to little more than a more benign occupation.

The reason that both the West and the Ramallah leadership have embraced the ‘security first’ doctrine is that they were convinced that a two-state solution would eventually emerge, come what may, because it is in Israel’s ultimate demographic interests that it should; therefore one should first placate Israel by agreeing to its demands. The root premise is that Israel is intent on having and maintaining a Jewish majority within Israel, and that with time – and a growing Palestinian population – it would have to acquiesce to a Palestinian state simply in order to maintain its Jewish majority. But over the last 19 years – despite many opportunities – Israel has come no closer to withdrawing to the 1967 borders, the essential precondition for the creation of a Palestinian state. The question is: if the demographic logic is indeed so compelling, why hasn’t it done so? The answer, I believe, is that the premise is wrong.

At a January 2008 negotiating session with Ahmad Qurei and Saeb Erekat which is described in the Palestine Papers, Tzipi Livni spelled out Israel’s strategy:

Israel was established to become a national home for Jews from all over the world. The Jew gets the citizenship as soon as he steps in Israel, and therefore don’t say anything about the nature of Israel … The basis for the creation of the state of Israel is that it was created for the Jewish people.

‘Israel is the state of the Jewish people,’ she had stressed a couple of months earlier, ‘and I would like to emphasise the meaning of “its people” is the Jewish people, with Jerusalem the united and undivided capital of Israel and of the Jewish people for 3007 years.’ ‘Your state,’ she said to the Palestinian negotiators, ‘will be the answer to all Palestinians, including refugees.’ Erekat interpreted her definition as a matter of linguistics and told her: ‘If you want to call yourself the Jewish state of Israel – you can call it what you want.’ He passed on to the Americans his working premise that ‘Israelis want the two-state solution … sometimes more than Palestinians themselves.’ But what Livni was saying was that she wanted Israel to be a Zionist state based on the Law of Return and open to any Jew. To secure such a state in a country with very limited territory means that land and water must be kept under Jewish control, with differential rights for Jews and non-Jews – rights that affect everything, from housing and access to land, to jobs, subsidies, marriages and migration. The fear is that if Israel became a Jewish majority state with fixed borders, the inevitable demand for full equal rights for minorities would herald the end of Jewish special rights and of Zionism itself.

A two-state solution therefore does not solve the problem of how to maintain Zionism, it compounds it. The size of the non-Jewish population of Israel would be reduced from 40 or 50 per cent to 20 per cent if a Palestinian state were created, but the inherent contradiction of a non-Jewish minority with equal rights would undermine it as a Jewish state. Israel’s only answer is to keep its borders undefined while holding on to scarce water and land resources, leaving Palestinians in a state of permanent uncertainty, dependent on Israeli goodwill.

A decade ago, I was a member of Senator George Mitchell’s staff on his first foray into the region, after he was asked to head a ‘fact-finding committee’ into the causes of the second intifada. Even then, Livni’s vision was evident in the Israeli strategy of excluding Palestinians from the body politic while making them subject to its methods of control. The Occupied Territories assumed an elastic, shifting geography in which the rule of law was suspended under cover of the law. It was Sharon who pioneered the philosophy of ‘maintained uncertainty’ that repeatedly extended and then limited the space in which Palestinians could operate by means of an unpredictable combination of changing and selectively enforced regulations, and the dissection of space by settlements, roads Palestinians were not allowed to use and continually shifting borders. All of this was intended to induce in the Palestinians a sense of permanent temporariness. Maintaining control of the Occupied Territories keeps open to Israel the option of displacing Palestinian citizens of Israel into the Territories by means of limited land swaps. It also ensures that Israel retains the ability to force future returning refugees to settle in their ‘homeland’, whereas a sovereign Palestinian state might decline to accept the refugees. It suits Israel to have a ‘state’ without borders so that it can keep negotiating about borders, and count on the resulting uncertainty to maintain acquiescence.

Israel’s vice-premier, Moshe Ya’alon, was candid when asked in an interview this year: ‘Why all these games of make-believe negotiations?’ He replied:

Because … there are pressures. Peace Now from within, and other elements from without. So you have to manoeuvre … what we have to do is manoeuvre with the American administration and the European establishment, which are nourished by Israeli elements [and] which create the illusion that an agreement can be reached … I say that time works for those who make use of it. The founders of Zionism knew … and we in the government know how to make use of time.

Sever Plocker, the deputy editor of Yediot Ahronot, wrote in January that Avigdor Lieberman’s recent plan for a Palestinian state without borders in half of Judea and Samaria was, on the basis of his earlier discussions with Binyamin Netanyahu, more or less the prime minister’s plan too:

Netanyahu argued that the current situation on the ground in Judea and Samaria is stable and safe, and constitutes, for all intents and purposes, a solution to the conflict. The Palestinians already have three-quarters of a state … they have a flag, an international telephone prefix … All that will remain for the government to do, hinted Netanyahu, is only to agree to a change in name of the entity from ‘authority’ to ‘state’ and to toss it a few more bones, a few token signs of sovereignty, such as the right to mint its own currency – and peace will reign for 70 years to come.

The demise of the ‘peace process’ has given us a rare moment of clarity: since the release of the Palestine Papers, the fiction underlying it has become clear to everybody. Such clarity enables new possibilities to emerge. I recall that 30 years ago Namibian independence, which had seemed irredeemably blocked, was suddenly enabled by an unforeseen paradigm change in the region: the implosion of the USSR. Might the coming change in Egypt be an equivalent catalyst for Israelis and Palestinians?