A group of former British ambassadors have joined a campaign calling for Tony Blair to be removed from his role as Middle East envoy after his recent attempt to "absolve himself" of responsibility for the crisis in Iraq.

The letter, organised by the makers of George Galloway's film The Killing of Tony Blair, says the 2003 invasion of Iraq was to blame for the rise of "fundamentalist terrorism in a land where none existed previously".

The signatories, led by Blair's former ambassador to Iran Sir Richard Dalton, describe the former prime minister's achievements as Middle East envoy as "negligible".

Other former diplomats to sign the letter are Oliver Miles, who was ambassador to Libya when diplomatic relations were severed in 1984 after the killing of WPC Yvonne Fletcher, and Christopher Long, ambassador to Egypt between 1992-95.

Other signatories include former London mayor Ken Livingstone, the human rights barrister Michael Mansfield QC, the former Liberal Democrat peer Lady Tonge – who resigned her party's whip in 2012 after declaring that Israel would not last forever – the former Tory prisons minister Crispin Blunt, George Galloway, the Green MP Caroline Lucas and the Daily Telegraph columnist Peter Oborne.

A spokesperson for Blair dismissed the letter on the grounds that it had been drawn up by an "alliance of hard-right and hard-left" whose members are "viscerally opposed" to the former prime minister.

The letter, written before Friday's seventh anniversary of Blair's appointment as the representative of the "quartet" on the Middle East, says the former prime minister's achievement as Middle East envoy are "negligible" and he is guilty of seeking to please the Israelis. The quartet consists of the UN, the EU, Russia and the US.

The letter says: "We, like many, are appalled by Iraq's descent into a sectarian conflict that threatens its very existence as a nation, as well as the security of its neighbours. We are also dismayed, however, at Tony Blair's recent attempts to absolve himself of any responsibility for the current crisis by isolating it from the legacy of the Iraq war.

"In reality, the invasion and occupation of Iraq had been a disaster long before the recent gains made by the Islamic State of Iraq and the Levant (Isis). The sectarian conflict responsible for much of the war's reprehensible human cost was caused in part by the occupying forces' division of the country's political system along sectarian lines."

It added: "In order to justify the invasion, Tony Blair misled the British people by claiming that Saddam had links to al-Qaida. In the wake of recent events it is a cruel irony for the people of Iraq that perhaps the invasion's most enduring legacy has been the rise of fundamentalist terrorism in a land where none existed previously. We believe that Mr Blair, as a vociferous advocate of the invasion, must accept a degree of responsibility for its consequences."

The signatories say that Blair has failed to achieve any breakthrough as the quartet's representative, though they acknowledge his limited mandate that involves building the governance of the Palestinian Authority.

The letter says: "It is our view that, after seven years, Mr Blair's achievements as Envoy are negligible, even within his narrow mandate of promoting Palestinian economic development. Furthermore, the impression of activity created by his high-profile appointment has hindered genuine progress towards a lasting peace.

"Seven years on there are still over 500 checkpoints and roadblocks in the West Bank. The Gaza Strip, severely damaged by Israel's 2009 bombing, remains in a humanitarian crisis, with 80% of its population reliant on foreign aid for survival. Israel continues to build settlements that are illegal under international law. According to the Palestinian Authority's former Chief Negotiator, Nabil Shaath, Tony Blair has "achieved so very little because of his gross efforts to please the Israelis".

The letter is also critical of Blair's business interests. "Tony Blair's conduct in his private pursuits also calls into question his suitability for the role. Mr Blair has been widely criticised for a lack of transparency in the way he organises his business dealings and personal finances, and for blurring the lines between his public position as Envoy and his private roles at Tony Blair Associates and the investment bank JPMorgan Chase."

The letter is addressed to John Kerry, the US secretary of state; Sergei Lavrov, the Russian foreign minister; Ban Ki-moon, the UN secretary general; and Cathy Ashton, the EU's foreign policy chief.

The retired ambassador Oliver Miles, who noted in 2009 that two members of the Chilcot inquiry into the Iraq war "are Jewish", said: "Tony Blair is the wrong man in the wrong job. The wrong man because he is identified with the war in Iraq; criticism has concentrated on his misreading of intelligence and his lies, but equally important was his failure to plan for the peace, with the result we see today. The wrong job because patching up the Palestine economy sounds good but avoids the real issue, the repression and misery of the occupation; that is what the quartet should tackle. Seven years on it's time to blow the whistle."

Crispin Blunt said: "It's time to end Tony Blair's personal calvary as quartet envoy following his disastrous statesmanship in office on the Middle East. His role as envoy was neutered politically almost as soon as it began, and is now a distraction from the increasingly desperate need for a comprehensive peace deal."

George Galloway said: "I have begun the process of parliamentary impeachment of Tony Blair. The House of Commons will vote on that later this year. His position is collapsing along with the state of Iraq he helped destroy. His tenure as Quartet envoy is now untenable"

A spokesperson for Blair said: "These are all people viscerally opposed to Tony Blair with absolutely no credibility in relation to him whatsoever. Their attack is neither surprising nor newsworthy. They include the alliance of hard right and hard left views which he has fought against all his political life. Of course he completely disagrees with them over the Middle East. He believes passionately in the two state solution but also believes that can only be achieved by a negotiation with Israel."

The spokeperson said of Blair's role as the Middle East envoy: "The truth, and anybody who knows anything about the situation in respect of Palestine knows this, is that transformational change is impossible unless it goes hand in hand with a political process. There was hope that this could progress with the recent US led talks which were underpinned by a hugely ambitious economic plan spearheaded by Mr Blair."

The spokesperson said of the criticisms of Blair's business interests: "Mr Blair has done no work for JP Morgan in the Middle East – he is the chair of their International Advisory Council – where he provides advice on global political issues."