The government is to scale back plans to create a new generation of secret courts after Nick Clegg wrote to cabinet colleagues to criticise the proposals as unacceptable in their present form.

As parliament's human rights committee condemned the courts as unnecessary and potentially damaging to public confidence, government sources confirmed that Clegg wrote to fellow ministers on the National Security Council to demand safeguards.

The deputy prime minister, who warned that he could not sign up to the plans unless changes were made, is confident that the justice secretary Kenneth Clarke will amend his plans.

Clegg told the Guardian there would be parliamentary "stress testing" of separate plans to extend the powers of the intelligence agencies to monitor the public's emails, telephone calls and social media communications. His intervention shows that the Lib Dems are adopting a more assertive approach on civil liberties. The Lib Dem leader wants civil liberties groups to be given the chance to challenge the government proposals on surveillance, which he supports, when draft clauses of a new parliamentary bill are published next month.

Clegg, who told the Guardian that he is acting as a restraining influence on the security services, is understood to have been struck by the damning report on the secret courts by parliament's joint committee on human rights. MPs and peers on the committee dismissed the government's argument for secret justice as being based on "spurious" claims that the current justice system puts national security at risk.

The courtroom secrecy proposals – a response to the public airing of evidence during litigation brought on behalf of Binyam Mohamed and other former inmates of the Guantánamo Bay detention centre – would allow ministers to decide what material could be concealed from the public, the media and even claimants during civil trials. Clarke argues that because some evidence disclosed in those cases was based on material supplied by the CIA, greater secrecy is needed to protect the control principle, which allows governments to determine how their intelligence is disseminated.

Clegg is demanding that the courts are only used in exceptional cases where there are national security concerns; that they complement the current system of public interest immunity; that judges, and not ministers, will decide when the special courts are invoked; and that they will not apply to inquests.

One government source said: "There is a balance to be struck between protecting national security and ensuring that we continue to have open justice. Nick Clegg feels the plans go too far in one direction. The number of cases held in these courts under the current plans would be very small. Nick Clegg's proposals would make this a tiny number."

The justice secretary stressed that no final decision had been made on his plans, outlined in a green paper, which are currently out to consultation. He said the special courts were designed to ensure greater justice because the public interest immunity system means that intelligence cannot be heard at all if it is deemed sensitive to national security. This is intelligence which would risk lives or open up secret methods to public scrutiny if published.

Under Clarke's plans, such evidence would be viewed only by vetted lawyers known as special advocates, who could not discuss with the claimants or their lawyers what they had seen or heard in secret court sessions. The closed material procedures – used in terrorism-related immigrations appeals – would be extended to any civil trial in which government ministers ruled that evidence was too sensitive to be made public, including trials in which ministers are themselves defendants.

The report says the proposals would cause harm to: individuals facing the death penalty in other countries who are seeking the disclosure of evidence while "fighting not only for their liberty but for their life"; public trust and confidence in the government and courts; the principle of open justice; and the media's attempts to report matters of public interest.

The proposals were drawn up by the Ministry of Justice after consultation with MI5 and MI6. Jonathan Evans, the director general of MI5, is said to have been lobbying for a change in the law.

The committee's report criticises the government for making claims for the need for secrecy that it says are too vague and too broad, and for drawing up proposals that it says are disproportionate.

The report says the US authorities appear to be suffering from "a misperception" that UK courts cannot be trusted to ensure sensitive national security material is not disclosed, and that such perceptions should not be addressed through legislation. The report quotes a BBC interview with Lord Macdonald QC, the former director of public prosecutions, who said: "I don't think we should allow foreign intelligence agencies to dictate how we organise our justice system."

The report dismisses ministers' claims that the introduction of closed material procedures would enhance the fairness of courts, because judges, sitting in secret, would have access to all relevant material. The government has produced no evidence to support this claim, it said.

Clarke said his department's proposals were "a commonsense solution to a genuine problem in a very small number of cases". He maintained that the green paper was intended to save the government from paying compensation to claimants whose cases had no merit, adding that "the current system is not working and needs to be reformed".

But the committee's chair, Labour MP Hywel Francis, said he and many others saw the plans as a radical departure from "our longstanding traditions of open justice and fairness", and he was "troubled" that Clarke did not appear to see this.

Special advocates working within the immigration tribunal system had told the committee that closed material procedures were inherently unfair, and the committee agreed. The report repeated a warning by Lord Kerr, a supreme court judge, that "evidence which has been insulated from challenge may positively mislead".

Shami Chakrabarti, director of Liberty, said: "Perhaps this damning report shows parliament finally asserting itself over the increasingly cocky demands of the spooks. First they want private chats with judges to replace open justice; then total access to all our internet browsing and communications. No scrutiny for them and no privacy for us. Is it time to ask who runs Britain?"