John Gummer, now Lord Deben, says left and right should worry about terrorism being used as an excuse to curtail freedom

A Conservative peer and former cabinet minister has attacked the UK media's "lackadaisical" response to the US whistleblower Edward Snowden and called on "defenders of liberty" to speak out against invasion of personal freedoms by the intelligence services.

The former Tory party chairman John Gummer, now Lord Deben, said the revelations should be a cause for concern "from right to left" as spying agencies too easily use terrorism as an excuse to invade civil liberties.

His intervention comes after David Cameron and Andrew Parker, the head of MI5, criticised the Guardian for endangering national security by publishing stories about mass surveillance of internet users based on Snowden's leaks.

The reports reveal GCHQ has been tapping internet cables since 2008, potentially allowing the organisation to see the emails and web activity of millions of users. While there has been public outrage in the US and Germany about the extent of spying on internet users, the issue has had less political attention in the UK.

Deben said the UK had a "duty of guardianship" to its citizens and warned it was dangerous to accept the word of the intelligence agencies.

"If you look back into history, the times at which people lose their freedoms are always times when the loss of freedom is excused by national emergency or by the need to fight terrorism," he said. "It means that whenever that's the argument used, people who believe in freedom should be extremely quick to stand up.

"I am very surprised at the way in which the press in Britain has been so lackadaisical and not seen that there are issues here of huge importance. At a time when the majority of the press are making a great deal of fuss about what they see to be a threat to press freedom, here is an issue of freedom that I'm very surprised they have not caught on to and recognised. Who guards the guardians? It's absolutely crucial. This is the moment in which it should have been an issue from right to left."

Deben, who is also an independent adviser to the government on climate change, said there was a serious political debate to be had about how far surveillance should be allowed to go.

"You can't just hide it by saying 'well, we live in a world threatened by terrorism'," he said. "Are we going to allow the terrorists to remove the very freedoms we defend? I'm not blaming anybody, I'm merely saying this ought to be a real national debate and people who are involved in the very good business of surveillance, which is a very necessary part of a free society, ought to know that what they do is subject to very real concerns by people outside."

He spoke out after Sir Malcolm Rifkind and his joint intelligence committee heard evidence from the heads of the intelligence agencies MI5, GCHQ and MI6 in public for the first time. The committee decided to broaden an existing inquiry into whether the intelligence laws are "fit for purpose" in the wake of the Snowden revelations.

Deben said the public should not assume the spying agencies are acting within proper boundaries.

"My big concern is that we kind of take it for granted and say I'm sure [the intelligence services] are doing their best, I'm sure people wouldn't do anything that's wrong," he said. "I don't think that's how you should do it. They should make sure those who are responsible should know their every action has to be taken in the context that others will know about it.

"They must know there are codes which are really very seriously enforced so you don't overhear the telephone calls of the chancellor of Germany or use your power to force land the airplane of the president of Bolivia because you think somebody whom you fear is on board. That's the other thing that really shocked me. We have to speak out. At a time in which the media is so determined to talk about freedom of the press, this is a freedom the press is supposed to be defending."

Asked about GCHQ's Project Tempora, which collects data from internet users by tapping 200 subsea cables, Deben said there had been an "alarming amount of silence from some who could tell us more".

"The only thing I can do is to say, well look here, freedom means you have to be constantly on your guard against those who use terrorism and the need to defend against it as an excuse for actions which are manifestly unacceptable," he said.

Politicians appear split on the role the Guardian played in publishing Snowden's leaks exposing mass surveillance.

The former defence secretary Liam Fox has written to Alison Saunders, the new director of public prosecutions, about the matter. Referring to reports that some leaked material obtained by the Guardian contained details of GCHQ agents, Fox said in his letter he would like to know whether passing this information "to foreign journalists and bloggers" constituted an offence under the Terrorism Act.

William Hague, the foreign secretary, and Philip Hammond, the defence secretary, endorsed comments made at the recent intelligence and security committee hearing at which Sir John Sawers, the head of MI6, said Britain's enemies were "rubbing their hands with glee" over the disclosures.

But despite Fox's letter to the CPS, the two ministers sidestepped questions about whether they would like to see a prosecution. "Action against any particular media outlet in any particular country, unfortunately, is not going to reverse the damage that has been done," Hammond said on Sunday.

Meanwhile, in a parliamentary debate last week a cross-party group of MPs, including Tory Dominic Raab, Labour MP Tom Watson and Liberal Democrat Julian Huppert, argued there should be greater oversight over intelligence agencies.

A Guardian News & Media spokesperson said: "The loss of classified data was not the responsibility of journalists but of the intelligence community itself. It is only the involvement of global newspapers that prevented this information from spilling out across the web and genuinely causing a catastrophic leak."