Home secretary says she is 'determined' to push through 'vitally important' surveillance laws despite opposition from Nick Clegg

A coalition clash over the home secretary's "snooper's charter" legislation has opened up at the highest level, with Nick Clegg bluntly telling Theresa May: "We cannot proceed with this bill and we have to go back to the drawing board."

But the home secretary hit back in a column for the Sun newspaper, making clear she was "determined" to push through "these vitally important laws", raising the spectre of a political showdown between the two parties in coalition over the touchstone issue of civil liberties.

The deputy prime minister's open opposition is fuelled by a storm of criticism from a high-powered committee of MPs and peers who, in a report on Tuesday, describe the proposed legislation's "sweeping powers" as "overkill" and warn that it "tramples on the privacy of British citizens".

The parliamentary scrutiny committee, which includes the former cabinet secretary Lord Armstrong and three former cabinet ministers, says the home secretary's draft communications data bill must be completely rewritten if it is to meet the committee's substantial concerns about its scope, ineffective safeguards, cost and lack of wider consultation. "This bill is dead in its current form," one MP on the committee said.

The MPs and peers describe the Home Office's estimated price tag of £1.8bn over 10 years as "fanciful and misleading".

They also warn in their report that a "request filter" at the heart of a new tracking system of everyone's emails, web use and social media messages is "essentially a federated database of all UK citizens' communications data".

A second high-powered committee of peers and MPs has also criticised May's plans to monitor everybody's web and mobile phone use.

The intelligence and security committee, which is appointed by the prime minister and not parliament, has warned that her draft bill needs more work, more consultation and a coherent explanation of what the data will be used for and the safeguards that will be in place.

The normally cautious committee says it accepts there is a serious problem that requires action but warns that its concerns "must be addressed in advance of the bill being introduced".

The bulk of the ISC report, which is based largely on evidence taken from the security services and the police in private, is to remain secret but a six-page summary published on Tuesday reveals detailed concerns. These include the writing into the legislation of the precise categories of communications data to be collected by the internet and phone companies.

It also highlights concerns over the technical arrangements being contemplated to intercept the traffic data of overseas-based internet companies such as Google and Facebook that may not co-operate with requests from the British police.

They say that the Home Office must demonstrate "due diligence before resorting to the use of deep packet inspection" – known as black boxes – to collect such "third party" communications data travelling across British network from such overseas companies.

May responded by saying she remained "open-minded" about the drafting of the legislation but insisted she remained determined to introduce it before the session ends next spring and get it on the statute book before the next election.

"Labour tried to bring a bill forward previously but they ducked the tough decision in the end. I am certain I am not going to duck that decision," May said.

She added that she hoped to open cross-party talks with Labour to get her draft bill on the statute book without the support of her Liberal Democrat coalition partners.

Writing in the Sun on Tuesday, May says: "Countries across the world are taking action now to help them track paedophiles and terrorists who abuse new technology to plot their horrific crimes. We must not get left behind."

But the shadow home secretary, Yvette Cooper, immediately ruled out the suggestion that Labour could be called upon to help get the legislative plans through parliament. She said the report showed May was "making a complete mess of a very important issue" and "needed to urgently rethink this legislation".

Cooper's reaction means that May faces further delays in her timetable.

The MPs and peers on the committee are demanding another round of consultation before any new bill is produced. May will face a race against time if she is to bow to the almost overwhelming political pressure to completely redraw her legislation and still get it on the statute book before the 2015 general election.

Clegg, who insisted the bill be put through the pre-legislative scrutiny committee, said the report made serious criticisms of the bill, on its scope, proportionality, cost, checks and balances and the need for much wider consultation.

"It is for those reasons that I believe the coalition government needs to have a fundamental rethink about this legislation. We cannot proceed with this bill and we have to go back to the drawing board. We need to reflect properly on the criticisms that the committee have made, while also consulting much more widely with business and other interested groups."

Clegg agreed with the committee that fresh legislation was needed to modernise the capability of the police and security services to track internet and mobile phone use. "But that must be done in a proportionate way that gets the balance between security and liberty right," he said. "Any modernisation of the powers, including possible new legislation, must meet the concerns of the joint committee by having the best possible safeguards and keeping costs under control."

The peers and MPs say that if clause one of the bill that "gives the home secretary sweeping powers to order the retention of any kind of communications data by any communication service provider" was significantly narrowed, and safeguards were put in place to ensure that any new powers were not abused, then a workable bill could be introduced.

"It would allow the security services, law enforcement agencies and a few other public authorities access to the communications data they need to protect and serve UK citizens without trampling on the privacy of those citizens. That is something the current draft bill does not achieve," says the report.

The peers and MPs rejected the Home Office case that the extensive powers were needed to "future-proof" the legislation in the rapidly changing world of technology. They suggest that instead of a blank cheque to require the collection of limitless categories of communications data, each specific type, such as internet protocol addresses or weblog histories, should require specific parliamentary approval.

They also require specific legislative approval for potentially up to 600 other public bodies, including the egg marketing inspectorate, to get access to the data. At present the home secretary is proposing to use her powers to give them access on a case-by-case basis.

The former Tory Home Office minister Lord Blencathra, who chaired the scrutiny committee, said: "The breadth of the draft bill as it stands appears to be overkill and is much wider than the specific needs identified by the law enforcement agencies.

"We urge the government to reconsider its zeal to future-proof legislation and instead concentrate on getting the immediate necessities right."

The report says the volume of communications data, which tracks who is talking, texting, messaging and tweeting whom, when and from where but not the content of the messages, has grown considerably since original legislation was passed in 2000.

There are now more than 500,000 requests a year for such data from internet and phone companies.

"We accept there is a case for legislation which will provide the law enforcement agencies with some further access to communications data, but we believe the draft bill pays insufficient attention to the duty to respect the right to privacy, and goes much further than it need or should," conclude the MPs and peers.

Julian Huppert, the Lib Dem home affairs spokesman, who sat on the scrutiny committee, said it found the Home Office draft bill to be far too broad, very expensive, poorly evidenced and potentially unworkable. The home secretary claimed originally that "only suspected terrorists, paedophiles or serious criminals will be investigated".

He said: "Even she now accepts that was not true, and data would be kept on all of us."

He told the BBC Radio 4 Today programme the proposals gave the government "huge and dangerous" powers and that the bill could proceed only if the Home Office looked again. He said: "They must consult properly, talk to the experts, do proper costings and have a far more limited proposal.

"It was absolutely ridiculous for the home secretary to ask for a snooper's charter based on political scaremongering, when what we actually need is sensible reforms working with the major tech companies."

James Brokenshire, security minister at the Home Office, said the government was "intent" on pushing the bill through, though he accepted more work needed to be done.

He said the joint committee report acknowledged the need for changes.

"They have accepted the principle of legislation, they have accepted that there is a need for us to make this change and that actually is quite an important point. They are not rejecting, saying there isn't a problem here."

He acknowledged there was a legitimate debate to be had about the balance between security needs and individual liberty.

"We believe that the changes that will meet the substance of these recommendations can be met in a reasonably short order but we know that we need to work this through with the coalition, that there have been reasonable points that have been made … what we want to do is to work through all of these points, recognising that the deputy prime minister has set out his concerns and making sure that the substance of the recommendations from the joint committee are addressed."

He added: "There are always differences in coalition – that's what coalition is about – but we are intent on getting this through because this matters, this is about saving lives and protecting the public and that is our absolute focus on getting that right."