Nick Clegg has vetoed Theresa May's rewritten "web snooper's charter" plan, killing off the home secretary's last remaining hopes of getting any communications data legislation in the Queen's speech.

The Liberal Democrat leader told David Cameron and Theresa May on Wednesday that he could not support the home secretary's latest proposals to monitor internet and social media use because they were unworkable and disproportionate.

The decision provoked delight within his party and among privacy campaigners: "This is truly an immense moment for any liberal, and every Liberal Democrat," said Julian Huppert, the party's backbench home affairs spokesman.

"These plans were based on scant evidence, scaremongering and a disregard for our personal lives; treating everyone as a suspect and our online activity as 'fair game'," he said. The announcement was also hailed by the party's president, Tim Farron.

The privacy campaign Big Brother Watch tweeted: "The snooper's charter is dead, long live Clegg."

Conservative critics of the legislation welcomed the decision. The Tory former shadow home secretary David Davis said Clegg had done the right thing. "This draconian law would have been a massive, unnecessary extension of the state's power," he said. "It would have trampled all over the privacy of innocent people without improving our security one jot."

Davis said it raised the question as to why the Home Office spent more than £400m developing the proposals when it clearly never had enough support within government to go ahead.

May has fought hard for the legislation, designed to fill a growing gap in the ability of the police and security services to access records of the web and social media activity of serious criminals and terrorists.

The deputy prime minister sent the original legislation – the draft communications data bill – back to the drawing board after insisting it was first scrutinised by a committee MPs and peers.

The committee's withering verdict described it as "overkill", complained that it "trampled on the privacy of British citizens" and said its cost estimates were "fanciful and misleading". They did, however, agree that new legislation was needed to plug the gap caused by rapid changes in technology.

The revised proposals tabled by May offered significant concessions but did not include movement on access to blogs or people's internet histories, or on requiring British phone and internet companies to intercept data from overseas providers.

Both parties campaigned on general election promises to roll back the surveillance state and May's proposed legislation did not form part of the coalition agreement, so Clegg's veto is not necessarily a coalition-breaker.

Clegg announced the decision on his weekly LBC radio phone-in programme. "There is work that clearly needs to be done on issues where I think most people would reasonably think you do need to keep up [with] the technology," he said.

"[But] the full-scale approach of basically saying you're going to scoop up and hold huge amounts of data for instance, literally recording every website that you visit and everybody visits so you've got this great treasure trove of data which you can then dip into if you need to, I don't think the British public would support that, I don't think it's in many ways workable and I don't think it's necessarily proportionate."

He said a blanket record of websites visited and communications on social media was "not going to happen with Liberal Democrats in government", and referred to the necessity of obtaining a balance between security and liberty.

Clegg's only concession during the interview was that the government should look at whether there should be an internet protocol (IP) address for every device, which police and security services have lobbied for. Further work is expected on how new legislation might be introduced to deal with the limited problem faced by the police in matching IP addresses with individual mobile phones or computers.

Clegg later told party members in an email: "In the coalition agreement we committed to 'ending the storage of email and internet records without good reason', moving away from Labour's database state. But this proposal wouldn't meet that test: expanding the collection of personal data without a solid justification for doing so, and without allaying concerns about the workability of these changes.

"There is always a careful balance to strike between security and individual liberty and I have always agreed that we must help our law enforcement agencies keep up with the challenge of policing in the internet age – like the technical issue of what to do when there are more mobile devices with not enough IP addresses to go round. But the idea of a wider bill didn't get the balance right. It would have been neither proportionate nor workable and would not have enjoyed widespread support in Parliament or across the country at large.

"The police provide an invaluable service and we should give them our support to get on with fighting crime at every level. But the proposals on the table were not the right solution and will not go ahead."

The prime minister's spokesman was reluctant to be drawn on the developments, or whether legislation in this area would feature in the Queen's speech on 8 May. He suggested discussions were ongoing, but appeared to acknowledge that a problem had developed. He said: "The reality of technological change has not gone away. These are sensitive issues around this and discussions are continuing on how progress is to be made."

The internet industry was left seeking urgent clarification from Whitehall on whether or not the current proposals would be brought forward.