Pressure on the culture secretary, Maria Miller, has been ramped up by a poll suggesting a large majority of voters think she should be dropped from the cabinet, stripped of her responsibility for press regulation and thrown out of the House of Commons over her expenses.

A Survation poll for the Mail on Sunday found that almost three-quarters of voters (73%) thought her 32-second apology to the House of Commons on Thursday was inadequate, and similar numbers (75%) felt David Cameron was wrong to offer her his support.

Some 78% of those questioned said she should forfeit her cabinet post as culture secretary, 66% said she should lose powers over press regulation and 68% said she should be "sacked" as an MP – something that is not currently possible, as the government has yet to introduce the power of recall promised in the 2010 coalition agreement.

Meanwhile, the Sunday Telegraph quoted an unnamed "senior minister", speaking on condition of anonymity, as saying: "In my view she has clearly behaved in a way that is incompatible with what she should be doing as a cabinet minister. The decision to keep her on undermines the prime minister because he has talked about a new kind of politics."

Miller's apology came after a cross-party panel of MPs overruled parliament's standards commissioner, Kathryn Hudson, who recommended after an inquiry that the culture secretary should repay £45,000 in expenses for a house she shared with her parents.

The Commons standards committee instead decided she needed to hand back just £5,800 and say sorry for failing to co-operate fully with the inquiry.

The watchdog in charge of MPs' expenses said it was time for the House of Commons to give up the power to police itself over standards and ethics, warning: "MPs marking their own homework always ends in scandal."

Sir Ian Kennedy, chairman of the Independent Parliamentary Standards Authority (Ipsa), told the Sunday Times: "We have made great progress in cleaning up the problems of the past. To avoid further damage to parliament in the future, it should have the confidence to give away powers in regulating itself and see that independent regulation is the best, most transparent way forward.

"MPs marking their own homework always ends in scandal. It happened with expenses. It will happen with standards investigations too. Ipsa has shown that independent regulation of parliamentary behaviour can work. Our reforms have cleaned up the system."

Kennedy said MPs should "learn a lesson" from the independent system of regulation of expenses introduced in the wake of the 2009 scandal and ensure that Hudson too is "given the freedom to carry out her work and not have her wings clipped by MPs".

Letters released following Miller's apology revealed that she told Hudson it would be "irrational, perverse and unreasonable" to uphold the complaint against her and warned that she could go over her head to ask the MPs on the standards committee to intervene.

In another message, she wrote: "In light of the evidence that is before you … to continue to regard this spurious complaint as a serious matter would give it credence it does not deserve and undermine the inquiry process in comparison to issues that really are serious matters."

John Mann, the Labour MP whose complaint sparked the commissioner's investigation, said: "These emails show that Maria Miller bullied and threatened the independent commissioner."

But Grant Shapps said he believed Miller had just been frustrated at the long-drawn-out inquiry. The Tory chairman told Channel 4 News: "I don't want to get into the semantics of which words should have been used. The simple fact of the matter is that Maria has accepted in her statement to parliament that this perhaps could have all been handled much faster.

"I'm sure she was frustrated that it hadn't been and that's why she said she of course unreservedly apologised. This has been now very thoroughly investigated in a huge amount of detail, every aspect of it. It's come to a conclusion … It's happened now, she's made an apology and the prime minister has said it draws a line under it, and that's of course what it does."

He also defended the culture secretary's special adviser Jo Hindley, who was caught on tape telling a reporter investigating the expenses story that she wanted to "flag up" the fact that Miller would be meeting her editor to discuss the Leveson inquiry into press ethics.

Despite the claim of the then Telegraph editor, Tony Gallagher, that this amounted to a threat, Shapps insisted that Leveson had been mentioned only in the context of the doorstepping of Miller's elderly father, who was ill at the time.

Cameron has twice publicly voiced his support for the culture secretary, but on Saturday omitted her from a list of several cabinet ministers whom he singled out for praise in a speech to the Conservative spring forum.

The deputy prime minister, Nick Clegg, declined to come to Cameron's aid, saying: "All the issues to do with her position and indeed to do with the behaviour of her office, alleged or not, is entirely a matter for the prime minister."

There was further embarrassment for Miller when a string of mocking messages was sent out on the Twitter feed of her own Department for Culture, Media and Sport.

The first bogus tweet from the @DCMS account said: "Seriously though guys which one of us hasn't embezzled and cheated the taxpayer?? #FreeMariaMiller."

It was swiftly followed by "@Maria_MillerMP is like a modern day Robin Hood, she robs the poor to help the rich" and then "Is @Maria_MillerMP guilty? We will let the public decide".

A spokeswoman from the DCMS confirmed that the account had been hacked and said they had "absolutely no idea" who was responsible.