David Cameron has been warned that his attempts to shed the Conservative Party's "nasty party" image are being undermined by organisers of a mass rally against gay marriage.

About 900 people, the majority of them active members of the Conservative Party, will crowd into Birmingham Town Hall at lunchtime today to hear the former Archbishop of Canterbury, George Carey, the former Home Office minister Ann Widdecombe and others give speeches on why there should be no change in the legal definition of marriage to recognise same-sex couples.

The meeting will be ringed by a heavy police presence to prevent anyone getting in apart from party representatives and others with valid passes to the Tory conference. Several organisations, including the public sector union, the PCS, are holding what they call a "Picket the Bigots" protest outside the town hall. They accuse the rally's organisers of wanting to turn back the clock on the rights won by gays and lesbians.

"This is a very unfortunate way to open the Conservative Party conference," Peter Tatchell, the veteran gay rights campaigner said yesterday. "It sends a signal that the Tories are still the nasty party, many of whose members continue to support homophobic discrimination despite the attempts of David Cameron to modernise the party. The average person will view this rally as an insight into unreconstructed conservatism."

But the rally organisers were boosted by a survey of constituency party chairmen published yesterday, which found seven out of 10 wanted Mr Cameron to abandon his promise to change the law during this parliament; and more than half said the party had lost members because of the gay marriage issue.

Lord Carey is expected to tell today's rally: "We can be sure any move away from our traditional understanding of marriage is to put our society on a slippery slope where the unintended consequences could be shocking."

The group have also won the backing of backbench MP David Davis. He told a fringe meeting in Birmingham yesterday: "Gay marriage is an issue for the church, not the state."

Alistair Thompson, an organiser of the rally and former Tory parliamentary candidate, said: "It's complete nonsense to try to say this is about the 'nasty party'. There are a lot of questions the government has failed to answer about the seriousness of redefining marriage.

"I think when they have looked into the implications and the consequences of this, they will start to sympathise with the vast majority of people in the party. The rally resonates with what members of the party, constituency party chairman, and ordinary members of the public have said."

Coming on the first full day of the Tory conference, the rally is an embarrassment for Mr Cameron, whose senior Cabinet allies rallied to his support yesterday. The Chancellor, George Osborne, told Sky News: "I support gay marriage because I believe Conservatives support the institutions of commitment."