Local Conservative chiefs have issued a series of warnings to David Cameron about declining membership in some areas before next year's election, with "lethargic and depressed" supporters, a continuing "mass exodus" over gay marriage, and the loss of 170 members in the prime minister's own constituency.

Tory sources insist their core membership will be roughly stable at more than 134,000 when full-year figures are published later this month, but low morale in some local parties is revealed in their annual reports for 2013.

They show an average drop in full membership of about 10% across constituencies that have published their figures over the last two years. Labour has a membership of about 187,000, while the Liberal Democrats have increased their numbers by about 7% over the last year to more than 44,000.

The worst fall-off for the Conservatives was in Romford – a key battleground for Ukip – where almost 400 people, more than half of the local party, left in the course of a year. In Chippenham, a marginal seat that the Tories will be hoping to take off the Liberal Democrats, the association wrote: "Membership continued to fall in 2013, the ongoing mass exodus over issues such as gay marriage."

It said there were signs that this could be slowing but "from a managerial perspective, the committee found it hard to motivate people in 2013 [and] consequently the Political Supper Club petered out after a few events."

Iain Duncan Smith, work and pensions secretary, saw a third – 160 – of his local full members stop their subscriptions in Chingford and Woodford Green; the local Tatton association of George Osborne, the chancellor, saw numbers fall by 90; the Richmond association of William Hague, the foreign secretary, was down by 115; while transport secretary Patrick McLoughlin's Derbyshire Dales lost a quarter – 124.

In Cameron's Witney constituency, the association saw its number of full members fall by 170 to 1,083, which the party said it was "addressing". It added that it was disappointed to be in deficit for the first time in years but its finances had been affected because the association "was asked to pick up the unforeseen cost of a Westminster Report newsletter on behalf of our Member of Parliament in the sum of £4,371".

Meanwhile, in the Kenilworth and Southam constituency of the justice minister Jeremy Wright, the association warned its members had become "lethargic and depressed" because of issues with the boundary review, and were also "affected by the same-sex marriage bill, HS2, 'gasification' and the prospect of a new settlement".

In Theresa May's Maidenhead seat, the local party noted: "We lost 16 members because they disagreed with certain government policies and so we are still nearly 20% down from three years ago." Health secretary Jeremy Hunt's South West Surrey said: "Subscriptions declined partly through attrition and partly due to disagreements with policy."

At least 15 associations attributed the drop in members to the prime minister's same-sex marriage legislation or unspecified unhappiness with national decisions, although many did not give a reason.

The Conservative party revealed official membership figures for the first time in years last September, indicating that it has halved since Cameron took over as leader in 2005. In contrast, Ukip is the party with the fastest-growing membership, having more than doubled over the course of this parliament to 39,000.

The annual reports provide an opportunity for local Conservative associations to review the last year. In the Reigate constituency of Crispin Blunt, where there was an unsuccessful effort to deselect him, they noted that the local party enjoyed "summer drinks and duck racing" at his home. It did not give a reason why, but Philip Hammond's Runnymede and Weybridge constituency noted: "The association did not succeed in distributing an "In Touch" leaflet this year."