Andrew Lansley may not survive as health secretary once the government's "pause" over its controversial NHS shakeup ends, the leader of Britain's doctors has said.

Dr Hamish Meldrum, chairman of council at the British Medical Association, told the Guardian he believed Lansley may have to step down if the health and social care bill underwent major changes.

"I deal in policies not personalities, but I agree that it might be difficult for him to stay on if we see the significant changes to the reforms that we [the BMA] are asking for. But that's not a decision that's mine to make – it's for the prime minister," Meldrum said. "But yes, it's true that his credibility would be badly damaged if the bill was watered down because he [Lansley] is so personally associated with the reforms."

The BMA leader's intervention articulates a growing feeling both at Westminster and in NHS circles that Lansley could lose his job because of the difficulty created for the coalition by the deep unpopularity of his radical restructuring plans for the NHS in England.

David Cameron has publicly backed the health secretary and insisted that the ongoing "listening exercise" on the legislation will lead to many more people understanding and accepting the plans. But the prime minister is frustrated that Lansley has made the NHS once again a problem issue for the Conservatives, so soon after they had neutralised it.

Health policy experts believe that David Laws, the Liberal Democrat who resigned as chief secretary to the Treasury over his MP's expenses soon after the coalition was formed last May, may return to cabinet as Lansley's replacement. It has also been speculated that Philip Hammond, the transport secretary, or Stephen Dorrell, health secretary from 1995-1997 in John Major's administration and current chair of the Commons health select committee, could take over.

Friends admit that Lansley's failure to win hearts and minds for his planned reorganisation has left him exposed. "Andrew's intelligent and thoughtful and, as David Cameron says, no one knows more about the NHS than him," said one. "But he has become too bogged down in the detail of the reorganisation and shown himself to be more of a thinker than a politician, and his lack of engagement over his plans with either health professionals or the public has left him somewhat isolated."

One health expert who is taken seriously in Downing Street said: "The question is: is he too toxic? I can't believe he will stay on. To me it's only a matter of time before he is replaced. Andrew Lansley means well and he knows a lot and he wants the best for the health service, but it hasn't worked".

Lansley is widely seen to have failed to "do the politics" before unveiling his NHS masterplan. "Much of what's in the health bill could command quite wide support, but he didn't warm people up or get the key stakeholders on board," said a pro-reform NHS policy specialist. "In such difficult times for the NHS you need a health secretary who can schmooze, do deals and get people on board and that's absolutely not Andrew Lansley."

Some of those working closely with Lansley on the "listening exercise" say that, despite criticism of his plans from an unprecedented array of bodies and individuals, and the Liberal Democrats, he still does not believe that they need major changes. Allies, though, say that he is "sufficiently flexible to respond to good ideas", is putting considerable energy into engaging with stakeholders during the two-month pause and will clarify and improve his plans while maintaining their key principles, such as handing control of commissioning healthcare for patients from primary care trusts to GPs.

One in three GPs plans to quit the NHS in the next five years and some blame the health reforms, according to a snapshot poll of 576 doctors for the magazine Pulse.

• Join the debate on the future of the NHS on the NHS reforms blog