The environment secretary, Owen Paterson, will announce on Tuesday that the government is delaying its plan to cull thousands of badgers, probably until next year at the earliest, amid growing concern about the cost and effectiveness of the controversial scheme.

Paterson has been forced to return from an official trip abroad to oversee the U-turn, which represents another setback for the government. It is the latest in a string of embarrassments for No 10 which culminated in the resignation last week of the chief whip, Andrew Mitchell, for swearing at a police officer – prompting Conservative party grandee, Lord Tebbit, to lambast David Cameron's operation as a "dog of a government".

The decision will be welcomed by leading scientists who have expressed severe doubts about whether the cull will work and by animal rights and welfare activists who have continued protesting throughout the long process. The depth of public feeling was also highlighted by a 150,000 e-petition started by the musician Brian May.

The shadow environment secretary, Mary Creagh, welcomed the delay. She said: "We warned the government that this cull was bad for farmers, bad for taxpayers and bad for wildlife. The badger cull showed how out of touch the government is and this delay shows ministers are too weak and incompetent to deliver it."

The go-ahead for the controversial badger cull was given by the Department for Environment, Food and Rural Affairs (Defra) nearly a year ago. Farmers believe a cull is essential to stop the spread of bovine tuberculosis which is leading to the slaughter of many of their cattle; opponents claim the main problem is transmission between cattle and that a badger cull could make matters worse because fleeing badgers spread the disease more widely.

Last year, 26,000 cattle were slaughtered and the disease cost taxpayers £90m, including compensation to farmers.

As the final preparations for the cull were made, a census showed there could be twice as many badgers as were originally thought. Farmers complained this would increase the cost of the cull and they could not afford to foot the bill if required to kill at least 70% – the proportion that scientists say must be achieved for the cull to succeed because escaping badgers would spread TB more widely and increase, not decrease, cattle infections.

Ministers will also have been aware of a tricky week ahead as the emotive issue is scheduled for its first full debate in the House of Commons on Thursday – with the strong chance of a government defeat – and a serious legal challenge has been mounted by the charity, the Badger Trust. It filed a "pre-action" letter over the weekend, the final step before seeking judicial review, citing costs, public safety around the unmarked cull zones and uncertainty over whether the cull would kill enough badgers to be effective.

After reports of tense negotiations over the weekend, Defra is thought to have decided that it could not afford such a risk of failure.

Announcing the delay is a blow to the government: ministers, led by the former environment secretary Caroline Spelman, have spent months insisting the cull could work in the face of bitter opposition.

The government and especially the prime minister's team in Downing Street are already under fire for a series of U-turns, botched announcements and embarrassments.

Along with the drawn-out Mitchell saga, Cameron has created a mess over energy policy, plans for House of Lords reform have fallen apart, a major boundary change is in jeopardy and the chancellor, George Osborne, has had to drop a series of unpopular policies announced in his budget.

Paterson has strongly backed a badger cull since he replaced Spelman last month, but is likely to escape the worst of the embarrassment because he is new to the job and will be seen to have acted decisively when the problems emerged.

The planned cull had suffered a series of recent blows including the discovery that there were up to twice as many badgers in the culling zones as had been expected. That sharply increased the cost of hiring the marksmen required as they were to be paid a bounty per badger killed.

Whitehall sources told the Guardian that spiralling costs and other complications had left farmers wanting to pull out of the cull: "Paterson and No 10 had to persuade the National Farmers' Union to continue with the cull to avoid another U-turn."

On Friday, the NFU president, Peter Kendall, said: "We are working bloody hard to make sure this is deliverable. The latest numbers are making this more challenging."

The government's claim to a "science-led" policy was derided by Lord John Krebs, the architect of a landmark 10-year badger culling trial. He called it "mindless" and signed a letter with 31 other eminent scientists demanding the government reconsider its plan.