Plans to roll out the controversial badger cull pilots nationwide across England have been dropped by Owen Paterson after a damning independent report found the shoots were not effective or humane.

The two pilot culls, in Gloucestershire and Somerset, will continue with improvements recommended by the independent expert panel (IEP), including more and better-trained marksmen. But plans to start badger culls in 10 other areas have been abandoned, the environment secretary announced on Thursday, telling MPs he was taking the responsible approach.

"This disease is the most pressing animal health problem in the UK," Paterson said, noting that 26,600 cattle were slaughtered in 2013 and that the disease had cost taxpayers £500m in the past decade. But he accepted that "on effectiveness … the culls did not make as much progress as we hoped." He said the cull operators had often faced a "disgraceful amount of intimidation from some of the more extreme protesters".

The shadow environment secretary, Maria Eagle, said the abandoned roll-out was a humiliating climbdown for Paterson. "Consistent with his inept handling of this shambles he has put prejudice before science, secrecy before transparency, conflict before consensus and posturing before good policy," she said.

Prof Rosie Woodroffe, a leading badger expert who conducted a landmark decade-long trial of badger culling, said even the two pilot culls should be halted. "The pilot culls performed so poorly in effectiveness and humaneness, I would stop and invest in something more promising," she said.

The culls, aimed at curbing the rise of tuberculosis in cattle, were dismissed by senior scientists as "mindless" before they started and have provoked huge public opposition since, and led to ministers losing a vote in the House of Commons. The night-time shoots failed to kill enough badgers in the allotted time, which scientists warned could lead to escaping badgers spreading TB more widely and increasing it in cattle.

The IEP report, published on Thursday, revealed the pilot culls had failed even more comprehensively than previously thought. New, more robust estimates of the proportion of badgers killed within the initial six-week limit found that fewer than 39% of those in Gloucestershire had been shot and fewer than 48% in Somerset – far short of the 70% minimum and lower than earlier estimates from Paterson's department. Risky extensions to the cull periods made little difference, according to Woodroffe.

The IEP report is also damning on the humaneness of shooting free-running badgers at night. It found it was "extremely likely" that between 7.4% and 22.8% of badgers were still alive after five minutes and therefore at risk of "experiencing marked pain", above the 5% maximum allowed. The experts also concluded that fewer than half the badgers were shot in the recommended target area.

Paterson said he accepted the IEP recommendations to "improve the accuracy and field-craft" of shooters. He added that if the free-shooting of badgers could be "perfected", he still wanted to see the culls rolled out in the future. Paterson previously said he wanted 40 culls across the nation.

Opponents of the cull have argued that vaccination of badgers and cattle is a better strategy and Paterson said: "I am proposing a scheme for [badger] vaccination projects around the edge of the most badly affected parts of the country, in an attempt to create a buffer zone of TB immunity to stop the disease spreading further." He also said large-scale field trials of cattle vaccines were being designed, but said a usable vaccine was many years away.

Opponents have also argued that stricter testing and controls on cattle movements are the key to cutting TB. In Wales, where a planned badger cull was abandoned, the number of cattle slaughtered has fallen from 11,671 in 2009 to 6,102 in 2013, a 48% drop, following more stringent testing. The number of cattle slaughtered in Great Britain fell by 15% in 2013 following some new controls being introduced in England.

The campaigner and Queen guitarist Brian May said: "I am disgusted that David Cameron and Paterson insult us all by continuing this spectacular failure."

Mark Jones, at the Humane Society International UK, said: "Whilst the abandonment of the planned badger cull roll-out this year is a welcome U-turn, it is utterly indefensible that the government is carrying on with its discredited cull in Gloucestershire and Somerset. [Those] farmers are being led down a failed path and Paterson is leading the way like the Pied Piper."

The RSPCA's David Bowles said: "We are delighted [Paterson] has started to listen to the strong feelings of the public, their MPs and the scientific evidence that the culls were ineffective and inhumane."

But Meurig Raymond, president of the National Farmers Union, which has strongly backed the cull, said: "As pilots, there was always going to be the potential to make improvements as a result of knowledge gained. TB remains a terrible disease for cattle and cattle farmers where it is persistent. It is hugely important that any cattle controls go hand in hand with measures to tackle the disease in badgers and culling must play a part in that."