A cabinet minister has called TV presenter Alan Titchmarsh a "complete muppet" for criticising the government's countryside policies.

The environment secretary, Owen Paterson, accused the gardener of not paying attention to what was going on.

The rebuke came after Titchmarsh questioned the response to ash dieback disease and warned that the Conservative party had lost its roots in rural areas.

"Time was when the Tory party was the party of the shires and understood how the countryside works," he said at the launch of the 100th Chelsea flower show last month.

"There seems to me very little investment in rural areas and the countryside. We have to look after agriculture and horticulture: growing things. This is how we feed ourselves."

The former presenter of Gardener's World said the horticultural trade warned of ash dieback in 2009, but nothing was done to stop imports of the species.

He said ministers' advice to wash boots, dogs and children after visiting a woodland would have "minimal effect" when the disease is already widespread. "I think expecting people to wash their dogs, boots children after a woodland walk is like sticking a plaster on a broken leg.

"This is fungal-led and wind-borne. It is here now. It is a matter of watching and waiting. We need to cull where possible when we find it, monitor and look out for resistant strains. We also need more stringent rules on imports."

But interviewed in the Daily Telegraph on Saturday, Tory MP Paterson – who was brought up in Shropshire and is a keen horse rider and huntsman – said: "He's a complete muppet, missed completely everything we're doing."

Paterson set out plans on Thursday aimed at controlling Chalara fraxinea, including keeping the ban on the import or movement of ash trees in place.

His department is also considering appointing a "tree tsar" – a chief plant health officer equivalent to the chief veterinary officer who leads the response to animal disease threats such as foot and mouth and bluetongue. But the measures were criticised by the National Trust as "limited and weak", too focused on minimising costs and "surrendering the British landscape to this disease".

The ash dieback fungus was first identified in the UK in February 2012 in a tree imported from the Netherlands to a nursery in Buckinghamshire. It has now been found at 136 sites linked to imported plants and a further 155 sites in the wider environment, which government scientists think were infected by wind-blown spores from continental Europe. The disease has devastated ash trees in many countries including Denmark, where 90% have been infected.