The former Conservative cabinet minister Lord Tebbit has said young unemployed people should be made to pull up ragwort from roadside verges in return for benefits.

The 83-year-old Tory grandee made the proposal in a letter to Matt Shardlow, chief executive of a charity called Buglife, which is concerned about the effect of declining ragwort on bees and rare insects.

In his reply to the charity, Tebbit said ragwort was a major problem in his part of East Anglia and proposed it could be weeded out by “Neets” – young people who are not in education, work or training – and “low level criminals”.

He wrote: “I suggest you come to the Norfolk/Suffolk border areas of East Anglia. Landowners who wish to control ragwort face an impossible task when roadside verges are dominated by it to an extent I cannot remember in the past.

“There would be little cost to bring that under control if Neets and low level criminals were required as part of their contribution to the society which finances them, or which they have abused … to uproot this weed.”

Tebbit later told the Guardian: “Given a bit of organisation, they [unemployed young people] would be happy doing something constructive. That’s something constructive for them. It’s appealing, it gets rid of a weed which is a danger to some animals and helps landowners in the cultivation of their land.

“That was my thought that caused me to suggest the idea … in a way it’s a form of national service, of doing something for society in a way in which anyone unless they are physically disabled can participate.”

Asked whether he acknowledged some might find the idea of forced labour in return for benefits controversial, he said: “It’s workfare but I think there are some powerful arguments for workfare and so does [Labour MP] Frank Field for example. It’s not a way-out idea in that sense.

“If you go back to the Beveridge report on which the whole welfare state has been based, you’ll find he took the view that youngsters who had never worked should not receive benefits because they have not contributed anything.

“I am much more modest about this than Beveridge was and I suspect Ernie Bevan might have been on my side in it. I just think a lot of those youngsters want something to do which is constructive.”

However, Chris Bryant, the shadow welfare reform minister, said the comments reflected the “values of the Victorian workhouse” in which out-of-work people were forced to perform demeaning, unpaid labour.

“There’s one weed that I would like to uproot: it’s sitting in the House of Lords. Lord Tebbit’s proposal, which effectively equates being out of work with being a criminal, is both offensive and ludicrous,” he said.

“It betrays the deeply toxic attitude the Tories have towards people who rely on the social security net for any period of their life. Rather than acting to end the scourge of insecure, unskilled, low pay jobs, they think up ever more creative ways to demonise those that they have failed.”

It is not the first time Tebbit has made controversial suggestions about the unemployed. He is famous for suggesting in 1981 that they should get on their bikes to find work.

His stance on ragwort – a plant often sprayed with herbicides by local authorities because of its reputation for killing horses and grazing animals – may also annoy environmentalists.

Shardlow, the chief executive of Buglife, said: “We were surprised that Lord Tebbit suggested that the unemployed and criminals should be forced to pull up ragwort, particularly as ragwort is an important part of our native biodiversity, supports 30 species of insects and helps to sustain the now fragile bee populations that we need to pollinate crops.”

Shardlow said that the poor reputation of ragwort was undeserved and argued that cases where horses and other livestock appear to have been poisoned are the result of poor animal husbandry, not the spread of the plant. He said that while ragwort may be more obvious on roadside verges in some areas, it declined by 39% in England between 1998 and 2007. One of the insects dependent on ragwort, the cinnabar moth, has declined by over 80% in the last 35 years.

Richard Benyon, a former environment minister, was criticised by ecologists in 2011 when he posted a picture on Facebook of himself pulling up the yellow-flowered plant.

Declaring he hated ragwort, the Tory MP said he was “on the warpath for those who let this vile weed spread”, prompting anger from experts who said at least 30 insect and 14 fungi species are entirely reliant on ragwort.