In the small farming settlement of Palong in rural Malaysia, the EU is a dirty word. A few months ago, these men – mostly in their sixties, their skin brown and weathered from years working the land – had never even heard of the European parliament: but now their livelihoods are at risk because of it.

The crop they harvest is palm oil, one of Malaysia’s biggest exports worth $20bn a year. The European Parliament has voted overwhelmingly to ban the use of palm oil in all European biofuels by 2020, citing environmental concerns. If it is approved by the European Commission, the impact on the 650,000 smallholder palm farmers in Malaysia will be devastating.

“I have spent all my money on the palm oil farm, I have recently planted new trees that will last for the next 25 years, and my whole family relies on this. It’s how my kids afford to study,” said Hussain Mohamed, 66, who has had his 10-acre plantation for over 30 years, and supports his entire family on the income.

“Our whole community here totally depends on palm oil. But now we hear about the EU biofuel ban and this came as a shock to us, we still do not understand it and everybody is scared of what is going to happen to us in two years time. We have all signed petitions,” he said.

Shaking his head angrily, he said the uncertainty had already affected the selling price of palm oil. “I am proud of my farm, I really take care of the trees every day. How can they say I am bad for the environment?”

In the past few years, campaigners have gathered mountains of damning evidence that the growing demand for palm oil from the EU and China has fuelled massive illegal deforestation in Indonesia and Malaysia by governments and big corporations, forcing several species to the brink of extinction.

It’s the same colonial attitudes, the white man imposing their rule on us from afar.

Yet for Malaysia’s smallholder farmers, many of whom were rescued from poverty when the government’s land authority, Felda, gave them 10 acres of land to harvest palm oil in the 80s, the allegations of environmental destruction are baffling. They account for 40% of Malaysia’s palm oil output and yet none engage in any land-grabbing, the slash and burn or deforestation practices that were pivotal proponent for MEPs voting to ban palm oil in biofuels.

Lehan Samiran, 62, who has had his 10 acres since 1981, shook his fists angrily. He has three children in university who are all supported by the income from his palm oil farm. “Why are the EU doing this to us?” he said. “They have not thought about the farmers, about the human price. The statement ignored us. They are so far away they do not care. We are not harming the environment, we are looking after it. I am very angry, everybody is very angry. How can we not be, this will affect our rice bowl.”

Workers collect palm oil fruit in Sepang, outside Kuala Lumpur. Photograph: Samsul Said/Reuters

The EU is Malaysia’s third biggest palm oil customer, and much of it is imported to make biofuels. The farmers and Felda also fear the resolution would be just the first step towards banning palm oil outright in the EU. Iceland supermarket has already said it will not use palm oil in its own- brand goods.

The proposed ban in biofuels, which is part of a revision of the EU’s renewables directives, has already prompted the rumblings of an international trade dispute, with Malaysia and Indonesia threatening retaliatory sanctions against EU goods in response, and Malaysia reportedly threatened to cancel defence contracts worth £5bn if the UK backed a palm oil ban in Brussels.

MEP Kateřina Konečná, who was the first to bring the motion to ban palm oil to the European parliament said that while she understood the plight of the small farmers, “these smallholders are a slice of big national and international corporations who are responsible for deforestation.”

But a spokesperson for Felda, questioned why palm oil had been singled out over other vegetable oils used to make biofuels - many of which, like rapeseed, are grown in EU countries - calling it “crop apartheid” and condemned the EU for disregarding “the hundreds of thousands of lives it will destroy”.

“It’s the same colonial attitudes, the white man imposing their rule on us from afar.” he said. “If the EU respect Malaysia as a sovereign country and as a partner in development and trade, they should not put this unfair restriction on us, and instead work with us on environmental concerns. Freezing us out is wrong.”

He was echoed by Dato’ Haji Aliasak Bin Haji Ambia, the President of the National Association of Small Holders Malaysia. “Palm oil has allowed the rural poor in Malaysia to develop our own land, lift ourselves and our families out of poverty, and take control of our own economic destiny,” he said.

With the Malaysian election coming up on 9 May, this is also a pivotal issue. Felda farmers are a key support base for the ruling coalition, Barisan Nasional, and prime minister Najib Razak has made promising to defend palm oil one of the talking points of his campaign.

Greenpeace this week published further evidence that palm oil has driven further forest clearance in Papua, Indonesia, tainting its reputation further. Yet a ban does not necessarily solve the environmental problems.

The market for biodiesel in the EU is still growing, with a large percentage currently made from palm oil. Should the ban be implemented, other vegetable oils will have to be be used in its place, such as soybean oil, which has also been proved to be environmentally destructive, responsible for mass deforestation of the Amazon rainforest.

While oil palms occupy 9.2 million hecatres of agricultural land and produce 31.8% of global oils, the soybean and rapeseed crops would require 10 times this amount of land to produce the same yield.

There are also wider political ramifications of the EU ban. Banning palm oil in biofuels is likely to also weaken the EU’s influence in southeast Asia, and hand even more more power to China – the biggest palm oil customer in the world – at a time when the EU has expressed concern about increasing Chinese domination of the region.

But for the country’s farmers, this is simply about sustaining the food in their rice bowls. Nur Harun Muhammad, 64, stood among the palms that he has tended to every day for the past 35 years and smiled widely.

“See, not poison, not bad,” Nur said, showing off the bright yellow palm oil that had oozed out onto his hands. “When you go back to England please, please tell them palm oil is good.”