It wasn’t very long ago that a banana was just a banana – just a curved, yellow fruit. All you knew, if you bought a bunch in 1986, was that they cost around 97p per kilo. You weren’t told if they were organic or pesticide-free. You didn’t know if they came from Costa Rica or the Dominican Republic. And you certainly weren’t invited to worry about the farmers who grew them – or if their children went to school, or whether their villages had clinics. You just picked up your bananas and walked to the next aisle for your coffee or tea or chocolate, none the wiser about where they came from either, or about the people who farmed them.

Back then, the countries that grew these commodities and many others were still known as the Third World, and the habit of not caring about their farming conditions was a legacy of their colonial past. For centuries, trade propelled the colonial project, and exploitation was its very purpose. The farmers of Asia, Africa and South America were forced to raise the crops that the empire’s companies wanted, to work the crops in abject conditions, and to part with them at ruinously low prices. In the last century, the empires melted away but the trade remained lopsided – with the imbalance now rationalised by the market, which deemed it “efficient” to pay farmers as little as possible. In the 1970s, a Ghanaian cocoa farmer often received less than 10 cents out of every dollar his beans earned on the commodities market; as a proportion of the retail price of a chocolate bar, his take was smaller still. Child labour was common. The chocolate companies prospered and their customers shopped well; the farmers stayed poor.

Then, in the late 1980s, you began to hear more about these farmers, encountering their stories on television or in newspapers or even on the labels of the packages you bought. The reasons were manifold. Environmental awareness was on the rise. The prices of some commodities were crashing, placing agricultural incomes in even more acute peril than usual. There had already been small groups pushing for more equitable trade: “little do-good shops scattered in cities around Europe, selling products … bought at fair prices directly from small producers abroad”, as one pioneer described it. By the early 1990s, these disparate initiatives began to coalesce into a larger international struggle to radically reform our relationship with what we bought. Trade had long been unfair by design, but now there was a growing movement to make consumers care about that unfairness, and even to help rectify it.

The crown jewel of this movement was Fairtrade International, an umbrella body formed in 1997 out of various national chapters that had sprouted over the previous decade. Fairtrade was founded on the conviction that consumers could make the marketplace more moral. The spine of Fairtrade’s philosophy has always been price: simple, clean, the kind of measure that economists like to deal with. If companies pay farmers equitably, Fairtrade believes, other benefits cascade out as well. Farmers can hire adult workers, rather than employing children; they can send their kids to school, and buy medicines; they can improve the yields of their farms by using better fertilisers. Producers must meet a number of standards to qualify for Fairtrade: rules about labour conditions, for instance, or waste disposal. But for companies, the core of their ethical responsibility towards their producers is defined by price.

So Fairtrade works by forming a kind of “virtuous triangle” of ethical business. It recruits farmers and farming cooperatives as members, asking them to meet its standards; periodically, Fairtrade sends inspectors to these farms around the world, ensuring they are still compliant. At another vertex of the triangle, Fairtrade enlists companies to pay a minimum price for commodities from these member farms if market prices plunge, and offers to certify products made from such ethically sourced commodities. The final corner is the customer, who can, with a little counsel, be galvanised to shop consciously, and to buy Fairtrade-certified products even if they cost a few pence more.

The Fairtrade certification mark has become widely familiar: a green-blue-black logo, resembling a yin-and-yang with a tear in its middle, soothing the consciences of shoppers the world over. Roughly $9bn worth of Fairtrade products were sold in 2017, their raw material sourced from 1.66 million farmers. Around 2,400 companies, including multinational giants such as Tesco and Marks & Spencer, pay licence fees to their national Fairtrade chapters to use the mark on their products. Fairtrade has become a byword for ethical consumption; 93% of British shoppers now recognise its mark.

But its two real triumphs are in the realm of ideas, not numbers. First, and most obviously, Fairtrade challenged the entrenched model of the commodities business – the belief that a farmer’s lean income was just an unavoidable reality of trade economics. Second, somewhat more stealthily, it cemented the notion that the modern corporation would be ethical if only someone held up a lamp and showed it the way. Capitalism didn’t have to be feared; the market would figure out its own checks and balances, through labelling agencies such as Fairtrade and Rainforest Alliance, without having to be regulated by any higher authority. Persistent and egregious inequality could be solved by deft pleats and tucks to the garb of trade, rather than by a full reconsideration of its fabric and seams.

Now, though, Fairtrade’s success in helping to build a world of heightened expectations of sustainability is perversely paving the way to its own demise. Companies are losing faith in labels such as Fairtrade – losing faith in their ability to secure the future of farming and the future of commodities that drive corporate profit, but also losing faith that these independent stamps of sustainability carry any value at all any more. Instead, the world’s giant food multinationals are taking matters into their own hands – setting up their own in-house certification programmes, appraising their own ethics to their own satisfaction.

Although the fate of Fairtrade is not necessarily the fate of fair trade, the former has kept the latter alive as an object of aspiration. A weakened Fairtrade portends the enfeebling of the very idea of fair trade. This poses a serious problem, because Fairtrade relies on companies to constitute one half of its virtuous bazaar. Without the buyers, the Fairtrade bazaar folds up, and companies return to their own devices, to pursue only what’s best for themselves. Increasingly, companies are finding it easier and more expedient to produce their own certifications, said Trishna Shah, a Euromonitor analyst who tracks ethical labels: “A big change is happening.”

What we’re looking at, in short, is something like a corporate capture of fair trade – and it comes at a time when agriculture is already in crisis. The world’s population expanded from about 4.4 billion in 1980 to more than 7 billion by 2011, but the proportion of people employed in agriculture shrank by 12%. Globally, the average age of a coffee farmer is 55. No one from a younger generation seems to want to grow crops any more. The work is difficult and, in the developing world, nearly always unrewarding. And the coming decades, with all the challenges of a changing climate – waves of heat, virulent pests, droughts and floods – appear grimmer still. The lives of farmers are already poised to get worse. A move towards unfairer trade will only push them back towards the exploited miseries of the past.

The single most damaging blow to Fairtrade came two summers ago, and it was delivered by Sainsbury’s, which once proudly boasted that it was the world’s largest retailer of Fairtrade products. In May 2017, without any warning, Sainsbury’s broke the news that its own-brand teas would no longer be certified by Fairtrade.

“It was a bombshell,” Paul Tiony said. He had travelled to Nairobi that month, from his tea farm in Nandi Hills, in western Kenya, to take part in a half-day workshop run by Fairtrade Africa. Other tea farmers, from Malawi, Uganda, Kenya and Rwanda, were there as well, at the Pride Westlands hotel; so was a team from Sainsbury’s. The company called a meeting at the end of the workshop, and Tiony grew excited. Sainsbury’s must have some kind of good news, he thought. But after gathering almost 50 workshop participants in a conference room of the hotel, Sainsbury’s revealed that its in-house tea brands were abandoning Fairtrade. Instead, an executive said, it was piloting its own ethical label – and calling it “Fairly Traded”. Later, Fairtrade estimated that these changes would affect nearly 250,000 farmers and workers.

Among these were the 8,000 members of Sireet, the tea-farming cooperative to which Tiony belongs. Sireet’s produce is certified by Fairtrade, and Sainsbury’s used to be, by far, the largest buyer of Sireet’s Fairtrade tea. So Tiony was dumbstruck at the shock announcement. “Even Fairtrade Africa officials didn’t seem to know much about it,” he recalled. “They were asking: ‘Why are you doing this?’” Someone inquired: “Are you moving out of Fairtrade because of Brexit?”

Facebook Twitter Pinterest Human tea bags protest outside Sainsbury’s AGM. Photograph: Andy Aitchison / Oxfam

Sainsbury’s had no real answers, Tiony said – but it seemed to be doubting that Fairtrade was working as promised. One Sainsbury’s executive singled out one of the central principles of Fairtrade’s model: the so-called “premium” above the minimum price that companies must pay, and which cooperatives must use to build schools, or run clinics, or improve their communities in other ways. Sainsbury’s was concerned that the premium was being poorly used, and that it was money wasted. “But just a few days earlier,” Tiony said, “someone from Sainsbury’s had come to look at all our projects.” The farmers tried to argue with the executives; the executives insisted they weren’t going to change their minds. “The meeting ended just like that.”

The announcement kicked off an immediate storm. In an open letter, Fairtrade’s tea producers in Africa accused Sainsbury’s of seeking “power and control” over farmers in a manner “reminiscent of colonial rule”. A coalition of British non-profits urged consumers into a weekend of action under the rubric: “Don’t Ditch Fairtrade”. Buy a pack of “Fairly Traded” tea and then give it back, they suggested: “Let the sales assistant know you are returning it because it’s not Fairtrade.” In Parliament, MPs pressed Sainsbury’s to reconsider; one accused the company of lying, and another complained to the Advertising Standards Authority that the name Fairly Traded would mislead shoppers who confused it for Fairtrade. (Sainsbury’s declined to comment for this story. “The launch of Fairly Traded was a bit of a disaster,” a former executive told me with a sigh.)

The Fairly Traded experiment ignited fears of a full divorce, in which Sainsbury’s would abandon the rest of its Fairtrade products – its coffee and sugar; its flowers; the 650m bananas it sells every year – and move out of the Fairtrade house altogether. That would, the Guardian wrote at the time, bring Fairtrade “crashing down”. So far, this hasn’t happened, but Sainsbury’s is still only a couple of years into its experiment with Fairly Traded. The divorce could still come.

The departure of Sainsbury’s wasn’t a one-off. Around the world, the largest agribusiness companies are quitting independent certification, either because they think they can do sustainability better in-house, or because they see an opportunity to craft standards that fit their own purposes. Soon after Sainsbury’s, the global confectionary giant Mondelēz – whose vast holdings include Cadbury and Toblerone – pulled several of its chocolate bars, including Dairy Milk, away from Fairtrade and into an in-house certification scheme called “Cocoa Life”. Nestlé had launched a similar programme, “Cocoa Plan”, back in 2013; between them, Nestlé and Mondelez control roughly 40% of the British chocolate market. Starbucks has “CAFE Practices”; Barry Callebaut, the Swiss cocoa producer, has “Cocoa Horizons”; US giant Cargill has “Cocoa Promise”; McDonald’s has its own “McCafé Sustainability Improvement Platform”.

Get the Guardian’s award-winning long reads sent direct to you every Saturday morning

All these firms still have a few products that are certified by third parties – by Fairtrade, or by other agencies such as Rainforest Alliance or UTZ. (The latter two merged in January 2018.) But the ratio is dwindling. The great energies of these corporations, said Trishna Shah, the Euromonitor analyst, are being lavished upon their own, bespoke schemes – each with its own little logo, its own definition of fairness, its own explanatory literature on a web page somewhere.

For shoppers, this promises bewildering times ahead at the supermarket: more and more instances of what people in the industry call “label fatigue”. The shelves already crawl with sustainability logos: more than 460 of them on food and beverage packages, and a third of them created over the last 15 years. There are little chromatic decals to testify that tuna is dolphin-friendly; that coffee is bird-friendly; that “wild-collected natural ingredients” are up to FairWild standards; that a slice of salmon is Salmon-Safe and follows Best Aquaculture Practices; that a bottle of wine is LIVE, which means that it fulfills Low Input Viticulture and Ecology benchmarks. One logo promises a carbon-free product, another a carbon-neutral one, and a third a carbon-reducing one. The more labels there are, the less we know about them – about what they stand for, and about how meaningful they are.

In a crowd, there’s room to hide. When Fairtrade began, labels were rare and precious things; when you bought a product with a label on it, you knew what you were getting. In contrast, a company today can launch its certification logo into the large ocean of labels, confident that most shoppers will not know what the logo guarantees, but that it will nonetheless salve their consciences. “I think companies are hoping that label fatigue is an enduring trend,” said Elizabeth Bennett, a political economist who co-edited the Handbook of Research on Fair Trade. “They’re hoping that consumers are tired of learning what 30 different labels in one sector mean, and that we’ll all just think: ‘Any claim of sustainability is an improvement over no claim.’” This makes it easy for companies to resort to the ruse of “greenwashing” – pretending to be ethical without really being anything of the sort.

Several people in the corporate world offered an even more depressing version of this story: companies are sidling away from third-party certifiers because their optimistic project – the idea that the market can be heedful of its own abuses and correct itself – has, in a grand sense, failed. After decades of work, these certifiers have been unable to truly alter the imbalance in global trade; they have struggled to protect farmers, or to arm them for their various social, economic and environmental battles.

The proof is in the dire projections for the future of farming, showing how vanishing agricultural biodiversity, the warming climate and the ageing and impoverishment of farmers are all endangering the world’s crops – which means they are also endangering the supply chains of companies who rely upon farmers in Asia, Africa and Latin America to manufacture chocolate bars, coffee pods, or cotton T-shirts. By launching all these in-house programmes, companies are trying to take matters into their own hands – if only to secure their sources of commodities, and their bottom lines.

The Fairtrade movement was born in 1988, when a Dutch non-profit introduced the first certification for ethically traded coffee. The label was named Max Havelaar, after a seminal anti-colonial novel by the Dutch writer Eduard Dekker, published in 1860. In the novel, Dekker described how the Dutch in Indonesia, having made themselves “masters of the country”, compelled the Javanese farmer to grow sugar and coffee, instead of rice for his family. The government “punishes him if he sells what he has produced to any purchaser but itself; and it fixes the price actually paid”, Dekker wrote. “And because the entire trade must produce profit, that profit cannot be got in any other way than by paying the Javanese just enough to keep him from starving.” Naming the label after the novel was a blunt acknowledgement of the straight line connecting colonial exploitation to corporate exploitation. When the label launched, three Dutch coffee brands signed on to source their beans from a farmers’ cooperative in Mexico, paying a rate above market price that was agreed to be fair. The following year, coffee prices crashed following a breakdown of trade talks: the cost of a pound of coffee on the commodities market fell to rates last seen in the 1930s. Coffee farmers faced ruin. The universe seemed to be emphasising the urgency of Max Havelaar’s mission.

The label was a swift success; within a year, 65% of Dutch consumers had heard of Max Havelaar, and the certified coffee found its way on to shelves at the biggest supermarkets. In tandem with the coffee crisis, Max Havelaar inspired other “alternative trade” initiatives across Europe. In Britain, the Fairtrade Foundation was set up by NGOs and church groups in 1992. By 1997, the various national Fairtrade chapters were gathered into an umbrella body, now called Fairtrade International and headquartered in Bonn, Germany.

In the voluminous literature on its website, Fairtrade sets out the prices it calculates to be “fair”. As of mid-July, the minimum price for a metric tonne of cocoa beans was $2,000; for an 18.14kg box of bananas from Cameroon, €6.40; for 1,000 fresh coconuts from the Windward Islands, $112. But Fairtrade doesn’t stop with ensuring these minimum prices; it also demands that producers be paid an extra premium – $11 for the Windward Islands coconuts, for instance – and that these premiums be spent to benefit the community.

Facebook Twitter Pinterest Fairtrade cocoa farmers in Ghana, Africa. Photograph: Karen Robinson/The Observer

Fairtrade insists that producers make these investment decisions democratically, which means that farmers have to organise into collectives such as Sireet. There is an undeniable power in Fairtrade bringing these collectives to the negotiating table, said Pauline Tiffen, the editor of the Journal of Fair Trade. “The tea industry used to be so hierarchical until quite recently that it was like something out of Dickens,” she said. “And cocoa still felt like it was operating in the 18th century. Until 20 years ago, if you went to any big trade meetings on cocoa, you’d see no women, no people of colour.”

The debate over whether Fairtrade’s systems truly transform the lives of farmers has raged for nearly all of Fairtrade’s lifetime. Its numbers, in relative terms, are still puny. The 1.66 million farmers in the Fairtrade network form a speck within the 2 billion or so people who make their living from agriculture. The sale value of Fairtrade’s products may add up to $9bn, but the global market for coffee alone is worth $200bn.

A more dramatic sign of Fairtrade’s struggles is that it can’t sell, on its terms, all the volumes of commodities it certifies. In 2016, of all the coffee grown as Fairtrade, only 34% of it could be sold at the minimum price. There were no takers for the rest; farmers had to unload the surplus into the standard “unfair” market, at the lower prices that the market determined. For cocoa, the rate is a bit better, 47%. For tea, it’s much worse, only 4.7 %. There are tonnes of harvest for which Fairtrade fails to find any fair-minded buyers at all.

Even the benefits that Fairtrade promises to deliver to farmers have been called into question. In 2014, Fairtrade entered into a public squabble with a team from Soas University of London, which had examined how well Fairtrade did in shrinking poverty in Ethiopia and Uganda. The study, funded by the British government, found that schoolchildren had worked on Fairtrade-certified farms. The researchers compared wages on Fairtrade-certified tea and coffee estates – always owned by smallholders, in keeping with Fairtrade’s belief in championing the small farmer – with wages on larger, uncertified plantations. Workers on the Fairtrade estates were paid less and often laboured in worse conditions, they concluded. Fairtrade called these “distorted comparisons” and rejected the researchers’ conclusions.

In Kenya, the tea farmers of Sireet have experienced some of the frustrations of Fairtrade, but also the sudden precariousness of life without it. The cooperative came together 14 years ago, when 6,230 farmers raised money to buy nearly 1,000 hectares of land to cultivate their tea. Sireet has been a Fairtrade producer from the very beginning, its chairman, Wilson Tuwei, told me. Sireet’s farmers worked hard to meet Fairtrade’s rules and standards, he said, but even so, the cooperative was never able to sell more than 13% of its harvest at Fairtrade rates.

Even that feels like a bonanza, though, compared to what has happened over the last two years. After Sainsbury’s stopped its purchases, Sireet’s sales of Fairtrade tea plummeted. “Last year, we were at not even 1%,” Tuwei said. Sireet’s careful plans started to go up in smoke. “We’ve stopped most of the projects we were funding with the Fairtrade premium. There were dispensaries. We were paying school fees for some children. There were irrigation projects. They’ve all become white elephants now. It has really affected us.”

The rupture between Sainsbury’s and Fairtrade over tea has become the most prominent case of a large company concluding that Fairtrade wasn’t quite cutting it any more. The details of that rupture convey why Sainsbury’s was dissatisfied with Fairtrade, but also why replacement corporate schemes such as Fairly Traded feel flimsy or stifling to those meant to benefit from them.

Two people who have worked with Sainsbury’s told me that the company had been unhappy with Fairtrade for years. When I spoke to these anonymous sources in April, both sounded exasperated at the way Sainsbury’s had handled its exit, but they made the case that its reasons were sound. “We were paying these premiums, but it wasn’t clear where the money was going. Fairtrade isn’t good at keeping tabs on it,” one of them said. “It wasn’t always going to medicines and schools and things like that, as we found through our own investigations.”

Other Fairtrade partners had raised this as well, she added. “We wanted more data … Instead, there was just a systemic inefficiency.” Sainsbury’s, as a corporation, was constantly pressured to be transparent, she said. “But it turned out that Fairtrade wasn’t quite as transparent. That was frustrating.” The other Sainsbury’s source confirmed this. “If you wanted to find out what a cooperative had done with a whole wad of Fairtrade premiums, or what the impact of that had been over the years, nobody could tell you. Or you’d get some anecdotal stuff.”

A Fairtrade spokesperson told me, in response to these claims, that the organisation wanted farmers to spend their premiums the way they wished, setting their own goals and monitoring their own progress. She added that, since April, Fairtrade has “strengthened the requirements on Fairtrade premium use, management, and reporting, in order to promote best practices and increase transparency”. Now, if a cooperative’s annual premiums exceed $150,000, it must hire an external auditor to inspect the way it accounts for the money.

Facebook Twitter Pinterest UK supermarket with Fairtrade bananas. Photograph: Sean Spencer/Alamy

Even as Sainsbury’s was stewing over the premiums it was paying out, the second Sainsbury’s source told me, it had begun to “rain down with standards” over the past decade. Companies and non-profits introduced a fleet of new labels and certifications, their names all seeming to blend hazily into each other. Pasture for Life. New Forest Marque. Food Made Good. Soil Association. Free Range Dairy Pasture Promise. LEAF Marque. “Then Fairtrade starts to look just like a sliver of the pie – an expensive and not very satisfactory one,” said the source. “And Sainsbury’s thinks: ‘I’m sure we can do this ourselves, I’m sure we know what needs to be delivered.’”

Fairtrade didn’t think so. Under Fairly Traded, farmer collectives no longer receive their premiums directly, or decide how to spend this money. Instead, they must work on “action plans” and convince a Sainsbury’s board of experts in London to release their funds. In their open letter, Fairtrade’s tea producers protested that the company was characterising these premiums as “donor money” rather than earnings that rightfully belonged to the farmers. Fairtrade called the board “a further layer of bureaucracy controlled by a group far removed from day-to-day life in Africa”.

It feels like a paradox: that, in a moment when sustainability and ethics are more important than ever, corporations want to desert an agency such as Fairtrade, the most famous certifying entity in the field. Companies are under pressure – from customers, governments, banks and investors – to prove their sustainability credentials. Signatories to the UN’s Principles of Responsible Investment now manage around $89tn in assets. The network of Sustainable Stock Exchanges – more than 80 bourses around the world – guide and sometimes even require listed companies to report on their sustainability.

In a way, this dialled-up consciousness explains why companies are searching for alternatives to independent labelling systems, said Fredrik Galtung, who runs TrueFootprint, a firm that analyses data to tell businesses if their various sustainability projects make any sort of impact. He detects, in the industries he works with, a hardnosed sense that unsustainable practices will result in economic risk. So he isn’t surprised that companies, finding Fairtrade insufficient at tackling that risk, are rushing to erect their own standards. “If Fairtrade producers could show that their farmers were 10 or 20 years younger on average, that would be something. If they could show they’re more climate-resistant, that would be a massive advantage. But we’re unable to see all this.”

Fairtrade denies that its future is dim, or that a schism has appeared between itself and the companies with which it aims to work. In his discussions of these issues, Dario Soto Abril, who has been the CEO of Fairtrade International since 2016, is a model of genial tact. He’s in an unenviable position. He must keep his relations with corporations warm; Sainsbury’s still retails plenty of Fairtrade-labelled products, for example, and Mondelēz uses Fairtrade’s skills to track payments and commodities up and down the supply chain. At the same time, Soto must project his conviction that it is best for everyone concerned – shoppers, farmers, but especially companies – if they all commit fully to Fairtrade.

Agriculture has never been under more stress, Soto said. The world is so densely connected now that a tremor in one part of a supply chain sets off instant price fluctuations elsewhere. Corporate power is consolidating. Just three giants – Barry Callebaut, Cargill and Olam – control 60% of the world’s cocoa-processing business, for example; this gives them enormous leverage over the prices they pay to producers. “Cocoa farmers still live in poverty,” Soto said. “Banana farmers find that there are hidden costs of production that their sales are not covering.”

The effects of climate change alone are “so accelerated”, Soto said, “that we think if nothing of significance is done by 2050, at least half the land under coffee production now will be lost, one way or another.”

Last year, the UN released a map of the world predicting where yields were likely to drop or rise before 2050. In West Africa, South Asia and Southeast Asia, which have been plantation-farmed by colonists and then corporations, harvests will shrink. In North America and Europe, which have already cornered the profits of centuries of agribusiness, harvests will expand. The map is iniquity compounded upon iniquity.

Soto insists Fairtrade is flourishing: “Our credibility remains high. We get new partners all the time.” Sales of Fairtrade-certified products have swelled year on year around the world, partly as a result of companies exploring new markets. On the other hand, farmers are entering Fairtrade at a slowing pace. Between 2015 and 2016, Fairtrade managed to bring 3,199 new workers into its network; the previous year, it recruited 9,955.

“It’s important to note that Fairtrade can’t solve everything,” Soto said. “We’re here trying to solve challenges as old as humanity. We need more committed partners. We need more innovative ideas.” But Soto is also forgiving of companies that have chosen to set up schemes to certify themselves. “I think companies are looking to test something new,” he said carefully. “We need to allow for that. I will never doubt that companies will do the best they can.”

For the chief executive of an organisation founded on the premise that companies need to be cajoled or nagged into paying farmers a living income, this sounded like an astonishingly trusting statement. Soto’s most barbed opinion concerned farmer premiums and the larger issue of involving producers in decision-making. “I’m with you when you say that these companies aren’t giving farmers a seat at the table,” Soto told me. “You have to allow farmers a say in the income generated by sales. They have to own their premiums. That was our biggest suggestion to Sainsbury’s.”

Last October, the Fairtrade Foundation in the UK acknowledged that, in the new era of self-certification, its function had to evolve. Since companies are setting up their own sustainability schemes, said Lord Mark Price, who chairs the Foundation’s board of trustees, “the role of Fairtrade is to help those organisations be the best they can be”. Fairtrade will still operate its certification programme; the yin-yang logo will still be seen on products. But for Fairtrade to climb into the role of a consultancy – a supplier of guidance, rather than an umpire enforcing a line for companies to toe – is the most conclusive evidence yet of the transformation under way. And it raises two questions. How will companies behave if they have to testify only to their own code of ethical sourcing? And in what shape will that leave the future of agriculture?

If the animating principle of Fairtrade is “price,” that of most corporate in-house programmes is “yield”. These programmes may differ in other details. Some are just two years old; others were established more than a decade ago. Some pay their farmers Fairtrade-like minimum prices; others do not. But they are united by their unwavering focus on yield – on squeezing more produce out of a hectare of land. They consider it the best way to protect and grow their supplies of coffee or tea or cocoa, and thus the best way to save agriculture.

The investments are massive. Starbucks has sunk $100m into CAFE Practices, a programme started in 2004 to source sustainably grown coffee. Roughly 99% of Starbucks coffee now falls under this programme, said Michelle Burns, a senior vice president who heads global coffee and tea for the company. Mondelēz will have spent $400m on Cocoa Life, its sustainable cocoa scheme, by 2022. Nearly half of all Mondelēz cocoa is already sourced under this scheme, said Christine McGrath, who heads global impact and sustainability for the $80bn snack giant; she expects that number to have scaled to the full 100% by 2025. Both companies claimed they found existing third-party certifiers inadequate. “There were no programmes we could find that had an emphasis on quality,” Burns said. “Nothing that was comprehensive enough.”

In my conversations with Starbucks and Mondelēz, farmer welfare rarely came up. The tacit assumption seemed to be that if the companies help farmers improve their productivity, their lives will improve in tandem. Starbucks doesn’t set a minimum price, or pay a premium for every tonne of coffee it purchases, the way Fairtrade would have it do. Instead, it pays “strategic premiums” for preferred producers. As a result, last September, when coffee prices fell more than 20% to around $1 a pound, Starbucks had to inject a one-time $20m subsidy to its farmers in Central America.

Mondelēz also does not set a minimum price. It pays “loyalty premiums” to the farmer collectives with which it chooses to work, and Cathy Pieters, director of Mondelēz’s Cocoa Life programme, said it was up to them to decide what kind of “action plans” can be funded with the money. She added, though, that Mondelēz commissions large NGOs to work with these cooperatives in determining and crafting these action plans. Once again, as with Sainsbury’s and its Fairly Traded programme, the premiums intended to benefit farmers aren’t quite wholly theirs to spend as they see fit.

Facebook Twitter Pinterest Fairtrade tea producers in Malawi. Photograph: Chris Terry/Fairtrade

Fairtrade is half-owned by its producer cooperatives, so its standards and metrics are decided in large part by the representatives of farmers. The standards of Cocoa Life, CAFE Practices and other schemes aren’t like that; they are written by the companies themselves, in the companies’ own best interests. No independent third party certifies their success or failure in meeting these standards and publishes those results. Companies reporting on their own sustainability frequently fail “to prioritise planet over profit”, Hans Hoogervorst, who heads the International Accounting Standards Board, said in a speech in April. “Greenwashing is rampant.”

Industries are increasingly left to audit themselves, in part because governments have shredded their budgets in the name of austerity. From Volkswagen to Boeing, we don’t have to look very hard for evidence that companies fib and cheat when they get to mark their own homework. Bringing sustainability certification in-house is unlikely to empower farmers, or to renovate the structure of trade. “But it’s simplistic to say that companies will act in ways that are injurious to farmers,” said Fredrik Galtung of TrueFootprint, the sustainability analysts. In his view, the crises in agriculture are so stark that companies are motivated to protect farms and farm workers – if only to secure their own future profits. But TrueFootprint’s scorecards indicate that most companies only report their spending on sustainability; almost none of them investigate the impact of these schemes on farmers or on natural resources like air and water. Galtung described it as “just being happy to check off boxes”.

Economists like to examine incentives, and to deduce behaviour from them. If the incentives of these companies are misaligned with those of their producers, it’s only a matter of time before they produce conflict and rupture. Companies perceive Fairtrade to be rigid, and say that keeping sustainability programmes in-house gives them “flexibility”. That word is really standing in, though, for “control”: control over how commodities are priced, how to select or discard producers, how farmers farm, even how they live. This may look, for firms and even for consumers, like efficiency, but the effects can be dysfunctional. When four firms – Archer Daniels Midland, Bunge, Cargill and Louis Dreyfus – control between 70% and 90% of the global grain trade, they can bulldoze governments into clearing forested land for crops, crowd out smallholder farmers and push more and more processed grains into our diets. Many of the failings of global agriculture are not the result of multinational companies having too little control; they are the result of companies having too much.

That lesson is available to students of the past as well. After the publication of Max Havelaar shocked the public, in 1901 the Dutch government formulated a new ethical policy – one that would treat Indonesia as a moral responsibility rather than simply as a territory to be exploited. Millions of guilders had flowed from these islands into the Dutch treasury – 187m between 1867 and 1878 alone – and so the government committed to spending 30m guilders over the first decade on the welfare of the Indies’ farmers. Not surprisingly, they received agricultural credit and irrigation canals, because the Dutch corporations were avid for better yields. They also gained more schools and more roads; they were not, however, encouraged to industrialise, because this would have inflated the cost of local labour and led to competition with Dutch businesses.

Are your tinned tomatoes picked by slave labour? Read more

The Dutch ethical policy came undone as soon as the colonist’s aims started to clash with those of the colony. The harvests improved, but not enough to warrant the investment. The Great Depression set in, and the government thinned down its expenditures even as Indonesian farmers slid into debt. Educating the populace began to seem like a mistake; there was no work for young Indonesian men who had attended university, and they inevitably drifted into the nationalist movement. By 1940, the government had ceased to fund the welfare projects.

An unnamed administrator, who had once supported the ethical policy, watched with disappointment as the meagre progress unravelled and said: “As soon as we withdraw our hands, everything sinks back into the marsh.” But what the Dutch didn’t recognise was that they were the ones who made it impossible for the Indonesians to rise from the marsh in the first place.

• This article was amended on 24 July 2019. An earlier version said that Fairtrade enlists companies to pay a minimum price, which is always set above market price. In fact, the Fairtrade minimum price is charged only when market prices drop below a pre-determined level. Sainsbury’s pulled its own-brand teas out of Fairtrade, rather than ceasing to sell any Fairtrade-certified teas; companies buying Fairtrade must meet other standards in addition to price; and the minimum price for bananas from Cameroon is €6.40 per 18.14kg box, not per 1kg. These have been corrected.

• Follow the Long Read on Twitter at @gdnlongread, and sign up to the long read weekly email here.