When Tony Blair deployed British troops in Afghanistan, ending the illicit production and supply of opium was cited as a key objective. In 2001 the prime minister linked heroin use in the UK with opium cultivation in Afghanistan: "The arms the Taliban buy are paid for by the lives of young British people buying their drugs. This is another part of the regime we should destroy." Yet after 10 years of effort with tens of thousands of troops in the country, and having spent billions trying to reduce poppy cultivation, Afghans are growing more opium than ever before.

As the December US troop draw-down deadline approaches, the UN Office of Drugs and Crime estimates that last year Afghanistan produced nearly $3bn worth of opium, and its derivatives heroin and morphine. Since 2002 the US has provided more than $7bn for counter-narcotics efforts and agriculture stabilisation programmes.

John Sopko, the special inspector general for Afghan reconstruction, told a US Congress subcommittee recently: "On my trips to Afghanistan in 2013 and earlier this year, no one at the (US) embassy could convincingly explain to me how the US government counter-narcotics efforts are making a meaningful impact on the narcotics trade or how they will have a significant impact after." The illicit global trade in drugs has an estimated annual turnover of $320bn and the war to stop it costs $100bn a year.

In a country such as Afghanistan, with weak institutions, remote areas ripe for poppy cultivation and a well-established smuggling network, we are fighting a lost battle. It is well understood that not only does illicit trade migrate towards "ungoverned spaces", particularly those inhabited by people in dire poverty, it then makes matters far worse.

In 2012 the International Institute for Strategic Studies published Drugs, Insecurity and Failed States: The Problems of Prohibition, concluding that "the present enforcement regime is not only failing to win the 'war on drugs', it is also a major cause of violence and instability in producer and transit countries". Afghanistan exemplifies this in spades. The opium trade is corrupting Afghan institutions at all levels – arming insurgents and warlords, and undermining security and development.

In short, the war on drugs has failed in Afghanistan, and without removing the demand for illicit opium, driven by illicit heroin use in consumer countries, this failure is both predictable and inevitable. If we cannot deal effectively with supply, then the only alternative would seem to be to try to limit the demand for illicit drugs by making a supply of them available from a legally regulated market.

Half of the world's opium is grown for the legal opiates market of which the UK grows 3,500 hectares. This legitimate drug trade does not fund the Taliban and warlords, and there is no reason why it cannot be expanded to include non-medical trade and use.

I am not the first former ambassador who has served in a drug-producing country to call for an end to prohibition. In 2001 my colleague Sir Keith Morris, the former UK ambassador to Colombia, told the BBC that if drugs were legalised and regulated the "benefits to life, health and liberty of drug users and the life, health and property of the whole population would be immense".

Many more have made the same plea. In 2002 the home affairs select committee called on Britain to initiate a debate at the United Nations on alternatives to drug prohibition – including legal regulation. One of its members was David Cameron MP.

I understand why some politicians are reluctant to take up this debate. Before going to Afghanistan my own instincts told me that it could not be right to decriminalise drugs. But my experience there has convinced me that all political parties need to engage seriously, without trying to score points off each other.

I was deeply moved when I came across an article written by a mother who had lost both of her sons to heroin overdoses. In the unregulated prohibited market there is no quality control, no purity guide, and no safer use advice. Had her two boys been able to acquire their heroin from a doctor, they might well still be with us. In fact thousands of dependent users around Europe are already prescribed heroin, including a handful in the UK, with great benefits to them and society as a whole.

Tony Blair was absolutely right to make the link between opium production in southern Afghanistan and heroin use in Britain. But it is clear now that he and others were wrong to think this link could be broken through military action internationally and police enforcement domestically.

Putting governments in control of the global drugs trade through legal regulation will remove the incentive for those in fragile, insecure regions to produce and traffic drugs. Putting doctors and pharmacists in control of supply in the UK will save lives, improve health and reduce crime. Ultimately we could improve the underlying lack of wellbeing that drives so many in the UK and Afghanistan into lives of degradation and misery.

For the sake of both Afghans and British citizens, senior politicians must take responsibility for the failings of global prohibition, and take control of the drug trade through legal regulation.