It is time to admit the obvious. The "war on drugs" has failed, at least in the way it has been waged so far. In Latin America, the "unintended" consequences have been disastrous. Thousands of people have lost their lives in drug-associated violence. Drug lords have taken over entire communities. Misery has spread. Corruption is undermining fragile democracies.

And, after decades of over-flights, interdictions, spraying and raids on jungle drug factories, Latin America remains the world's largest exporter of cocaine and marijuana. It is producing more and more opium and heroin. It is developing the capacity to mass-produce synthetic drugs.

Continuing the drugs war with more of the same is ludicrous. What is needed is a serious debate that will lead to the adoption of more humane and more effective strategies to deal with the global drug problem. Earlier this year the Latin American Commission on Drugs and Democracy, which I co-chaired with the former president of Colombia, César Gaviria, and the former president of Mexico, Ernesto Zedillo, released the first high-level statement ever to endorse harm-reduction generally and decriminalisation of marijuana specifically.

The core conclusion of the statement is that a paradigm shift is required away from repression of drug users and towards treatment and prevention. The challenge is to reduce drastically the harm caused by illegal narcotics to people, societies and public institutions.

To move in this direction, it is essential to differentiate between illicit substances according to the harm they inflict. The status of addicts must change from that of drug buyers in the illegal market to that of patients cared for in the public health system. Police activities can then be better focused against the drug lords and organised crime.

The shift towards harm-reduction efforts and decriminalisation has already begun. Recently, a landmark ruling by Argentina's supreme court and a law passed by Mexico's Congress have for all practical purposes removed criminal penalties in those countries for the possession of small amounts of drugs for personal and immediate consumption.

Colombia was the first country to take this step. A decision by its constitutional court in 1994 scrapped penalties for private consumption. Bolivia and Ecuador have liberalised their drug laws. Change is also imminent in Brazil. The chief justice of our highest court made a public appeal for clarification of the differentiation between drug user and drug dealer. A current ambiguity in the law effectively opens opportunities for police corruption and extortion. Brazil's legislature is about to consider a new law to remove penalties for the consumption of small amounts of marijuana.

This is consistent with the broader trend in Europe: the Netherlands decriminalised years ago; Portugal followed in 2001, stressing that criminalisation drove resources away from treatment and deterred people from seeking help for addiction – the number of people using drugs before decriminalisation was higher than afterwards. In the United States, backing for decriminalisation and treatment alternatives to prison is growing, but has still not achieved a critical mass of support and momentum behind traditional – failed – punitive policies remains strong.

There is still a long way to go. The trend towards decriminalisation for possession helps to empower a public health paradigm. It breaks the silence about the drug problem. It enables people to think in terms of approaching drug abuse in a way that is not first and foremost a matter for the criminal justice system. Reducing the harm caused by drugs goes hand in hand with reducing consumption.

Repressive policies towards drug users are firmly rooted in prejudice, fear and ideological visions, rather than in cold and hard assessment of the realities of drug abuse. The approach recommended in the commission's statement does not imply complacency regarding narcotics and their purveyors. Abuse of drugs is harmful to health. Abused drugs undermine a user's decision-making capacity. Needle-sharing spreads HIV/Aids and other diseases. Addiction can lead to financial ruin and abuse of family, especially children.

To be credible and effective, decriminalisation must be combined with robust prevention campaigns. The profound drop in tobacco consumption in recent decades shows how public information and prevention campaigns can be effective when they are based on messages that are consistent with the experience of those they target.

No country has devised a comprehensive solution to the drug abuse challenge. And a solution need not be a stark choice between prohibition and legalisation. Alternative approaches are being tested and must be carefully reviewed. But it is clear that the way forward will involve a strategy of reaching out, patiently and persistently, to the users, and not the continued waging of a misguided and counterproductive war that makes the users, rather than the drug lords, the primary victims.

• Fernando Henrique Cardoso was president of Brazil from 1995-2003