The US war on drugs has cost one trillion dollars and resulted in 45m arrests. And yet nothing has changed, argues film-maker Eugene Jarecki, a polemical campaigner to reform America's drugs laws. So what did the prisoners in a New York jail think when he showed them his documentary?

Once consigned to the fringes of libertarianism, the argument for the legalisation of drugs has received an unlikely boost in America in recent months with the release of a documentary entitled The House I Live In. Coinciding with the decision by the states of Colorado and Washington to legalise marijuana, the film won the Grand Jury prize at the Sundance film festival last year and has arrived at a moment when Americans are beginning to reconsider the efficacy of their nation's drug policy.

Packed with facts, stories and polemics, the film traces the history of America's changing attitudes to drugs and the role of the criminal justice system in shaping those attitudes. It also focuses on a variety of different individuals – inmates, prison guards, judges – caught up in the massive industrial process of incarceration. And there is a powerful strand running through the film in which David Simon, the man behind the TV series The Wire, explains why the war on drugs has proven a counterproductive disaster.

It was made by Eugene Jarecki, who comes from a family of film-makers. One brother, Andrew Jarecki, made the celebrated documentary Capturing the Friedmans, while another, Nicholas Jarecki, is the director of the Richard Gere film Arbitrage. For his part Eugene has built a reputation for making cerebral, opinionated and factually rich documentary films such as The Trials of Henry Kissinger and Freakonomics.

Like his films, Jarecki is not a man of reserved or compact opinion. He describes himself, accurately, as a "transcriber's nightmare". He speaks in great sprawlingly digressive paragraphs about whatever moves or riles him, which could be anything from his admiration for the primatologist Jane Goodall to his contempt for the film Zero Dark Thirty: "What Kathryn Bigelow did in that movie," he tells me with uncharacteristic concision, "was as close as an artist can come to committing a war crime."

But the subject that really causes him to let fly is America's war on drugs. The failure of that policy, dating back to the Nixon administration, is the main target of The House I Live In. Since the film was released, Jarecki has been touring America to put his case across. Over the next few weeks and months he'll be meeting lawmakers and criminal justice officials in seven states from California to New York. But perhaps the audiences he speaks to with the most practical knowledge of the iniquities of drug legislations are the prisoners he visits in institutions across America.

Last month I accompanied Jarecki on a trip to Otisville correctional facility in upstate New York, an area scarred by recession but which boasts several prisons. The House I Live In contends that, in towns such as Otisville, prisons have become a critical part of the local economy, which means there is an economic imperative to maintaining a high prison population.

Otisville is considered a "medium sized" prison in the American system, but as it seems to occupy the whole side of a long hill, and is surrounded by miles of fences and razor wire, it looks massive. To save time for a debate, Jarecki screened a shortened version of the film inside a large hall that doubles as the prison's basketball court. There are about 100 prisoners, or "offenders" as they're known at Otisville. The US prison system features a disproportionately high number of black prisoners, and the audience at Otisville is overwhelmingly black. One of the striking statistics the film quotes is that while just 13% of crack cocaine users are black, black Americans make up 90% of those imprisoned for crimes relating to crack cocaine.

Central to Jarecki's argument is the contention that drug laws were introduced as a means of controlling ethnic minorities. Thus initially, his film states, opium was outlawed in California as a means of targeting the Chinese population, and similarly marijuana and cocaine were both prohibited when they began to be used by, respectively, Hispanics and black people. Jarecki seems to suggest that the American authorities were less interested in criminalising drug use than in persecuting racial groups.

"I think that racism and xenophobia, these are impulses and human tendencies that course through the veins of the creature to varying degrees," Jarecki tells me later. "America has had an extraordinary struggle with racism, underscored so poignantly by the degree to which we seek to stand as a symbol of democracy and human rights. America has a special irony in this respect. Those forces have a pre-cerebral subliminal role in the guidance of society whether we like to admit it or not."

For all its critical structural analysis of American society, The House I Live In is at root a personal film that tells the story of Nannie, a black woman who was the family housekeeper when Jarecki was a boy. Eugene is the son of Henry Jarecki, a psychiatrist and highly successful entrepreneur, and the family lived in an affluent suburb of New Haven, Connecticut. But when Henry's job took the family to New York City, Nannie went with them. She left her son James, who was around Jarecki's age, behind in New Haven. With little parental oversight, James got into drugs and crime and ended up dying young of drugs-related complications, but not before fathering a son who is now serving a 30-year prison sentence.

The version of the film the prisoners see at Otisville is about half the length of the original, but it's still bursting with information and interviews and Nannie's story. Another theme the film explores is the repressive effect of mandatory minimum sentencing which has left many thousands locked up for shockingly long periods for sometimes minor crimes.

When the screening is finished there is no applause from the audience but instead a kind of tense silence. Did they not like the film? Later Jarecki will tell me that this is a typical reaction: "They're in shock," he says. "They've just seen their lives up on screen."

Jarecki walks up on stage, a plump white Jewish man from a wealthy background, and addresses an auditorium full of well-built and underprivileged black men, a large number of whom have been inside for a long, long time. It's a situation in which many others might feel intimidated, but Jarecki doesn't hesitate. "Why do you think I'm here?" he asks boldly.

One of the offenders suggests that it's to exorcise the guilt he feels about Nannie's son James.

"You're here to learn from us," says a different prisoner.

"To get more material," adds another.

"You're here to enlighten people on the war on drugs and how it's about social control," says a fourth.

Jarecki offers agreement to all these answers, but is soon off talking about the military-industrial system, the pharmaceutical system, and the lack of democracy that enables corporate interests to dominate Congress. It could sound like the ravings of a conspiracy theorist, but the film-maker is on a roll and a number of the prisoners are busy taking notes.

"How many of you are serving life?"

Around half the hands – about 50 people – go up.

"Do you realise there's more people in this room with life sentences," he says, "than in the whole of Britain? There are just 41 people with life sentences in Britain."

There is a communal sigh of disbelief and a lot of despairing head shaking. No one is aware that this statement is incorrect. Although Britain has a far more liberal approach to sentencing, especially relating to drugs, there has been an expansion in prison numbers and the size of sentences in the past decade or two. There are now around 8,000 prisoners with life sentences in the UK. It's true that between 41 and 51 of them, depending on varying definitions, are serving whole-life tariffs, but there are no prisoners at Otisville serving life without parole, the US equivalent of a whole-life tariff, so the comparison is misleading.

Afterwards Jarecki tells me there was no intention to mislead. He said he wasn't trying to draw a direct analogy and that anyway "life" is a harsher sentence in the US, even with the possibility of parole. That's undoubtedly true, although Jarecki's critics might well see a tendency to simplify complex issues. The key fact remains, however, that there are 41,000 prisoners serving life without parole sentences in the US.

In a number of cases they are there as part of the so-called "three strikes and you're out" strategy instituted by Bill Clinton in which the third conviction, following two serious felonies, leads to a mandatory life without parole sentence. That means that a conviction for a non-violent drug offence, sometimes as minor as selling pot, can be enough to end all possibility of freedom. Jarecki says that he's heard that Clinton "has come to some very deep realisations about the wrongheadedness of the war on drugs".

As roughly one fifth of prisoners in the US are serving time on drugs charges, the men at Otisville know all about the punitive nature of the drug laws in America. Since 1971, when Richard Nixon announced the war on drugs, America has spent $1 trillion and made 45m arrests in the resulting skirmishes. The number of offenders imprisoned for drugs charges has increased twelvefold in the last 40 years. And yet illegal drug use has remained unchanged.

Jarecki argues that these figures spell a wholesale failure for a policy that is urgently in need of rethinking. He's not alone. There are signs that a growing number of Americans are beginning to reach a similar conclusion. Last year the states of Washington and Colorado legalised the recreational use of marijuana. It will be legal to possess 1oz (28g) of marijuana if you are over 21; it will be legal to grow up to six marijuana plants in your house; and it will be legal to give away up to 1oz. And in several other states the drug has been decriminalised, although it remains illegal under federal law.

The film-maker deliberately doesn't focus on marijuana, because it's not the source of the greatest injustices, although he does think it's significant as a kind of criminalising tool. "Marijuana is not a gateway drug to other drugs – that's pseudo science," says Jarecki. "But it is a gateway drug into the criminal justice system."

Yet whereas many Americans are sympathetic to relaxing the laws on marijuana, it's a different story with cocaine, heroin and methamphetamine. These are drugs whose effects have ruined individuals, devastated families and sometimes laid waste to whole communities. While no fan of cocaine or heroin, David Simon suggests in the film that cause and effect may run the other way in terms of social breakdown. When there is no legitimate economy to speak of in a deprived area, says Simon, it's only logical that an illegitimate economy will seek to fill the void.

One Otisville offender, Craig Coston, is a youthful looking 34 who was sent to prison for manslaughter when he was 16. "My mother was on drugs," he tells me. "I dropped out of high school and my family was the scene of every kind of dysfunction. My father was killed when I was seven. He was shot twice and he probably would have survived if he hadn't been under the influence of cocaine. If it hadn't have been for drugs, I wouldn't have become involved in crime."

Coston is not in favour of legalisation, and nor is Dale D'Amico, who is serving life for the attempted murder of a policeman. A former cocaine dealer who quit using in prison because he couldn't defend himself in fights, he says: "I'm not pro-legalisation. Treatment is the answer. You don't incarcerate someone for getting high."

Jarecki is all too aware of the taboo of legalisation. For this reason, he is reluctant to use the term. "I believe that legalisation is a word that has come to scare the majority of people who hear it," he says, driving back from the prison. "The day that America legalises drugs is the day there is a drug dealer on every corner. I also don't know I would be an advocate of it per se. I think drugs represent a massive health concern. I support the language of tax and regulate."

Essentially there are two main liberalising positions in the drug debate – decriminalisation and legalisation. Portugal is the prime example of decriminalisation. A decade ago the Portuguese effectively decriminalised all drugs use and, with the highest HIV rates in Europe among intravenous drug users, targeted addicts with clean needles and therapy. Although possession of drugs is still illegal, criminal proceedings only take place in cases where the drug user is caught with more than 10 days' worth of supply. The results have been encouraging, insofar as drug-related crime has fallen and fewer young people have become heroin users.

But the limitations and contradictions of this policy are exposed by the fact that while drug consumers can go about their business in Portugal without fear of arrest, the international gangsters that supply them remain subject to imprisonment. In other words, good for Portugal but it doesn't change the situation in, for example, Mexico, where, according to Human Rights Watch, 60,000 died in drug-related violence between 2006 and 2012, although some estimates are even higher.

Of course very few of these deaths will have had anything to do with Portugal. They are overwhelmingly related to the drugs trade north of the border in the US. And decriminalisation in the States would do little or nothing to alter the situation in Mexico, Colombia and elsewhere.

But would legalisation have any greater impact? However Jarecki dresses it up, tax and regulation is just another way of saying legalisation, because no sane voice is arguing for making hard drugs uncontrolled substances. The question is how can you tax and regulate a substance – for example cocaine – whose production and international distribution is controlled by criminal cartels?

"I see your point," says Jarecki, "but you're making an assumption that if you tax and regulate in the United States that business would continue as usual in places like Mexico. It would knock the shit out of that business. After America legalises, a huge domino effect would take place around the world. You may still have draconian penalties in Afghanistan for producing opium but you have draconian penalties there for reading a book."

It's true to say that the drugs war in Latin America, which is where the bloodiest frontline of the action is located, is underwritten by the US government, and therefore American taxpayers. As it does little but create instability and violence for the countries involved, it seems reasonable to assume that without US encouragement and financial support, the respective governments would follow suit and regulate and tax drug production.

Still, the theory of the domino effect gives rise to some intriguing and complex possibilities. What if all the dominoes don't fall? What if, for example, America legalised, but Britain and the other European countries did not? At the moment the British government, like the American government, is dedicated to maintaining the status quo on drugs. Even a not particularly radical report last year from the home affairs select committee, recommending reform and the possibility of decriminalising cannabis, was speedily dismissed by the government.

"Drugs are illegal because they are harmful," said a government spokesman. "They destroy lives and blight communities. Our current laws draw on the best available evidence and as such we have no intention of downgrading or declassifying cannabis."

If Colombia legalised the production of cocaine, but it remained illegal in Europe, how and from where would illicit cocaine be imported? Would criminal cartels vie with legal multinationals to distribute their wares, creating further armed conflict? Or would cocaine return to being an expensive rarity used only by the very wealthy?

These are hypothetical questions, but they do point to some of the anomalies and difficulties that await any reform that isn't universal. As it's never possible to change the world all at once, such problems are unlikely to stall the argument for legalisation, because it's driven not by the utopia it hopes to create but the dystopia it seeks to escape.

As Jarecki puts it: "The solution is so obvious. That is why the criminal justice system is vulnerable to serious reform because the only thing obstructing serious reform is corrupt vested interest profiting from the broken status quo we have. Once you're at that point, let's smoke them out."

But it's the problem rather than the solution that is most obvious. If there is a consensus in this debate it's that the war on drugs has not been a success. However, there remain huge differences of opinion over the reasons why. For rightwing commentators such as Peter Hitchens, the explanation for the current crisis is simple. The law has not gone far enough. Hitchens recently published a book on the subject entitled The War We Never Fought.

Hitchens argues that a liberal elite intervened to prevent a proper war on drugs in Britain. There are several flaws in this argument, not least the 10,000 prisoners locked up on drugs charges. But perhaps the book is most undermined by its neglect of America, where the war on drugs has often been uncompromising – especially for those who have been incarcerated for decades.

A more sophisticated argument against legalisation is not that the war hasn't been fought, but that its failure should not be a reason for abandoning it. As the cultural critic and former prison doctor Theodore Dalrymple has said: "Saying the war against drugs is unwinnable is like saying the war against burglary is unwinnable and we should open our doors. Absurd. War is the wrong word. A dim-witted metaphor."

Jarecki has heard all the counter-arguments, but remains firm in his conviction. "Listen," he says, "no reform is perfect. You hear a lot of nonsense spoken by some advocates of legalisation. You hear that if you legalise, drug use would diminish. That's bullshit. As with the end of prohibition, consumption would go up. The other myth is that if you legalise and drop the price, violence will diminish. Also not true. Violence will migrate. You already have a cross-section of the population making a living at the point of a gun, so they will take it into other fields: larceny etc. But that's not bad news because the violent have now migrated away from the non-violent. What could be better than that?"

Once the violent have departed the scene, Jarecki would like to see recreational drugs treated as a health issue rather than a criminal one. But a great many changes would have to take place in America before such a fundamental shift in perspective could come about. They would involve Congress, the state department, individual states, and a dramatic overhaul in attitudes concerning drug use and punishment. There is a long road to travel before that day is within sight but Jarecki is rushing down it as fast as his film can carry him.

The House I Live In is available on DVD and iTunes; thehouseilivein.co.uk

• This article was amended on 24 May 2013 to include a Human Rights Watch estimate of 60,000 deaths in drug-related violence in Mexico in the past six years.

More on this story

Brad Pitt. Photograph: Jeff Vespa/WireImage

Brad Pitt: America's war on drugs is a charade, and a failure

The actor and executive producer of the documentary The House I Live In says US drugs policy needs a radical rethink

David Simon, creator of The Wire and contributor to The House I Live In Photograph: Geraint Lewis/Rex Features

David Simon on America's war on drugs and The House I Live In

The writer/director, who contributed to this hard-hitting documentary, on why US drugs policy has gone terribly wrong

Russell Simmons, hip-hop mogul and executive producer of the film. Photograph: Jeff Vespa/Getty Images for The Weinstein Company

Russell Simmons: the war on drugs made victims of the black community

The hip-hop mogul and executive producer of The House I Live In tells of the suffering he saw while growing up in Queens

Shanequa Benitez outside the Cromwell Towers housing project in Yonkers, New York. Below: Russell Simmons. Photograph: Samuel Cullman

Shanequa Benitez: how I started dealing drugs

Ex-drug dealer and contributor to The House I Live In on the perils of being drawn into the dangerous world of drug dealing