The right to bear arms that is enshrined in the US constitution is now most fiercely defended by rightwing libertarians. But it wasn’t always the case. In the mid-1960s, that decade of revolt and turmoil, Huey Newton, a 24-year-old law student in Oakland, California, realised that citizens of that state had the legal right to carry arms openly.

A teenage thug who taught himself to read, Newton had consumed revolutionary literature from Marx to Malcolm X and had become, in his early 20s, a political activist bent on promoting the rights of his fellow African Americans. But he was steeped in violence. After serving a short sentence for stabbing a man with a steak knife, he returned to college and, with his friend Bobby Seale, set up the Black Panther party in 1966.

We wanted to give this film the spirit of this revolutionary fervour Director Stanley Nelson

This was an era when black politics was inextricably bound up with the civil rights movement and the non-violent protests led by Martin Luther King. Yet increasing numbers of young blacks, appalled by scenes of African Americans being brutally treated by well-armed white police, were frustrated by what they saw as the passivity of the civil rights movement. There had been riots in Watts in Los Angeles in 1965. Malcolm X had called on black Americans to combat oppression “by any means necessary”, but he was killed that same year. There was a vacancy for a new black vanguard.

It was Newton’s idea to go out on armed patrol with other Black Panthers and bear witness to racist or provocative police action. For the most part, these patrols were limited to Oakland; the rest of the US remained largely ignorant of the young revolutionaries. But then, on 2 May 1967, the Black Panthers showed up at the California state assembly in Sacramento carrying loaded weapons. They were dressed in black leather jackets with rollneck jumpers, berets, sunglasses and shotguns. It was a stunningly cool and defiant image and overnight the Black Panthers forced themselves to the front of black radical politics in America.

Very soon, they were known throughout the world, inspiring black radicals and all-purpose revolutionaries in the UK, Europe, the West Indies and even Africa. Jean Genet, the French writer and activist, became a committed supporter. Hollywood stars flocked to get a glimpse of this excitingly dangerous new phenomenon. And the composer Leonard Bernstein held a celebrity-packed fundraising party at his New York home, complete with uniformed Panthers, which was famously satirised by Tom Wolfe who minted the phrase “radical chic” to describe the rich and famous toying with revolutionary causes.

More generally, the Panthers were taken up by the counterculture as a kind of domestic version of the anti-imperialist struggle being waged in the Vietnam war. As Todd Gitlin, a leading member of the US counterculture, stated in his book, The Sixties: Years of Hope, Days of Rage: “Nothing made the idea of revolution more vivid to the white left than the Black Panther party.”

Huey Newton, centre, on his return from China, with Elaine Brown on the left. Photograph: Bettmann/Corbis

This historic period is emotively captured in a new documentary, The Black Panthers: Vanguard of the Revolution, made by Stanley Nelson, a long-time chronicler of African American struggles. It’s a powerful film, full of dramatic footage and a pulsating soundtrack of funk and soul. And it could hardly be more timely. The past couple of years have seen a spate of incidents of police brutality towards African Americans, many of them captured on video. With protests taking place from Ferguson in Missouri to New York, we have been reminded that although there may be a black president in the White House, the plight of far too many African Americans is that of second-class citizens and victims of racist law enforcement.

Without making a direct correlation, the film shows that recent flashpoints have a long and tumultuous history. Yet it’s also unquestionably an idealised vision of the Panthers. The story it tells is of noble intentions undermined by the dark machinations of the US state, in the malign guise of the FBI and the omnipotent figure of its devious director, J Edgar Hoover.

There is little doubt the Panthers were targeted in ways that were often viciously excessive, including what amounted to extrajudicial killings. But there is even less doubt that among their own senior ranks were pathological killers, ideological madmen and depraved opportunists. But that side of the story is either only referred to in passing or ignored altogether. For example, the film features interviews with a charming and now grey-haired former Black Panther called Ericka Huggins. Although the image of the Panthers was potently male, more than half the members by the late 1960s were female. Their story has been sidelined and it is to the documentary’s credit that it seeks to redress the marginalising of female voices.

We hear Huggins speak about what initially galvanised her to join the Panthers, how she committed herself to the cause and how, with a heavy heart, she finally left, having been assaulted by an increasingly deranged Newton. What she doesn’t mention – and nor does the film – is the case of a 19-year-old Panther foot soldier called Alex Rackley. In May 1969, after her Panther husband, John Huggins, was killed in a shoot-out with black nationalists, Huggins moved with her baby daughter to New Haven, Connecticut, where she led the local chapter of the Black Panthers. For no good reason, Rackley, an illiterate young man, was suspected of being an informant. At that time, the FBI had infiltrated the organisation and fear of informants was widespread. And so Rackley was interrogated under torture. He was tied to a chair and had boiling water poured over him and was then left tied to a bed for three days in great pain, in his own mess. Not only did Huggins witness these scenes, she was recorded reading out a charge sheet of Rackley’s “confessions”, after which he was taken to a nearby swamp and shot dead. Huggins and Seale were charged with murder, kidnap and conspiracy, the allegation being that they commissioned the crime. After a celebrated trial, both walked free. Huggins’s defence was that she was coerced into playing along with the killers – Warren Kimbro and Lonnie McLucas – by the national “field marshal” of the Panthers, George Sams.

The Black Panthers: Vanguard Of The Revolution director Stanley Nelson. Photograph: Sam Alesh

Of this whole tragic episode, not a word is mentioned. Why? “Ericka Huggins was acquitted,” Stanley Nelson tells me on the phone from the States. “You can’t keep trying people in the media.”

Huggins was indeed acquitted – the jury found her not guilty by 10-2. But no one can say she wasn’t there during the torture. Nor that she didn’t remain in the Panthers for almost another decade. Why did she stay with an organisation that harboured such sadistic killers? “Because I was committed to the party, not to orchestrations of the FBI,” she tells me from her home in Oakland. “I was committed to serving people. That’s why I stayed.”

As far as she is concerned, the FBI were ruthless puppet masters, able to turn good men into cold-blooded killers through the power of suggestion. It’s a handy theory that absolves the Panthers of responsibility for their own actions and it’s one the film implicitly, and sometimes explicitly, supports. As Nelson says: “We did ask people about that [Alex Rackley] story. But to retry someone who’s been acquitted, I don’t see the relevance of it. What exactly does it prove? It’s a very complicated story. You have a psychopathic infiltrator from the FBI, spurring the Panthers on to commit these acts.”

The accusation that Sams was working for the FBI has been repeated many times but with no definitive proof and there is nothing to suggest the torture was conducted under FBI direction. All the same, when I ask Nelson whether there was a critical point at which things started to go wrong, he says: “If you wanted to single out one thing, I think it would be J Edgar Hoover and the FBI. I think the Panthers might have been able to sort out their internal problems themselves.”

This is, to say at the least, a large assumption, But it’s one that fits the film’s perspective. As Nelson rightly says: “The way the Black Panthers are remembered is not the way that they were thought of at the time.” And one judgment of the film is that it sets out to recover that earlier understanding of the Panthers, when they were seen by many as an inspiration rather than an embarrassment. “We wanted to give this film the spirit of this revolutionary fervour going on,” says Nelson.

In an immediate sense, it’s a question of image and style. The Panthers were striking in the first instance because of how they looked. This was the era of afros and black consciousness and no one did it with more panache than the Panthers. Seldom has a sense of danger, radicalism and street cool been combined to such mesmerising effect.

One former henchman in the film recalls a gathering of sympathetic Hollywood A-listers which Black Panthers attended in full black leather-and-guns garb. “Those stars,” he says laughing at the memory, “they just loved that shit. They just ate that shit up.”

Ericka Huggins is released from prison after the case against her for the murder of Alex Rackley was dismissed, May 1971. Photograph: Dave Pickoff/AP

There was a willingness in progressive circles to give the Panthers the benefit of any doubt, owing to their unimpeachable revolutionary credentials. For they didn’t just shout “kill the pigs”, like a lot of angry students, they really did go out and shoot the police dead. Newton killed Oakland police officer John Frey in a shoot-out after Frey stopped him in his car early on the morning of 28 October 1967. Newton suffered a stomach wound in the exchange and was arrested in hospital. There followed a nationwide campaign to release him.

In the film, we see crowds carrying placards with the legend “Chairman Mao says ‘Free Huey’”. Nothing could more efficiently sum up the gleeful naivety that at the time masqueraded as a revolutionary stance. Mao was responsible for the deaths of countless millions of his citizens and Newton was a deranged narcissist ever ready to pull the trigger. Fittingly, in 1971, Newton visited China and, while he didn’t meet Mao, he did have an audience with Zhou Enlai.

In a sense, the Free Huey campaign worked. Although Newton was found guilty of voluntary manslaughter, the conviction was overturned and in May 1970 he was released. Two further trials resulted in hung juries and the case was finally dismissed. By then, the party was in meltdown.

Revolutionaries tend to specialise in internal struggles and the Panthers were no exception. While Newton was in prison, their minister of information, Eldridge Cleaver, tried to take control of the party. Cleaver was an ex-con who had written Soul on Ice, a series of essays on his political journey from “supermasculine menial” to arch-black liberationist. In the book, he admitted to raping white women as “an insurrectionary act”, an outlook he recanted without ever quite losing the misogyny and racism that informed it.

In the film, we see Cleaver as a charismatic but unpredictable leader determined to engage the police in acts of violence. After leading a botched ambush in which two policeman were wounded and a 17-year-old Black Panther was killed, Cleaver fled to Cuba and from there to Algeria, where he set up an international office of the Black Panthers, receiving ambassadors from North Vietnam and North Korea. From Algeria, Cleaver also set about challenging the newly released Newton’s leadership, which resulted in a split in the party and much bloodletting. There is a fascinating recording played in the film of a phone conversation between the two men that sounds like the prelude to a prison fight, with Newton calling Cleaver a “punk”.

“Maybe Huey and Eldridge were both so flawed that it could never have worked,” Nelson acknowledges. “But the other thing is that we couldn’t find a picture of Huey, Bobby and Eldridge together. One or more of them was always in jail or exile. They never got to sit down eye to eye and work out their problems.”

Cleaver was joined in exile by his wife, Kathleen Cleaver, then a 23-year-old firebrand. Now a professor of law, she’s one of three impressively articulate women who figure prominently in the documentary. The other two are Huggins and, perhaps the most compelling of them all, Elaine Brown, a former cocktail waitress in a strip joint who joined the Panthers, like many black Americans, after the assassination of Dr King in April 1968.

Despite their self-consciously macho image, the Black Panthers did attempt to involve women. “We really wanted to talk to women about this,” says Nelson. “Elaine [Brown] said it very well, that ‘we didn’t get the men from revolutionary heaven’. We did not want to say that the Panthers succeeded in doing away with sexism. But we did want to say that it was at least on their minds.”

Kathleen Cleaver, photographed in Oakland, 1968. Now a professor of law, she was the wife of Eldridge Cleaver. Photograph courtesy of Jeffrey Blankfort

There was a period in the mid-1970s, in fact, when Brown became chair of the Black Panthers. However, the film doesn’t cover this era or the ironies involved in it. Newton and Brown were lovers (this is also not mentioned) and she was devoted to her leader. On one occasion, as she admitted in her autobiography, she slept with the white Hollywood film producer Bert Schneider, a Black Panther hanger-on, and in return Schneider made a $12,000 contribution to the rent of Newton’s penthouse apartment. Newton made Brown chair when he fled to Cuba in 1974.

He left the country to escape another murder charge. It was alleged that on 6 August 1974 Newton shot a 17-year-old street prostitute called Kathleen Smith in the face, apparently because she referred to him as “Baby”. She died after three months in a coma. Newton lived in Cuba for three years and stood trial for Smith’s murder on his return. But the main witness refused to testify, following an attempt to kill her, and Newton again walked free after two deadlocked jury trials.

Huggins soon learned that Newton had killed Smith. Why didn’t she leave the party at that stage? “I think you’re looking for answers that I really can’t give you,” she says defensively. “I’m a little confused about what you want me to say. Is there something you’re looking for?”

I ask for an explanation of why she remained loyal to a killer of a 17-year-old girl and she says that the shooting was an accident. “Nonetheless, a woman was killed. However, I had my work to do in the Black Panther party, so I continued to do it. That’s the best I can explain to you.”

Again, none of this is mentioned in the film.

Given the circumstances in which Huggins’s friend Elaine Brown took control, it could hardly be claimed as an unambiguous victory for feminism. In his memoir Radical Son, David Horowitz, a former Bay Area white radical turned rightwing activist, recalled his involvement with the Panthers and Brown’s leadership.

He claims that she had a taste for violence and certainly she wasn’t much bothered by Newton pistol-whipping a man so badly that he required brain surgery. Later she wrote: “It is a sensuous thing to know that at one’s will an enemy can be struck down… For a black woman in America to know that power is to experience being raised from the dead.” Horowitz lost his faith in the Panthers and, as a consequence, progressive politics as a whole after his friend Betty van Patter, who worked for the Panthers on Horowitz’s recommendation, was kidnapped and killed in 1974. Just before she disappeared, she had fallen out with Brown. The police never linked the disagreement to the crime and no one has ever uncovered any evidence to suggest Brown was involved. She has denied having anything to do with Van Patter’s death. But Horowitz and others remain convinced the Black Panthers were responsible for his friend’s murder.

Brown left the party soon after Newton’s return, when he authorised the beating of a woman who ran the Panther Liberation School. Finally, she had had enough of the party’s sexism and misogyny. As for the others, Seale wrote a cookbook, advertised Ben & Jerry’s ice cream and remains involved in black studies. Newton, a cocaine addict, was shot dead as a result of a longstanding gang feud outside a crack house in Oakland in 1989. Huggins went to the funeral of her rapist. What does she think of him now?

Watch the trailer for The Black Panthers: Vanguard of the Revolution.

“You know, Andrew,” she tells me. “I can’t think of any human being that hasn’t had to battle with their own internal struggle. The Huey I first met was my dearest friend and ally to everyone I knew. And I loved him. I know that every human in the world has brought about harm, so I don’t have a judgment, so to speak.”

Cleaver returned to the States in 1975 and got involved in various Christian sects before finally becoming a Mormon and a diehard Republican.

It was an ignominious ending for a revolution. But there was a moment, and it’s there to see in Nelson’s rousingly captivating film, when the future looked very different. And as long as African Americans are subjected to police brutality and racism, it remains a future that’s well worth remembering.

The Black Panthers: Vanguard of the Revolution is out on Friday 23 October. Click here for information and details of screenings with director Stanley Nelson and former Black Panther Mohammed Mubarak