When I was expecting my first child, my husband arrived home with a parenting book. Looking out from the cover, resplendent in a jazzy 80s sweater, was Bill Cosby twinkling confidently under the title, Fatherhood. Who, he seemed to be asking, wouldn’t want advice from “America’s Dad”?

Plenty of people today. In 2005, Cosby was accused of drugging, then sexually assaulting the director of a women’s basketball team. When the prosecutor declined to take it further, the 30-year-old launched a civil suit, in which 13 other women lined up to testify that Cosby had assaulted or raped them, too. Then the case was settled out of court and, somehow, Cosby’s reputation was barely dented. Cosby himself has repeatedly denied any wrongdoing.

In the years that followed, the comedian was given awards, went on talkshows, and had a weighty biography written about him, with little notice of the number of allegations made against him. Today, as he once more faces a criminal trial for sexual assault, and the number of women accusing him of sexual misconduct has spiralled to almost 60, this seems inexplicable. Which is why a new BBC documentary, Cosby: The Fall of an American Icon, asks accusers, colleagues and journalists how the silence continued for so long.

Cosby is hardly the first well-loved star to be accused of stomach-churning crimes. Yet his artistic and political achievements – and what the case says about race relations in the US – make the story unique, according to director Ricardo Pollack. The 79-year-old actor, he points out, is not just a “comedic genius”, whose 50-year career influenced generations of performers from Richard Pryor to Chris Rock, but a “role model and pioneer” who kicked down racial barriers.

It was in the early 1960s that Cosby made his name in standup. As the civil rights movement unfolded, the Philadelphia-born son of a maid and an alcoholic naval steward father somehow managed to entrance black and white audiences alike with his funny, perfectly crafted stories.

He had made it – and a black man making it in America was a big deal then Jennifer Lee Pryor

Bill Cosby with Robert Culp in the 60s TV spy caper I Spy. Photograph: AP

In 1965, this won him a part in I Spy as a secret agent, making him the first black actor to star in a network TV drama. He won three consecutive Emmys for the role, and a generation of fans. Movies, the animated series Fat Albert and commercials followed. In Pollack’s documentary, Richard Pryor’s wife, Jennifer Lee, notes: “He had made it – and a black man making it in America was a big deal.” Then, in the 1980s, came The Cosby Show. It was an instant hit, and a revolutionary one. Against a backdrop of what Pollack calls “Reagan and crack cocaine” with negative portrayals of African American communities flourishing in the media, the sitcom offered up an alternative vision: an aspirational, stable, upper-middle-class family.

Joseph C Phillips, who played Cosby’s son-in-law on the show, tells me it was something that had not been seen on US TV before. “She [Clair Huxtable, played by Phylicia Rashad] was a partner in a law firm, he [Cosby’s Cliff Huxtable] was an obstetrician,” he says. “They lived in Brooklyn Heights, their daughter went to Princeton, they were good looking. The kids weren’t all getting As, but their parents ran the household.”

With “universal issues of parenthood” rather than race at the heart of The Cosby Show, it became one of the most popular sitcoms of all time. That white audiences grew to love this black American family also helped to foster the idea that racial tension in the US was a thing of the past. As Phillips points out, the fictional Huxtables “became everyone’s family”. By election night 2008, George Bush’s former strategist Karl Rove was claiming the sitcom had paved the way for a black family in the White House.

For Cosby, the show not only cemented his wealth and power, it bound him in the mind of the public with his paternal, authoritative on-screen persona, Cliff Huxtable. After all, the Huxtables were said to be based on Cosby’s own brood. As Jelani Cobb put it in the New Yorker, Cosby was seen by many as “the embodiment of black dignity, a walking refutation of the worst ideas about us”. Off screen, he championed black causes, including donating $20m to a black women’s college in 1988, while on screen he raised the profile of Historic Black Colleges and Universities with the spin-off series, A Different World (1987-1993).

No one would believe me because it was Bill Cosby … the father of all fathers Lili Barnard

Lili Bernard in the documentary Cosby: Fall of an American Icon.

Lili Bernard – then an aspiring actor, now a visual artist – was mentored by Cosby in the 90s and says this commitment was visible behind the camera, too. Over the phone from her home in LA, she tells me that while most TV crews were exclusively white, The Cosby Show had black people “in hair, in wardrobe, in makeup, as production assistants”.

At first, she says, she believed that the paternal image was grounded in Cosby’s personality. “He often told me: ‘You are one of my kids, Lili,’” she remembers. But, a year after his mentoring began, everything changed. One day he told her to come and meet him in New Jersey so he could introduce her to a producer. Instead, she says, he drugged and raped her. She says in the documentary that when she threatened to go to the police, he told her: “You are dead to me … I will erase you.”

Now a mother of six, Bernard says she felt her life was being threatened and that “no one would believe me because it was Bill Cosby”, not just “the father of all fathers” but a powerful and rich man. Added to this was the responsibility of knowing that speaking out would “reveal to the world ... that a seemingly upstanding, iconic black male who has done so much to advance the black image is nothing but a lying, cowardly, depraved rapist.”

Victoria Valentino, 75, says she understands Bernard’s reticence in coming forward, as well as her anger. She said she waited more than 40 years before finding the courage to say that Cosby raped her in the early 1970s. Now a grandmother, Valentino says she was at her lowest ebb when she met Cosby. Her six-year-old son had recently drowned in a swimming pool, and she was beside herself with grief. One day she agreed to drive her roommate to Hollywood’s Cafe Figaro where the pair bumped into Cosby, who took them to dinner.

I saw him becoming so famous and so hypocritical. I couldn’t stomach it. I wouldn’t let my children watch his show Victoria Valentino

Victoria Valentino, in the documentary Cosby: Fall of an American Icon.

Valentino was miserable and “biding my time until I could go home” when she says Cosby gave them both a pill, telling Valentino, “it will make you feel better”. Afterwards, she says, she could hardly hold her head up, but after offering to drive the pair home, Cosby took them to an office and raped her while her roommate was unconscious.

Speaking to me from the US, she says it was hard to even confide in her friends, because “it was so disgusting and you feel so embarrassed and ashamed and humiliated”. In the documentary, she reels off the reasons why she assumed the police would not believe her, including the fact she had been a Playmate centrefold and a Playboy bunny. As the years passed, she says, she slowly put her life back together. But seeing Cosby on television “rankled me; I saw him becoming so famous, and so hypocritical with this ‘America’s Dad’ [persona]. I couldn’t stomach it. I wouldn’t let my children watch his show. And the older he got the more disgusting he looked to me.”

In 2005, another woman accused Cosby of sexual assault, this time to the police. Basketball coach Andrea Constand said that a year earlier, during a visit to Cosby’s house for career advice, he had drugged and attacked her. When the prosecutor did not take the case further, Constand filed a civil suit for sexual battery and defamation. Among the 13 women who came forward to strengthen Bernard’s case was former model Beth Ferrier. She waived her anonymity to say that Cosby drugged and raped her, too. The actor was forced to give four days of confidential evidence but ultimately agreed to settle.

Since then, the accusations have barely been discussed for almost a decade. Philadelphia Magazine ran a big article on the claims in 2006 but, for the most part, the media moved on. Pollack says people “didn’t want to engage with the story”. Instead, Cosby continued to be feted. In 2009, he was honoured with the Mark Twain prize for humour by Jerry Seinfeld and Chris Rock, where Michelle Obama was in the audience. In 2014, a former Newsweek editor wrote a 500-page biography on Cosby, scarcely addressing the rape allegations. Cosby planned a new spate of programmes, including a new comedy special with Netflix and an NBC sitcom.

Hannibal Buress’s Bill Cosby joke, which went viral and reopened questions about his public persona.

What stopped him in his tracks, says Pollack, were not the accusations of rape, but of hypocrisy. Since 2004, he had been a crusader; touring the country to lecture poor, black communities telling them that their problems were not down to structural racism but their own failings. In one infamous speech he put it like this: “There are people being shot in the back of the head over a piece of pound cake. And we all run out and are outraged: ‘The cops shouldn’t have shot him’... What the hell was he doing with the pound cake in his hand?”

It was too much for comedian Hannibal Buress. He was so infuriated at Cosby’s hectoring of young black men that he skewered Cosby in his standup act. “Pull your pants up, black people, I was on TV in the 80s,” he mocked, then deadpanned. “Yeah but you raped women, Bill Cosby.” A clip of the show went viral, and finally the allegations against Cosby were headline news.

Valentino was one of the alleged victims who saw the video. “I just froze. Then, a red rocket of anger exploded in my brain.” After years of thinking “he would be like other famous, powerful jerks who carry on their way and leave damage in their wake with no consequences”, she was finally ready to speak out. Yet she says the fact that it took a man joking about Cosby to make the public listen “shows the weight a man’s word carries over the weight of a woman’s word”.

Waves of women followed, with claims spanning Cosby’s entire career. Lili Bernard was among them. It wasn’t easy, she admits. For some people in the African American community, she says, the bleak history of black men in the US being wrongfully accused of rape made them sceptical of the allegations. While for others, the “America’s Dad” image carried a special weight “because there are many black people whose fathers are absent. Either because they are working two jobs, or on the road driving a truck, or wrongfully incarcerated … It’s hard for them to release the image [of] the father they never had.”

I was angry, I felt I had been lied to. I had long ago accepted he was human ... but this is something else Joseph C Philips

Former Cosby Show star Joseph C Phillips.

But Cosby’s lecturing had other repercussions. The Associated Press used it to argue that, as a “public moraliser”, he no longer had a right to privacy and his confidential evidence from 2005 should be made public. When the deposition was opened, it revealed that Cosby had admitted giving women Quaaludes with the intent of having sex with them, although he insisted it was consensual. This, in turn, resulted in prosecutors looking again at the Constant case and in 2015 Cosby was charged with three counts of assault.

Joseph C Phillips admits he did not believe the reports about Cosby, but when Lili Bernard, with whom he was friends, came forward, he had to change his mind. “I was angry, I felt I had been lied to. I had long ago accepted he was human ... but this is something else.”

As public opinion also started to swing against Cosby, Netflix and NBC pulled his new shows. His trial has yet to begin, but Valentino says that the alleged victims have already scored victories. Many of his accusers were prevented from pressing criminal charges because of the statute of limitations in their states, but have successfully campaigned to overturn that in California, Colorado and Nevada.

“I have a black girlfriend who won’t speak to me because I ruined the legacy of this great, black man,” she says, “but we have opened up the conversation about rape and sexual assault ... This is my legacy.”

Cosby: Fall of An American Icon, airs on 5 June, 9pm, BBC2