Not one woman was nominated for this year’s best director Oscar. But some of the hottest forthcoming movies are female-led – so has gender discrimination in the industry been busted?

A student called Ruth Bader Ginsburg has been invited to dinner by the dean of Harvard Law School, along with the eight other women in her year. Her wide-eyed eagerness is swiftly put in check when the dean asks each woman to stand up and say something about themselves, including “why you’re occupying a place that could have gone to a man”.

This is an early moment in On the Basis of Sex, a biopic about Ginsburg, who eventually became a US Supreme Court justice. It’s a scene that Mimi Leder, the film’s director, could identify with well. In 1973, Leder became the first woman to graduate from the prestigious American Film Institute Conservatory: “I never entered the world of film-making saying, ‘I’m entering a male-dominated business.’ I entered the world of storytelling because I felt compelled to tell stories. But throughout my career, I’ve been predominantly the only the woman in the room.”

On the Basis of Sex has been referred to as Ginsburg’s “superhero origin story”. With Leder, however, things were not so straightforward. This is her first movie for nearly 20 years. Like a great many female directors, she found herself prematurely and unfairly shut out of an overwhelmingly male-dominated Hollywood. But, like a great many female directors, she is now returning.

Leder’s career got off to a flying start. Having seen the dynamism she put into hospital drama game-changer ER, Steven Spielberg invited her to direct the first movie from his fledgling DreamWorks studio: 1997’s The Peacemaker, starring Nicole Kidman as a nuclear expert and George Clooney as the commando who has problems taking orders from a woman.

Leder’s next, Deep Impact, was a huge hit but its follow-up, the Kevin Spacey drama Pay It Forward, was not. The good scripts stopped coming and Leder began to realise she was in “movie jail” – the place directors are sent to after a commercial flop. “It’s hard coming back from that emotionally,” says Leder. “But I never stopped directing. I directed nine pilots for television and six went on to become series.” Leder’s subsequent TV credits include The West Wing, Shameless, Luck and The Leftovers, the HBO hit on which she was co-showrunner.

Movie jail can be a life sentence. You could fill a whole prison with women whose careers got off to a bright start, only to falter

When it comes to movie jail, sentences are far lighter for men. It is hard to think of a male director who hasn’t made a flop at some stage, then gone on to redeem themselves: David Fincher (Alien 3), Joss Whedon (Serenity), Richard Linklater (Fast Food Nation), Steven Soderbergh (Logan Lucky, Haywire), even the Coen brothers (Hail Caesar!). Some seem to have made an entire career out of failing upwards: Zack Snyder’s reward for such expensive flops as Legend of the Guardians and Sucker Punch was custodianship of Warner/DC’s superhero franchise.

Facebook Twitter Pinterest On the loose … Karyn Kusama, whose film Destroyer is on release. Photograph: James Bridges/Screen Gems/NYT

For women, though, movie jail can be a life sentence. You could fill a whole prison wing with similar cases to Leder’s, women whose careers got off to a bright start, only to somehow falter. But the prison breakout seems to be gaining momentum. Another escapee is on the loose right now: Karyn Kusama, who has returned with Destroyer, starring Nicole Kidman as a ravaged, world-weary, antisocial detective. Destroyer is a superior Los Angeles thriller in the hard-boiled, neo-noir tradition, but Kidman’s character is no gender-swapped male gumshoe: she carries the guilt of being a terrible mother, for one thing. And when was the last time Philip Marlowe had to give someone a hand job in exchange for information?

Like Leder, Kusama is back after a long absence. She made a big impression with her 2000 debut Girlfight – another story of a woman in a man’s world, starring then newcomer Michelle Rodriguez as a spirited Latina boxer. It won prizes at Sundance and Cannes, Variety hailed it as “the arrival of a major new talent”, and Kusama was soon snapped up by Paramount for Aeon Flux, a lavish sci-fi movie starring Charlize Theron.

The experience was not a happy one. Paramount wanted something like The Matrix, while Kusama envisaged a sort of dystopian Crouching Tiger, Hidden Dragon. The film was taken out of her hands and edited into an incoherent, 71-minute mess. For extra humiliation, the studio then asked Kusama to return and edit its mess into something slightly less messy, which still bombed at the box office.

Facebook Twitter Pinterest Back from the Twilight zone … (clockwise from top left) Kimberly Peirce, Lisa Cholodenko, Mimi Leder and Catherine Hardwicke. Composite: Reuters/Alamy/Rex/Getty Images

So Kusama found herself in movie jail, too. “Maybe it’s supposed to sound like a rite of passage,” she said a few years ago, “but so few women get any opportunity to have more than just the rite of passage – which is a big part, I think, of what we really need to be talking about when it comes to women’s careers in film.”

Kusama came back with 2009’s Jennifer’s Body – a sharp, feminist teen horror penned by Diablo Cody, and another commercial flop (although it is currently undergoing a well-deserved critical reappraisal). Low-budget horror The Invitation helped restore her reputation. But since Girlfight, Kusama has directed just four movies this century. In the same timespan, Richard Linklater has made 14, Steven Soderbergh 21.

According to the University of Southern California’s Annenberg Inclusion Initiative, of the top 1,100 Hollywood films from 2007 to 2017, a mere 53 were directed by women. That’s a lamentable 4%. And almost all of these women only made one movie (the exact figure is 84%, compared with 55% of men).

When you tally up the women directors with similarly curtailed careers to Leder and Kusama’s, it starts to look like a lost generation. Many made their names in the late 1990s and early 2000s, but spent the next two decades struggling in features or switching to TV. Patty Jenkins had a dream debut with 2003’s Monster, the story of serial killer Aileen Wuornos. It won Charlize Theron many awards, including the best actress Oscar. Jenkins’ reward? Fifteen years in film-development hell, supplemented by TV gigs.

Facebook Twitter Pinterest Wonder woman … Patty Jenkins. Photograph: Richard Shotwell/Invision/AP

Kimberly Peirce directed Hilary Swank to an Oscar in 1999’s Boys Don’t Cry. It took nearly a decade until her follow-up, 2008’s Iraq war drama Stop-Loss, followed by a remake of Carrie in 2013. Tamara Jenkins’ semi-autobiographical Slums of Beverly Hills played at Cannes in 1998. Her follow-up came nearly a decade later: 2007’s The Savages, which earned her a screenwriting Oscar nomination.

Playing alongside Jenkins’ film at Cannes that year was Lisa Cholodenko’s High Art. Cholodenko has made just three films since, the last being 2010’s The Kids Are All Right. It is not just about commercial failure, either. Catherine Hardwicke, who broke through with the Sundance-winning teen drama Thirteen, went on to direct the smash hit Twilight, which ought to have put Hollywood at her feet. But she was not invited back for the sequels.

The good news is that almost all of these women are back behind the camera, making female-centred films. Patty Jenkins, of course, saved DC’s comic-book universe (and undid Zack Snyder’s damage) with Wonder Woman, the highest-grossing film ever directed by a woman. She’s currently working on the sequel. Tamara Jenkins returned after a 10-year hiatus with last year’s excellent fertility drama Private Life. Peirce is now working on This Is Jane, about the women who provided abortion services in 1970s America. Cholodenko is directing a remake of the hit German comedy Toni Erdmann. And Hardwicke has a Tijuana-set crime thriller, Miss Bala, coming out in March.

Facebook Twitter Pinterest Fertility drama … Tamara Jenkins directing Kayli Carter in Private Life. Photograph: Jojo Whilden/Netflix

All cause for celebration, but you wonder what we might have missed. Had these women been afforded the same latitude as their male counterparts, they could be established auteurs rather than occasional directors. Some women have built such careers, of course: Sofia Coppola, Kathryn Bigelow, Ava DuVernay. But many more have either struggled or dropped out of film-making altogether.

“I think you have a generation of women who will never know if they could have been successes because they never had the opportunity,” says Melissa Silverstein, founder of Women and Hollywood, which campaigns for diversity and equality. The factors preventing women from having sustained movie careers are numerous, Silverstein says. There is institutional sexism, conscious and unconscious, as well as motherhood.

You can't have a conversation about women and the decisions they make without understanding how inhospitable the industry is to people with children

“You can’t have a conversation about women and the decisions they make without understanding how inhospitable the industry is to people with children,” she says, pointing out that directors such as Tamara Jenkins took time out to raise their children. “But once you get off, it’s very difficult to get back into this industry.”

The #MeToo movement has moved opinion in a positive direction, though Silverstein points out that most films around now will have been greenlit over two years ago, before #MeToo started. And younger women are already benefiting from the changed landscape. Silverstein cites Chloé Zhao, who has gone from hailed indie The Rider to directing a big Marvel movie, The Eternals. Then again, this year’s Academy award nominations demonstrate how far there is to go: Marielle Heller’s Can You Ever Forgive Me? is the only film directed by a woman to figure in any of the main categories.

Facebook Twitter Pinterest Changing landscape … Chloé Zhao, director of The Rider. Photograph: Taylor Jewell/Invision/AP

Silverstein says it’s about “people with juice using it for good”. Like those who have pledged to adopt an “inclusion rider”, requiring a certain proportion of staff on a project to be women, people of colour, LGBT people and people with disabilities. The list of adoptees now includes Brie Larson, Ben Affleck, Matt Damon, Michael B Jordan and, on a company level, Warner and HBO.

Nicole Kidman has pledged to work with at least one female director every 18 months – hence Destroyer, Coppola’s The Beguiled, and Jane Campion’s Top of the Lake. And just this week at Sundance, Time’s Up and the Annenberg Inclusion Initiative launched the 4% Challenge, urging actors and film-makers to improve on that depressing statistic by following Kidman’s example. The idea is gaining momentum: early accepters of the 4% Challenge include Tessa Thompson, Amy Schumer, JJ Abrams, Reese Witherspoon and, on Wednesday, a whole Hollywood studio: Universal Filmed Entertainment Group, which includes Universal Pictures, Focus Features and Dreamworks Animation.

Leder has been insisting on gender parity for years. Despite her Hollywood “comeback”, she has not been yearning to make movies all this time. She’s happy in TV. “There are so many incredible, great quality stories you can’t tell on the big screen,” she says. She is now directing and co-producing a series for Apple TV, starring Jennifer Aniston, Reese Witherspoon and Steve Carell. Again, she’s hiring as many women as men. “I see a lot of white guys out there going, ‘Gee, I’m not able to get a job any more.’ I think, ‘Now you know what it feels like.’”