‘This was the most important era in standup comedy; this is when it all changed,” says Michael Aguilar, executive producer of Sky Atlantic’s high-profile new US import, I’m Dying Up Here. “This is when comedy went from setup-punchline jokes to storytelling, to therapy, to creating characters.” The era was the early 1970s, and the standup scene was Los Angeles, where the careers of many soon-to-be comedy superstars, including Richard Pryor, Steve Martin, David Letterman and Robin Williams, were just being launched. “In 1973, when Johnny Carson moved The Tonight Show from New York to LA, that became the scene,” continues Aguilar. “It moved from the east coast to the west coast, and these guys were suddenly the kings of the Sunset Strip.”

Which is not to say that those making their first forays on to the spotlit stage enjoyed overnight success. Certain of the struggles portrayed in the comedy-drama series come straight from the annals of producer Jim Carrey, who optioned William Knoedelseder’s 2010 bestselling book on the period – also called I’m Dying Up Here – on which the show is based. “I lived in a closet when I first came to LA,” he confessed at a Television Critics Association panel earlier this year. “I met somebody at the Improv [a renowned comedy club] who said they had a room, but it turned out to be a closet.” Aguilar later tells me that while living in said closet, Carrey was also dating the singer Linda Ronstadt. “I am pretty sure he went to her place,” he quips.

Such dimly lit, booze-fuelled, late-night depictions of the comedy club world and curious lives of standup comedians would appear to be having a moment. HBO’s Crashing, the semi-autobiographical story of its creator and star Pete Holmes’s early days on the circuit is currently filming its second season in New York, where Amy Sherman-Palladino, creator of The Gilmore Girls, is also knee-deep in production on her new series for Amazon, The Marvelous Mrs Maisel, the tale of a housewife and would-be standup in the 1950s. The Big Sick, standup comic Kumail Nanjiani’s real-life tale of how he met his wife, is still doing good business in cinemas. And Funny Cow, starring Maxine Peake as a comedian on the northern working-men’s club circuit of the 1970s – which, by comparison, makes living in a closet in LA seem impossibly glamorous – is set for release later this year.

Watch a clip from I’m Dying Up Here.

I’m Dying Up Here stars Melissa Leo as Goldie Herschlag, the fierce matriarch of the fictitious Goldie’s, the club in which aspiring comedians, including Bill Hobbs (Andrew Santino), Eddie Zeidel (Michael Angarano) and Cassie Feder (Ari Graynor) vie for coveted spots; Goldie herself is closely based on the famed Mitzi Shore, who founded Los Angeles club The Comedy Store in 1972. “They really captured the feeling of the era, the feeling of the camaraderie and the competition, and the smartass quality of being a comic,” said Carrey of the show. “Oftentimes it wasn’t who was funniest on stage; it was who was funniest at the bar or who was funniest in the parking lot. That’s what mattered to comics.”

This authentic transposition of club to screen stands in stark contrast to the apparent problem in capturing another creative industry – the music business – and turning it into successful television drama. The series Empire and Nashville stand out as notable recent exceptions, but HBO’s big-budget period piece, Vinyl, set in the record industry of 1970s New York City, was canned after one season, in spite of a creative team that included heavyweights Mick Jagger and Martin Scorsese. Showtime’s comedy-drama Roadies, about the backstage world of touring bands, also lasted just one season. The Get Down – Baz Luhrmann’s ambitious Netflix show tracing the birth of hip-hop, also in 70s New York, and notable for costing a reported $120m for its first season – has recently been cancelled, too.

“The world of standup comedy, and the world of being a musician in a band both fascinate people – because it is nightlife, and travelling, and there’s a grungy aspect to it, so it seems a ripe world to exploit – yet it is so hard to capture the authenticity of those particular worlds,” says Sherman-Palladino. She should know; her father, Don Sherman, was a standup, and she spent her childhood in California “with a group of Jews sitting around in my backyard, talking about the good old days in the Catskills [the upstate New York area where standup boomed in the postwar years].” She later worked at The Comedy Store, for Shore, a friend of her father. “But even if you don’t know what it’s like to be in a band, even if you don’t know what it is like to be a standup comic, you can smell it when it is not authentic,” she says of audiences.

Aguilar, meanwhile, believes the contrast between the fortunes of the two scenes onscreen mirrors that of them off-screen, too. “I think the music scene, right now, is just not as vibrant as the comedy scene is,” he says simply. “You can’t get into The Comedy Store on the Sunset Strip on a Tuesday night unless you know someone who can get you on the list; it is completely sold out, and that was not the case five or 10 years ago. And, at the same time, Tower Records can’t stay in business one block further up the street.”

Ill mannered ... standup comedian Kumail Nanjiani plays himself in The Big Sick. Photograph: Nicole Rivelli/StudioCanal

The febrile political atmosphere is a significant factor in those fortunes, he believes. “As much as we connect to music emotionally, we connect with comedy intellectually, with a comedian telling you: ‘This is another way to look at the world,’” he says. “And I think, at certain moments, like now, or in the mid-1970s, there is a greater need for that.” As Carrey commented of the era depicted in I’m Dying Up Here: “We were coming out of Vietnam and Nixon. It was a very intense time, and that helped give birth to a new attitude and a new desire to express yourself and say those edgy truths.”

“You have an audience hungry for that again now because there is anger and fear and frustration,” agrees Aguilar.

The recent British-made documentary Dying Laughing features interviews with some of the most celebrated standups including Billy Connolly, Jerry Seinfeld, Eddie Izzard, Steve Coogan, Amy Schumer and Jo Brand. And there’s no doubt, according to Lloyd Stanton – one of its directors, who calls it “a love letter to comedians” – that such comics have a more crucial role than ever to play in the current political climate.

“As Chris Rock says in the film: ‘It sounds corny, but we are the last philosophers,’” Stanton says. “And that is what you think when you see Louis CK and Chris Rock, or you watch Colbert or The Daily Show: in an era of fake news, you go to these people, feeling that they are able to tell the truth.” Stanton and his co-director Paul Toogood will further examine the genre in a 10-part television series, exploring what they see as “this new golden age of standup”. Their previous work includes the documentary Something from Nothing: The Art of Rap. “It struck us that standup comedy was also, perhaps, similarly misunderstood, and that standup comedians possibly don’t get the praise they deserve as artists,” says Toogood.

Watch a clip from Dying Laughing.

With comedy, Stanton adds, there is an inherently high risk of failure: “It’s a bit like walking on stage and picking up a violin, and learning to play and write music in front of a live audience. You can’t practise on your own, you can’t rehearse on your own; it only works in front of an audience.”

“That’s what makes it very exciting,” enthuses Crashing creator Holmes. “There is no Google Hangout version of standup. I mean, there is; you can do a VR standup show, but it’s a little bit like sex, in the sense that there’s no digital substitute for a real, live audience.” For makers of scripted drama, standup also offers a useful vehicle to elucidate the social and political context, without as much heavy-handed exposition. David Flebotte, a former comedian himself who now heads up the writing team for I’m Dying Up Here, says: “When Cassie does her big blowjob routine on stage, we are not saying: ‘This is great standup’, but we are saying that for a woman to do this onstage in 1973 was a big freaking deal.”

“There is something very confessional about standup comedy, it’s as if you were in therapy,” notes Michael Showalter, director of The Big Sick. “So you have an environment in which your characters are entitled to speak their truth in a way that doesn’t feel contrived.” Showalter also believes the comparative lack of glamour makes standup a better fit for on-screen entertainment than music. “The banality of comedy is not sexy, but it’s very relatable,” he says. “I can’t relate to the life of a rock star. I can’t relate to what it is like to sleep all day, play a show and then party all night. But I can relate to a comedian like Jerry Seinfeld, where every little thing is funny to him but his life is really quite boring. He’s not doing cocaine off supermodels’ butts. At least, not that I am aware of.

“I think there is a fair amount of vanity in musicians; they care deeply about how they look, about the image that they project,” he continues. “That is not an appealing thing; people can’t relate to that idea of: ‘I’m so sexy, I am so comfortable with my sexuality, look at my body.’”

Housewife superstar ... Rachel Brosnahan in Amazon’s The Marvelous Mrs Maisel. Photograph: Sarah Shatz/Amazon

Comedians, he posits, are very much the opposite, lampooning themselves for laughs and stripping back vanity in favour of raw vulnerability. “Comedians shine a light on their own insecurity. So for an audience – at home or in a club – there is an invitation there from the comedian saying: ‘Don’t worry, however insecure you are, I am a thousand times worse.’” It’s a cleverly subversive way to get girls, I note: playing up your weaknesses. “Yes, you’re shy, you are socially awkward, and you go onstage and tell jokes, and suddenly everyone wants to talk to you,” Showalter agrees. “You are using your insecurity as a weapon, as a tool.”

For television and film scriptwriters, too, what could be a better tool for drama than a physical venue that serves as a showcase for human frailty and roiling insecurity? Of course, most punters don’t queue round the block at The Comedy Store consciously hoping to witness hours of raw vulnerability. But in a year characterised by anxiety, despair and political chaos, they are almost certainly seeking more than simply a belly laugh. “Music is about letting go of inhibition, about escaping the bondage of everyday life,” continues Showalter. “Whereas I think comedy, on a lot of levels, is about making sense of everyday life.”

And, goodness knows, we need all the help we can get with that right now.

I’m Dying Up Here starts on Wednesday 16 August, 10.10pm, Sky Atlantic; Dying Laughing is on Amazon Prime, iTunes and other streaming services; The Big Sick is on general release; the pilot for The Marvelous Mrs Maisel is on Amazon Prime