The atmosphere inside the Red Brick Cafe Bar on the night of 11 February 2018 was unexpectedly raucous. The Robinson College student venue – which, alongside undergrad-cheap pints, sells bags of dried pasta and tins of baked beans – was booked for an evening of music and comedy from a lineup of black and minority ethnic (BME) Cambridge University students. When one of the acts, standup comic Hasan Al-Habib, arrived, however, he was surprised to see the bar packed with revellers, many obviously drunk from a beer festival that had run throughout the day. So many people had spilled into the venue that the audience for the open-mic night had been pushed into a corner. “I walked in and immediately knew it was going to be terrible,” Habib recalls.

At 8pm the lights fell and, undeterred, the first act struggled to make her rendition of Nina Simone’s I Loves You Porgy heard over the din. Next, Habib, a Muslim who grew up in Birmingham, took to the stage. The point of the evening was to provide a space where BME performers could articulate their frustrations and struggles, and Habib had included a few jokes about race. “Microaggressions are small acts of everyday discrimination, but they do build up,” he ventured. “They can be really subtle things, like someone saying, ‘Where are you actually from?’ or, ‘Your English is really good.’ Or, ‘What’s in the bag, you fucking terrorist?’” Most of the crowd talked over the punchline. Habib limped through the rest of the set.

After a second standup act, the organisers took a break to address some technical issues. While the performers encouraged one another in a back room, one of the night’s organisers, Sara Poursafar, entered in tears. There had been a complaint, she explained, that Habib’s material was “racist to white people”. Customers had told bar staff they wanted to dance, not listen to comedy, she added. Despite the fact that the venue was booked, staff had decided to cancel the event. Poursafar was told she was free to reschedule.

Every standup comic has endured a rough gig in front of an uninterested crowd, but this was something else. “Literally and metaphorically, the night was created as a space for BME students to be heard,” Poursafar wrote in a blogpost the following morning. The cancellation had sent a clear message, she argued: “White comfort is more important than BME voices.”

In any other city, the cancellation of a student comedy night would be provincial news. But Cambridge is home to Footlights, Britain’s most illustrious and elite comedy society. Throughout the 20th century and well into the 21st, Footlights has been the club where writers, performers and producers meet as students and move, seemingly without friction, into establishment jobs in radio, television and film. Its alumni have set the tone of cultural eras from the 60s to today, via shows such as Beyond The Fringe, Monty Python, The Goodies, Not The Nine O’Clock News and, more recently, Peep Show, QI and The Great British Bake-Off. If an industry routinely accused of being too white and too male is to diversify, Cambridge student comedy has a key role to play in nurturing precisely the kind of new voices booked at the Red Brick Cafe Bar. Footlights’ silence on the event was telling.

Founded in 1883, Footlights has, during its 136-year history, produced a panoply of luminaries, including Peter Cook, John Cleese, Emma Thompson, Hugh Laurie, Douglas Adams, Sandi Toksvig and Stephen Fry. “Like a night sky in the countryside, the more you look, the more stars you see,” wrote comedian and Footlights alumnus David Mitchell in his 2013 memoir, Back Story. “Footlights seem[s] to be behind about half of the stuff worth paying attention to.” YouTube and podcasts, those freshly paved routes to success for young comedians, have done little to blunt the society’s influence in recent times. Former Footlights members continue to dominate positions of power and influence – from Olivia Colman, this year’s Oscar-winner for best actress, to John Oliver, America’s leading satirist du jour. Standup and 2012 Footlights president Phil Wang describes membership as a form of entry to the comedy mafia. “There’s a little nod,” he tells me, of encountering a fellow former Footlight on the circuit, “and you just know.”

Old-fashioned Monty Python-style comedy has a very white, specifically English sensibility that can shut people out

For Habib, Footlights was the root of his ambition to study at Cambridge, seeded at the age of 15 when he watched an episode of Celebrity Mastermind. The contestant, David Mitchell’s double-act partner Robert Webb, was asked what advice he might give to someone hoping for a career in comedy. “He gave this long-winded answer about Footlights,” Habib recalled. “At the end, John Humphrys said, ‘So, basically what you’re saying is go to Cambridge?’ Webb laughed and said, ‘Yeah.’ And I thought, OK, I’ll go to Cambridge.”

At 18, Habib applied to study chemical engineering. He was pooled (recommended for consideration by colleges other than the one to which he had applied), then rejected. He took a year out, then reapplied, only to be pooled and rejected again. After three years studying biochemistry at Imperial College London, he took his master’s in radiation biology at Oxford University. Then, in a final effort to enter Footlights’ orbit, he applied for a PhD at Cambridge. Finally he received an offer. A few weeks later, at the university freshers’ fair, Habib spied the Footlights stand. Here was the institution within an institution that, for seven years, had been his siren call. “I could have cried,” he recalls.

Footlights alumni Robert Webb and David Mitchell, 2005. Photograph: Linda Nylind/The Guardian

Despite the club’s daunting history, Habib was confident that he could secure a spot performing in a “smoker”, the name given to Footlights’ informal late-night performances. (A contraction of “smoking concerts”, the smokers of the early 1900s were cabaret nights watched by an exclusively male audience wearing dinner jackets; today women are permitted to both watch and perform, but smoking is forbidden.) Having regularly performed standup in London, Habib felt his material was strong. But three consecutive auditions in 2016, in which he performed routines about growing up in a Muslim household and joked about Isis (the dog from Downton Abbey), were met with silence by the all-white judging panel.

First years who try out for Footlights shows are often left disappointed. (Webb, who describes the Footlights committee as “the ruling elite, the crème de la clown”, was turned down in every audition during his first year.) Still, Habib suspected the theme, not the quality, of his work had led to his trio of rejections. “You could just see this reaction on their faces, like, ‘There is no way we can laugh at this,’” he recalls. (When he finally performed the Isis the dog sketch at a smoker the following year, it received a warm reception.)

Habib is not alone in his suspicion that the lack of cultural diversity in Footlights at that time created an unconscious bias against certain kinds of material. “Old-fashioned sketch comedy, in the style of Monty Python or Peter Cook, say, has a very white, specifically English sensibility,” says Wang who, like Habib, applied to Cambridge principally to join Footlights. “If the committee adheres too much to those old sensibilities, those who don’t embody them – who maybe, like me, also grew up on American standup and Asian comedy – can feel shut out during auditions.”

Exclusivity goes against the founding tenets of the club, which was started as a rival to Cambridge’s Amateur Dramatic Club (ADC), membership of which was once described as being limited to old members of the Eton and Harrow cricket elevens. In more recent times, Footlights has failed to reflect the diversity of the broader university population. While former club presidents include the comedians Richard Ayoade, whose father is Nigerian, and Wang, whose father is Malaysian, diversity in the club has continued to lag behind that of the wider university, where 26.8% of students currently identify as BME. “From my first foray into Footlights, I was hyperaware of the lack of BME comedians involved,” says Adrianna Hunt, a biracial student and current member of Footlights. “It compounded the overwhelming feeling of impostor syndrome, which naturally comes with studying at Cambridge when your identity diverges from white and privately educated.”

Current member Adrianna Hunt: ‘We are at the beginning of a long journey to make Footlights more accessible.’ Photograph: Gareth Iwan Jones/The Guardian

When Alex Franklin, who performed and wrote for Footlights’ Edinburgh show this year, appeared in a Footlights sketch show in his first year, it became, he says, “one of the most, if not the most diverse comedy night that year” (his mother is Chinese). The singular background of the Footlights committee had, seemingly, resulted in a kind of collective myopia; material about race was usually rejected. “Many of the old guard perceived it as too ‘edgy’ to be performed under the name of a supposedly apolitical society,” Hunt says. (While Footlights has a long tradition of lampooning political figures, overtly political material has traditionally been discouraged, members say; though the choice to keep material apolitical is, of course, a political one.)

In response, Habib began performing at BME-only open mic nights around Cambridge, where his politically charged material was actively encouraged – until the show at the Red Brick Cafe Bar.

***

The shutdown that night would prove to be a watershed moment in Cambridge comedy. At first, the story went broadly unnoticed. Varsity, a student paper, reported it, but there was little fallout from an incident one attendee, Jun Pang, claimed revealed the “inability of white people to handle jokes about whiteness”. No apologies were made and Footlights – by far the most powerful Cambridge comedy society – issued no statement.

A month later, the university hosted one of its best-known comedy nights, the Wolfson Howler, where students perform alongside professional comics (previous headliners include Josh Widdicombe, Sara Pascoe, Desiree Burch and Paul Sinha). The 2018 event took place on International Women’s Day but, due to a last-minute drop-out, featured an all-male lineup. This time, Footlights did not remain silent. The next day the club’s president, Ruby Keane, issued a statement criticising the lack of female representation. It was, she wrote on the Footlights Facebook page, “a seriously misjudged move from one of Cambridge’s most high-profile gigs”.

This public call-out rankled Habib. In the comments beneath the post, he pointed out Footlights’ all-white audition panel and the fact that the club’s largest show of the year, the Spring Revue, had had an all-white lineup. “Pretty much everything you do,” he wrote, accusingly, “is all-white.”

Habib’s comments “set off a snowball effect”, he says. Keane deleted her post; another member of the committee claimed Habib’s accusations could “discourage BME performers from auditioning”.

People who are privately educated come to Cambridge ready to perform comedy. They know how to put on a show

With tensions high, Keane called a meeting to tackle what she describes as “Footlights’ racial ‘blind spot’. “Keane says she wrote to the 11 other committee members, asking them to attend, but only two showed up. Nevertheless, the decision was made to host an open meeting in the Maypole pub next to the ADC theatre, home of the Footlights, where BME performers could air their grievances and discuss potential solutions. Keane, along with fellow committee member Louisa Keight, scheduled an event to take place a few days later.

Attendees recall a spirit of historic reckoning reminiscent of that forced by Eric Idle who, in 1964, helped secure the admission of women as members. (Women had been accepted as performers since the late 50s, but not members; Germaine Greer, then a mature student from the University of Melbourne, became the first female Footlight.)

This time, ten of the eleven Footlights committee members attended. “It was rammed,” recalls Habib, who estimates between 50 and 80 performers showed up. Ken Cheng, a former committee member who, in 2015, reached the final of the BBC new comedy award, compered. Cheng, who is British-born Chinese, invited BME members of the audience to begin by sharing their perspectives. There was no shortage of volunteers and people spoke, uninterrupted, for a full hour. “Multiple problems were raised,” Franklin recalls. “We discussed why Footlights was so disproportionately white, so private school-dominated, the huge problem of nepotism and how that accentuates biases.”

Unlike many other clubs at Cambridge, the Footlights’ committee was, at the time, hand-picked by the outgoing members. It was claimed they would often choose friends or people they went to school with. The situation was galling, people at the meeting explained, because of the powers this conferred: members can cast themselves as a writer or a performer in any of the society’s shows, including the one that goes to the Edinburgh fringe, one of the largest comedy festivals in the world, and performs in front of prospective agents. “People who weren’t actually very good were still getting on the committee and performing at Footlights Spring Revues,” Habib says.

Keane agreed there was a need for systemic change. She proposed moving to an open-voting system, which would block the outgoing committee from choosing their friends to replace them. “We agreed that anyone who had performed comedy in Cambridge should be able to sign up to vote,” she says. But while the conclusions of the meeting were unanimously agreed by the three Footlights members in attendance, when Keane reported back to the rest of the committee, she was overruled. “I think they had benefited from the system, and they weren’t keen to see it change,” she says.

Having publicly promised change, Keane felt unable to continue in her role as president. In a resignation email sent to the Footlights mailing list later that week, she wrote: “I hope… I have proven how sincerely I believe the committee and voting system needs to change if we’re ever going to have an inclusive comedy scene in Cambridge. In many ways it comes down to who is ‘in’ and who is ‘out’, and how incredibly hard it is to get ‘in’.” She urged the committee members to consider supporting open auditions and open-mic nights.

Monty Python members Graham Chapman, Eric Idle, Terry Jones, John Cleese and Michael Palin, 1970. Photograph: REX/Shutterstock

Keane’s resignation made the national news. “Diversity row at Cambridge Footlights as its president quits over lack of opportunities for non-white students,” ran one Daily Mail headline, to a stream of furious reader comments accusing Keane of being a “silly girl” and “entitled little thing”. Meanwhile, some members of the committee refused to acknowledge Keane’s resignation as one of protest, alleging she was pushed out after a miscommunication resulted in Footlights being temporarily deregistered as a university club. (Keane says the allegation was “a needless deflection of the issues at hand”.)

But her surprise resignation had a ripple effect. When the year ended, the committee members who had been against reform moved on, and the new committee went ahead and implemented everything Keane had promised. In 2019, for the first time in its 136-year history, Footlights is without a president, led, instead, by a democratically elected committee. “It’s not that we’ll never have a president again,” says Amelia Hill, the current secretary. “But while the society remains in a transitional phase, this is the way we need to be organised.”

***

Close to midnight, in a snug venue at this year’s Edinburgh festival fringe, Habib took to the stage alongside fellow Cambridge students Danny Baalbaki and Patrick Sylla. The trio call themselves the FootDarks (“We prefer alt-white,” they joke in the blurb for their event) and are the first BME-only troupe of Footlights performers. “We are not the only two Muslims in Footlights. We are the only two Muslims in Cambridge,” Habib jokes, as he introduces Baalbaki to the tightly squeezed-in audience. Their set is deliciously funny and challenging in the way it engages with race, class and privilege – politically pointed in a way that the main Footlights show, performed across the road in a more leisurely afternoon slot, is not.

A few hours before the night’s performance, I meet the three young men in the courtyard of a Hotel du Vin. Despite performing at the Edinburgh fringe, Baalbaki and Sylla tell me they still don’t feel as if they belong here. It is a sense of alienation that was instilled, Baalbaki says, at school. “I was sat down by my teacher and told: ‘We saw you put Cambridge down as an option. You should take it out.’” Nobody at his school had received an Oxbridge offer before. “They said, ‘If anyone’s going to go there, it’s not going to be you.’”

“Some teachers told me to lower my sights,” says Sylla, who was also discouraged. “I had a teacher who said, ‘In the past, we wouldn’t have suggested for certain kids to go to Oxbridge, because they’ll have a hard time.’ He meant black kids. I was like, ‘You’re an old white dude who is on my side, and even you think it’s going to be hard for me.’”

Messages like these teach a child, Sylla argues, that they do not have a right to exist in certain spaces. “People who are privately educated come to Cambridge ready to perform comedy. They’ve done plays, they’ve done drama, they’ve done theatre, they know how to put on a show. They’ve grown up with a sense of belonging.” For years, Footlights has earned a high proportion of its income by staging workshops and performances at private schools. A lack of exposure heightens the sense of alienation for minorities when auditioning for it, Baalbaki says. “If you come from a similar kind of school to those on the panel, you share a sensibility. You share cultural capital. So there’s an instant divide.”

With so much fame and privilege behind this society, we should use that power for good

This year, as part of the swingeing changes to the way in which Footlights operates, the society visited state schools in Manchester, Chester, Essex, Croydon and Dorset to host workshops for less-advantaged pupils – the first time, in recent years at least, that the club has made a concerted effort to engage with institutions from which it would not turn a profit (the troupe performed ticketed events in each town, using the proceeds to pay for the school visits). “We taught some general acting for the younger kids, demonstrated what a pullback and reveal was,” Hill explains. Many students, she says, have no understanding of the term “sketch comedy” until they explain it is similar to the skits they see on YouTube or the website CollegeHumor. “It just gives them a little head start,” she adds.

Former Footlights president Phil Wang. Photograph: Murdo MacLeod/The Guardian

While the society’s 2018 crisis may seem like a closed chapter, the challenge now is to lock in change. “We are still very much at the beginning of a long journey to making the Footlights more accessible and open,” Hunt says. As well as an increase in the number of BME-only comedy nights, Footlights now supports Stockings, comedy nights for women to try out jokes in a supportive environment. Footlights’ membership system has been overhauled. “We’re trying to set elected roles in stone to avoid nepotism in those future years when, for example, you have a lot of friends coming together,” Hill says. “We can’t guarantee it; we can only hope. But with so much fame and privilege behind this society, we should use that power for good.”

Not everyone is convinced of the usefulness of women- or BME-only nights, however. “The unforeseen negative effect of putting on special nights is that, if you give a certain demographic their special night, it can feel like you’ve ticked the box, so there’s no need for more profound change,” says Wang, whose mother studied at Cambridge in the 70s and encouraged her son to apply. “The imperative is on Footlights to ensure they’re diverse. Measures towards inclusion shouldn’t wind up having the effect of segregation.”

It’s an argument Sylla understands: “If people from ethnic minority backgrounds perform only at their shows, it creates separation,” he says. “But I can’t communicate the awkwardness of arriving somewhere and not understanding how to navigate the spaces. In my experience, these nights create a space where people who wouldn’t otherwise feel comfortable are able to perform.”

The issues with which Footlights has been reckoning reflect those in the wider industry. “From a purely comedic perspective, diversity of voices is just more interesting,” Hunt says. She became so fed up in her first year of hearing similar jokes about middle-class “quirks” that she kept a tally of the times hummus was mentioned on stage. (It’s a trend, Wang says, that extends beyond Cambridge: “For a while everybody seemed to have a hummus bit.”) Hunt goes on: “Endless pastiches of Monty Python and Mitchell and Webb are wearying. Diversity of membership means diversity of perspective. And diversity of perspective is the only thing, ultimately, that prevents comedy from becoming hackneyed.”

The complex issues of structural inequality, bias and class seen in Footlights also define the wider world of professional comedy. Some alumni are choosing to take up the fight here, on the wider stage. Having completed her studies at Cambridge, Keane, the first Footlights president to resign, has joined NextUp, a company that helps women and non-white comedians film their shows. Meanwhile, Habib has returned from the FootDarks run at Edinburgh with a diary full of meetings with agents. “It’s exciting,” he texts me, with the news. “But I have very few people to turn to for advice.”

• If you would like a comment on this piece to be considered for inclusion on Weekend magazine’s letters page in print, please email weekend@theguardian.com, including your name and address (not for publication).

• This article was amended on 30 October 2019. In an earlier version, details of two separate meetings of the Footlights committee to discuss BME issues were erroneously conflated into one event. Some text has been changed to clarify the timeline and circumstances of these two separate meetings.