The end of Big Brother was supposed to signal the death of reality TV, but more of us than ever are tuning into the X Factor and its like. Now, an extraordinary piece of research has revealed why our need for community has helped Simon Cowell become the king of Saturday-night entertainment

It is a Saturday night in the northernmost fringes of London, a place where the streets are lined with Golden Nugget Chicken outlets. Outside an anonymous building with blanked-out windows, a discarded plastic bag swirls in the breeze, and in the distance the arches of Wembley Stadium soar against the darkening sky.

At first glance it seems a doleful place. But in fact, this is where dreams are made and broken. Inside that reinforced-concrete building, men in black T-shirts and microphone headsets are swarming around like worker bees, ushering a constant stream of shrieking teenage girls to their seats, testing the sound levels and the autocue, ensuring that the audience is primed to clap and scream as loudly as possible once the lights go up. Because this is where, every weekend, the X Factor goes live.

As the theme music is pumped through the studio speakers, it is as though the entire crowd has been electrified by a giant cattle prod. We leap out of our seats as one, arms waving maniacally in the air as each contestant takes to the stage in a blaze of strobe lighting and sequinned backing dancers. When the judges deliver their verdicts, we boo as soon as Simon Cowell says anything remotely negative and cheer wildly when Cheryl Cole gives a twinkling, encouraging smile.

We are indiscriminately supportive of all the contestants. We empathise with them in a way we never normally engage with actors or celebrities precisely because they are real and because – at the touch of an interactive red button or the dialling of a phone number – we can have a say in their future. When Mary Byrne comes on stage, swathed in a black evening gown, we cheer loudly because we do not want her to go back to her checkout till at Tesco. When Rebecca Ferguson takes the floor, all sparkly eye shadow and pretty smile, we clap until our hands sting. When One Direction performs an upbeat love song in matching suit jackets, those of us who are not teenagers regress shamelessly to our adolescence. If the security guards had allowed us our cigarette lighters, we would be brandishing them now. At the back of the auditorium, several schoolgirls become breathless with excitement. "We love you Harry," they shout out in unison.

This is the X Factor, brainchild of Simon Cowell and the most popular programme on Saturday night. Each week, hundreds make the pilgrimage to the Fountain Television Studios in Wembley to be part of the live audience, and millions of us tune in at home to watch. Each week, the front pages of the tabloid newspapers will be emblazoned with headlines about Cher Lloyd's supposedly diva-ish antics or Louis Walsh's backstage meltdown.

As a result, many of us will spend more time in the virtual company of Mary, Rebecca and One Direction than we do with our real-life friends and family. In a modern world in which local communities have become increasingly fractured, where relatives live further apart from each other than ever before and where one in five of us will never speak to our neighbours, Cowell's creation seems to be filling the void.

And yet despite the fact that more of us seem to be tuning in than ever before, relatively little is known about who watches and why. All we know is that the X Factor – whether it signifies the reinvigoration of weekend family viewing or the disintegration of civilised society – is a reality-television phenomenon.



WHEN BIG BROTHER ended in August this year after 11 series, there were confident proclamations that this would mark the terminal decline of reality TV. But our enthusiasm for Simon Cowell's prime-time talent search shows no sign of abating. In fact, the audience has become bigger: the opening episode of this year's X Factor attracted a record 11.08 million viewers – an astonishing 48% of the total viewing audience – while the finale of last year's Britain's Got Talent reeled in more than 14 million. On the BBC, the latest series of The Apprentice and Strictly Come Dancing are routinely getting viewing figures of 8 million each. (Adding all these viewers together comes to 41.08 million, compared with the 29.6 million who voted in this year's general election.)

The audience has not only expanded in quantity but in social base, too. Whereas previously, it was always assumed that the X Factor, and shows like it, primarily appealed to teenage girls voting for a new celebrity heartthrob, a major new piece of research seen exclusively by the Observer shows that the programme is increasingly crossing social, class and gender divides. According to the survey of 3,514 people conducted by the research and strategy agency Brand Driver, it is actually male viewers who are more likely to vote – with 20% of men aged 18-34 voting multiple times for the same contestant.

This year almost everyone seems to be watching: from politicians (Michael Gove is said to be rooting for Matt Cardle; Ed Miliband for One Direction) to actors (Benedict Cumberbatch, the Sherlock Holmes actor, is a Cher Lloyd fan) to highbrow cultural commentators – a spoof YouTube video of Stephen Fry backing the bearded Brazilian Wagner has already attracted more than 500,000 hits.

Simon Cowell, the high priest of reality TV, seems to be manipulating the nation's psyche more successfully than ever before, despite the perennial scattering of media commentators who decry the genre as exploitative and socially destructive. So why are we so engrossed? After a decade of phone-in rows, vote-rigging accusations and celebrity-hungry wannabes with egos more bloated than their silicon implants, why does the British public remain so in love with reality television?

"It's always going to be popular," says Peter Bazalgette, the former creative director of production company Endemol and the man who brought Big Brother to UK screens. "The average show takes members of the public and sends them on a journey. We love to follow that because it's a cracking story which engages our emotion. It's not unlike a soap opera, except that these are real people and you get to vote them out one by one. That simple premise dominates everything from Britain's Got Talent to Big Brother and The Apprentice. What it creates is an extraordinarily powerful story arc where we get involved in the characters. That's why we watch it."

By now most of us know that the version of reality on offer is one shaped by a multimillion-pound business with slick production values, and yet we willingly suspend our disbelief week after week, month after month, in the name of entertainment. Is there something lacking in our daily lives that draws us so inexorably into Cowell's web?

Beverley Skeggs, a professor of sociology at Goldsmith's, University of London who has conducted studies into the representation of class and gender on reality television, believes it is, at root, a question of human connection. "I think the community feeling is part of the appeal of this kind of show," she says. "It's expressing a form of social value – people connect to other people who have the same values, who behave in the same way. We do get swept up in it, wanting to be behind somebody, wanting them to do well. That's why producers will milk the hard-luck story – those little snippets of someone working in Tesco – because all that intertextual stuff enables us to feel we have this sort of connection."

And perhaps, in a world increasingly dominated by Facebook and Twitter where friendships are made and broken at the click of the computer mouse, we feel more comfortable engaging with someone on the other side of the screen rather than chatting to them over the garden fence, as our grandparents might once have done. If we are already sharing the details of our private lives in Tweets and status updates, are we also becoming more accustomed to the notion of putting our intimate selves on display for the entertainment of others?

According to the Brand Driver research, which is one of the first substantial examinations into the allure of reality television, the key to the X Factor's appeal is precisely this ability to tap into the notion of personal relationships, into the idea that we want to form an empathetic bond with contestants and feel part of an extended social network. The survey found that almost a third of viewers stated that they watched the X Factor and felt "part of a community". One in 10 of those who had voted for a contestant claimed that they would miss a friend's wedding to watch the programme, and in many ways our connection with the X Factor appears to parallel the course of a romantic affair, with all the heightened emotion and irrational feeling that comes with it – 45% of viewers cry at contestants' stories, while 39% are left feeling that "anything is possible" after watching. When contestants are voted off unfairly, 73% of viewers get "annoyed", while 30 respondents actually claimed to be depressed when the show ended; almost half of viewers (49%) went on to have a rebound relationship – with Britain's Got Talent.

"The X Factor has a telling and unique ability to provoke extreme emotions in its viewers, from shouting at the judges to crying at the contestants' stories and journey," explains Tim Julian, the chief executive of Brand Driver. "This begs the question as to whether we are becoming a nation more able to relate to characters on the screen than in real life.

"I don't think it's any coincidence that our love affair with the X Factor is so potent right now, more than ever before, as Britain endures a period of relative austerity," Julian continues. "Although the X Factor is a thoroughly modern phenomenon, the show occupies the same social and emotional territory in consumers' lives as the silver screen did for cinemagoers during the depression of the 1920s."

And it could be that in a time of economic hardship, we are seeking out the simple and cheap– family entertainment that makes us feel part of something bigger. But the Australian academic Dr Bridget Griffen-Foley traces back the popularity of such shows even further – to the emergence of 19th-century periodicals which relied on reader contributions and "invited [them] to feel a sense of connection with the otherwise impersonal structure of the mass-market press".

The historian Dominic Sandbrook, author of State of Emergency: The Way We Were, Britain 1970-1974, sees reality television as a descendant of "1950s quiz shows and holiday-camp contests for things like Most Beautiful Baby. Almost the whole camp would turn out and lap it all up. Beyond that, you go back to the Victorian freak shows. I think reality TV is merely a manifestation of a very, very old craving that we have for imposing narratives [on events]. We love sentimental stories, such as Dickens and Little Nell; we love a tear jerker, and I don't think shows like the X Factor are any more crass or exploitative than those 19th-century penny dreadfuls."

Yet it seems that 21st-century viewers are looking for more than just simple entertainment. Part of the attraction is the sense of control the X Factor gives us: the sense that we can put right wider social wrongs by voting for our favourite contestants and that although our lives are being shaped by forces beyond our control – such as government cutbacks, widespread job losses or social deprivation – the ability to have a say in what happens to others in reality TV shows gives us back a much-needed sense of agency.

The most popular contestants almost always have a backstory of personal triumph over adversity which enables us to feel that we are helping them succeed, that we are giving them a break even if no one else will. "It's a fantasy of another life in a country where social mobility is nonexistent," says Professor Skeggs. "The idea that people like Susan Boyle can overcome hardship because we're voting for her makes it look like meritocracy is really possible."

According to Skeggs's research, the viewer reaction to someone such as Susan Boyle – whose note-perfect rendition of "I Dreamed a Dream" on last year's Britain's Got Talent racked up 500m hits on YouTube and prompted Time magazine to name her the world's seventh most influential person – is markedly split between middle- and working-class viewers. "The middle-class viewers thought it was getting something for nothing," she explains. "They thought that hard work and discipline were connected with doing well in education and should be rewarded with traditional rewards, not reality-television success. The working-class viewers saw it as a structure of opportunity when all other opportunities have closed down. It's another way of making something of your life."

And perhaps this is why Boyle, who grew up in a council house and was bullied as a child for her learning difficulties, or Alexandra Burke who came from an impoverished background, have proved such enduring figures.

"We like the idea that we can help someone like that make something out of nothing," says Gladeana McMahon, a psychotherapist who co-wrote the UK's Ethical Guidelines for Reality Television. "It makes us feel: if they can do it, so can I. It's uplifting. Part of us wants them to be successful and is urging them on."

It certainly seems that we are just as motivated to vote for positive reasons as negative ones – the Brand Driver study found that, among lesbian, gay and bisexual viewers of the X Factor, nearly 80% think that the programme has helped people come out as homosexual and the same number believe it has promoted the tolerance and acceptance of gay people generally.

Of course there are less noble motivations for watching, too: for every Susan Boyle or Leona Lewis, there is a caterwauling teenager who cannot hold a tune and yet remains convinced they are destined for stardom. "A part of us just loves it when people are awful and embarrass themselves – but human nature is contradictory like that, and reality television allows us to have it both ways," says McMahon. "We love reality TV because it gives us power, and it's absolutely true that this kind of programme – where we are so involved in what happens – gives our lives a dramatic narrative they might otherwise lack."

It also allows us the chance to participate in human relationships, both in terms of a shared audience experience (half of viewers say they watch the X Factor as an opportunity to spend time together as a family) and because we can influence what happens to a particular contestant we like.

A 1999 academic study by the University of Oslo claimed that reality television arose out of "the powerful urge for a sense of contact with the real" in an age defined by the chimeric interactions of the internet. The irony, of course, is that reality television is in itself a construct: a facsimile of real life designed by television executives to ensure that viewers keep coming back for more.



THE IDEA THAT Simon Cowell and his ilk are manipulating an unwitting public in order to line their pockets has been grist to the mill of those who sniffily view reality television as the end of civilisation. The Radio 4 presenter John Humphrys has been one of the most vocal critics, condemning the genre's "coarsening effect" and "sheer vulgarity". In his 2004 MacTaggart lecture at the Edinburgh Television Festival, Humphrys claimed that programmes such as Big Brother "turn human beings into freaks for us to gawp at. This is not just bad television in the sense that it's mediocre, pointless, puerile, even. It's bad because it's damaging."

But the underlying implication of Humphrys's diatribe – that the majority of the British public who enjoy watching such programmes simply cannot be expected to know any better – is comprehensively challenged by the recent research. The Brand Driver study found that a third of all X Factor viewers have a degree and a further 40% are postgraduates. The majority are affluent professionals – the household income of a regular viewer is around £3,000 a year more than a non-viewer, while 40% describe themselves as professionals or in management. And in what might come as a particular shock to Humphrys, a substantial proportion of X Factor watchers also regularly tune into Question Time (31%) and Newsnight (14%).

"All popular entertainment is dismissed by snobs," says Bazalgette. "It's the same attitude as they have towards food or sex – if you enjoy it, it must be bad for you. It's a kink in the Anglo-Saxon character, a Calvinist throwback."

In fact, most of us know we are being manipulated and that our emotional buttons are being shamelessly pressed every time there is a lingering close-up of a tear-stained contestant's face recounting the traumatic time their grandmother's budgerigar died. But because we have become so accustomed to such televisual shorthand, we are increasingly willing participants in the charade. We become, along with the contestants, part of the performance.

"There's an assumption among the middle classes that: 'I can see through this stuff be-cause I'm intelligent and educated, but the proles can't'," says Dominic Sandbrook. "It's incredibly patronising. A lot of people watch it slightly tongue-in-cheek. They can see the joins, but it doesn't matter."

Do we care that reality television is not actually real? Not according to Bazalgette: "That question misses the point. Reality television is a completely constructed premise. None of the people would be in it if we were just showing their normal lives. But what it does do is take human flesh and blood and challenges it in situations that bring out a person's true personality. So what flows from this constructed premise is extremely real. That's why shows work, because the public is after authenticity… They want to support people with talent and for them to win, but they punish pretension and two-facedness. On the whole, the public is positive, but they are judgemental."

It was TS Eliot who once said: "Humankind cannot bear very much reality." Perhaps this, in the end, is the key to Cowell's success: he acknowledges that we crave the appearance of reality, but that we also want the reassurance of a happy ending for those who deserve it and retribution for those who do not.

Either that or we just want to laugh at the man with the comb-over singing an out-of-tune Mariah Carey song.

Elizabeth Day's debut novel Scissors, Paper, Stone, is published by Bloomsbury in January