We already live in interesting times. What is surprising – in a world beset by terrorism, climate change, war and economic instability – is the human commitment to making the times even more interesting (read: horrible) than they need to be. Open up your Facebook page and, unless you have chosen your friends with uncommon care, you will see a cavalcade of chemtrails, false flags, vaccination plots, Illuminati schemes, Mossad blueprints to create Isis, and other tinfoil-hattery. As David Aaronovitch showed in his book Voodoo Histories, conspiracy theories answer a deep human need: the desire for the nobody on the street to be at the centre of the story, for a comforting sense that someone, somewhere, is in control.

This is no longer simply a fringe concern. In 2013, the World Economic Forum cited “digital misinformation” as a major threat to modern societies, alongside terrorism and “the failure of global governance”. Last February, researchers who had studied 1.2 million Italian social media users published a report that found that fans of conspiracy theories were far more likely to share “alternative news” than dull, verifiable, factual news content. They were also largely incapable of distinguishing between sincere conspiracy posts and parodies and fakes designed to reel them in. There is little reason to suppose that Italy is unique in this. Conspiracy enthusiasts can’t help themselves. They want to believe.

This is the fertile ground that awaits the revival in January 2016 of The X-Files, the globally successful sci-fi-horror drama that originally ran from September 1993 to May 2002, made international stars of David Duchovny and Gillian Anderson as the FBI paranormal investigators Fox Mulder and Dana Scully, and brought conspiracy theory into ordinary homes around the world.

Conceived as a combination of police-political procedural and pulp sci-fi – All the President’s Men meets The Twilight Zone – The X-Files skilfully mixed an overarching story of a secret, government-abetted alien invasion with self-contained episodes of popcorn horror concerning shape-shifters, serial killers, psychic powers and sundry other tales to astonish. The show’s ability to switch from ongoing story to anthology, from “mytharc” to “monster of the week”, gave it a unique appeal both to committed “X-philes” and to the casual viewer. So, too, did its postmodern versatility with genre, which could take in everything from psychological drama to Tobe Hooper-style slasher horror, from farce to psychedelia.

Viewers were also hooked by the strangely compelling anti-romance between Mul­der – an FBI agent obsessed with unsolved cases in the bureau’s X-files and with finding the sister he believes has been abducted by aliens – and Scully, the forensic pathologist assigned by Mulder’s superiors to keep an eye on him. The show’s creator, the former Disney screenwriter Chris Carter, had reversed the TV convention of the time by writing a grounded, rationalist woman and a male dreamer as his protagonists. His lead actors portrayed them with deft, minimal blankness. In a remarkable feat of popular acting, Duchovny’s and Anderson’s studious lack of affect both intensified the weirdness of the agents’ milieu and supercharged the barely visible romantic tensions that hung between these two asexual sex symbols like superstring.

Mulder and Scully would grow closer as the series unfolded, losing and finding one another many times over, always displaying only the tiniest evidence of attraction. This was romance by and for geeks.

By its third season in 1995, The X-Files had acquired the mantle of an American zeitgeist show. An emerging internet culture began to dissect it, with enthusiasts writing wish-fulfilment versions of the Mulder-Scully relationship through the new phenomenon of online fan fiction. Television logic insisted that viewers demand closure and answers but The X-Files operated on the more modern intuition that we want the mystery, not the explanation – the walk in the dark, not the arrival home. The bug of an endlessly expanding conspiracy was a feature that kept viewers coming back. The show came to embody Oscar Wilde’s aphorism: “The suspense is terrible. I hope it will last.”

Meanwhile, a “Scully effect” was obser­ved in which the character’s popularity began to encourage young women to take up careers in medicine, science and the law. The show’s look and paranoid style – flashlights in foggy darkness, deserted hangars, inscrutable meetings in oak-panelled rooms, half-seen scuttling extraterrestrial forms – quickly came to define sci-fi in the Nineties as much as the space opera tropes of Star Wars had in the Seventies, or Ridley Scott’s decaying industrial visions from Alien and Blade Runner had done during the early Eighties. Resistance was futile.

The X-Files, we were assured, was emblematic of the new scepticism of the Clinton years, a suspicion of institutions and authority that had festered through the Nixon and Reagan administrations to emerge as a full-blown default mistrust in the world as it was presented to us. The use of the Federal Bureau of Investigation – the show’s narrative framework – marked a definitive break with the past and with faith in government. The state’s most powerful agencies were no longer our protectors; they were selling us out. In the Fifties and Sixties, if aliens turned up, you would have called the FBI. By the Nineties, the FBI was bringing the aliens here.

If a show such as this could strike a chord in such a comparatively placid period as the Clinton presidency, you have to wonder how it will be received in the more tempestuous world of Julian Assange, Chelsea Manning, Anonymous and the proudly worn, bitterly combative reflex scepticism that the internet encourages.

“It’s a perfect time to come back with The X-Files, considering global politics,” Chris Carter told an audience in Cannes in October after a screening of the first episode of the new, six-part series. “We’re trying to be honest with the changes in dealing with digital technology and the capability of spying. Clearly we’re being spied on in the US and there seems to be no shame in it. Every day I look at the newspaper and I see a possible X-Files episode.”

I think that Carter is only half right in this. The phenomena he is talking about – WikiLeaks and global surveillance, political conspiracy and manipulation – are indeed prime X-Files material. But they belong to the real world of human agency. For all their enormity, they are in their own way mundane. And such material was only ever a part of The X-Files’ vast appeal, which rested equally (and synergistically) on the fantastical, the irrational and the transcendent.

Maybe this is why many fans secretly admit that they prefer the self-contained monster-of-the-week episodes to the sprawling bafflement of the programme’s ever-extending arc. General viewers certainly did, for the show’s true soul is located in the weird and the pulpy. Mulder and Scully were as likely to encounter a demon in small-town New Hampshire as government black-ops fixers.

The irresistible quality of The X-Files is its belief in a world beyond, making Mulders who devoutly want to believe out of us all. (For a glimpse of what the show would have been without that dimension, see the short run of episodes in season five in which Mulder becomes convinced that the conspiracy is a hoax after all, designed to distract from the machinations of the military-industrial complex. Somehow they’re just not X-Files enough.) Far from an exercise in scepticism, The X-Files was arguably the most idealistic, optimistic show of its time.

“I don’t really buy the idea that The X-Files somehow dramatised the cynicism of the Nineties,” says Robert Shearman, the author of Wanting to Believe: a Critical Guide to “The X-Files” and a playwright who scripted the acclaimed Doctor Who episode in which Christopher Eccleston went eyeball to eyestalk with the last Dalek in creation.

“To me, the show is actually an act of optimism because it suggests that there is something to uncover,” he explains. “It’s about belief and faith in something that ­exists on a higher plane. That’s a fantastically exciting and empowering idea.” Since the 9/11 attacks, Shearman says, darker drama has often asserted that everything is fraudulent, that leaders would go to war not for an idealistic belief but for a simple desire for money and power.

The show was a product of its time but not as is commonly thought. “The very idea that the corrupt and the murky could be centred on something as crazy as a government cover-up of aliens in Area 51 now seems almost quaint,” Shearman says. “The X-Files is actually a naively optimistic show from a time when America hadn’t been so deeply threatened and could turn its attention inward. Any show that opens every episode by asserting ‘The truth is out there’ is fundamentally pretty optimistic and open-minded.”

***

Popular fiction would be a quiet world indeed without random meetings on public transport. Some time in the mid-Nineties, when The X-Files was already a fully fledged pop-culture phenomenon, the acclaimed science-fiction writer William Gibson was flying home from Los Angeles to Vancouver when he found himself seated near Chris Carter. The first few series of The X-Files were filmed in Vancouver and it seemed to Gibson that half of the show’s production team was on every Air Canada flight. The two got talking, with the eventual result that Gibson – the father of cyberpunk and cyberspace – wrote two episodes of The X-Files.

“My daughter started watching the show as a young teenager and I recognised a lot of material that had been part of my own imaginative life,” Gibson tells me on the phone from his home in Vancouver. “I knew what they were riffing on and I started watching it pretty keenly. Also, it was fun seeing our locale used to depict every place on Earth,” he adds, laughing. “We would see them shooting the thing as we travelled around. There’s a scene supposedly set in Nebraska where you can see majestic mountains rising in the background.”

What fascinated Gibson was the show’s roots in folk myth and pulp fiction, in the American need to discover – or create – a deeper, darker story than this comparatively young country’s officially recognised history allowed. The alien conspiracy that runs through the show becomes linked to Native American culture and linguistics. Mysterious creatures or “cryptids” such as the carnivorous Jersey Devil, the human-sized insect Mothman of West Virginia and the goat-sucker vampire chupacabra of Mexican myth all featured in a show that ­often acted as a tour of the American subconscious. (In one of its self-referential comic moments, we see what Mulder gets up to during his downtime: he watches videos of Bigfoot as if they were porn.)

“The X-Files form of American folk paranoia wasn’t just part of the zeitgeist,” Gibson says. “It was a lot older and a lot deeper. It’s very much part of American culture. I’d been reading Fortean Times [the journal of unexplained phenomena, named after one of Mulder’s spiritual antecedents, the paranormal researcher Charles Fort, who was born in 1874] almost since it began and The X-Files was a gloriously pulped-out version of that.”

Gibson originally pitched Carter stories in that vein, ideas of American strangeness that were “core to the show”, but soon realised that what he wanted from him was the cyberspace stuff. He went on to write Kill Switch, featuring computer hackers and a malevolent artificial intelligence, and later First Person Shooter, in which Mulder and Scully enter a virtual-reality environment. “The first idea I had for The X-Files was a haunted website, a page that literally kills you,” he recalls. “The plan was to do a seriously scary story, not played for laughs at all. The reason they gave for not doing it was that they thought not enough people knew what a website was back then. They thought it’d be confusing.”

The show was more comfortable with that other great engine of the American imagination, the UFO scares of the Fifties. In movies, pop culture and rumour, a pandemic fear of the looming communist other was transferred into the fantasy realm of hubcap-shaped flying saucers, impending invasion and body snatchers that were indistinguishable from your all-American neighbours.

“It was a very knowing show that could both comment on the madness of this stuff and believe it wholeheartedly,” Gibson says. “There were elements that were just out there, Mothman folk lunacy. But when they weren’t being outright ironic about it, you could tell that they absolutely knew what they were doing.”

As with everything in the world of The X-Files, the notion of extraterrestrials as a proxy for the threatening other turns out to have deeper roots. Sharon R Yang, a professor of English at Worcester State University in Massachusetts and the editor of the essay collection “The X-Files” and Literature, points out that all-powerful external agencies are a recurring motif across literature – especially in her area of specialism, the Gothic.

“The Gothic tradition concerns things lurking in the beyond and monsters that often represent ourselves,” she says. “It’s traditionally looked at how big institutions like the church and state are in fact fundamentally corrupt. That idea of the evil within is a very X-Files thing. Both Gothic literature and The X-Files are about taking that walk into the dark woods and facing what we can’t define. That’s essentially what Mulder does in the show.”

Yang’s literary exegesis of The X-Files unpacks such a wealth of themes and resonances that you feel you ought to watch all 202 episodes again. There is Christian imagery in the motif of faith and Scully’s Catholicism and in the Erich von Däniken-like inclusion of biblical texts on alien artefacts (was God an astronaut?). There is the figure of the psychic detective, very popular at the turn of the 19th and 20th centuries, who treated the supernatural as a branch of science, yet would step into the worlds of spiritualism or theosophy. Bram Stoker’s Dracula (1897) was the X-Files of its day: “Van Helsing is very Mulder-esque,” Yang says. “They’re using the new technology of the time like photography, typewriters and recording machines to fight the supernatural, much as Mulder and Scully use this exciting, newfangled internet and these cool new cellphones in The X-Files.”

There are echoes of Shakespeare, too. What is Hamlet but a young, rebellious smart alec searching for the truth behind corrupted institutions? And who is Scully but Mulder’s logical, scientific Horatio? Their joint quest has consequences. Innocent people pay the price for their idealism (it’s no fun being a member of the Scully family). And, like Hamlet, the two agents are never quite the match for their unscrupulous, Machiavellian opponents. “When you face these monolithic corrupt institutions, the most you can hope for is one or two allies,” Yang says. “There isn’t a big group that’s going to protect you. But maybe there is your Scully.”

Embracing all of these themes, she argues, is the notion of The X-Files as an Arthurian quest, a moral and spiritual test for Mulder, who is compelled to go in search of his personal grails – his missing sister and the truth about the conspiracy – and is pushed to his limits by the experience. “A true quest is about discovering and accepting your human limitations but doing your utmost anyway,” Yang says. “The glory is in going on the quest at all.”

***

If you’re going to watch only one episode of The X-Files, then it should probably be Bad Blood from series five in February 1998. Written by Vince Gilligan, the future creator of Breaking Bad, this episode contains everything that is great about the show (horror, humour, an instantly comprehensible insight into the Mulder-Scully dynamic) and none of its shortcomings (it’s not waffly or pretentious and you don’t need a ring binder of notes to follow it).

Ostensibly concerning an outbreak of vampirism in Texas, Bad Blood is actually about Mulder’s and Scully’s different perspectives on one another and themselves. The episode retells the story of a late-night vampire hunt twice: once from Scully’s point of view and once from Mulder’s, like a 45-minute version of Akira Kurosawa’s Rashomon. In these alternating, only partially reliable accounts, Scully sees Mulder as an infuriatingly impulsive, rabbit-like enthusiast who isn’t remotely bothered when he accidentally stakes an innocent boy through the heart, whereas she is reason and duty personified. In Mulder’s eyes, he is almost pitifully needy of Scully’s approval, while she is a whining, hectoring martinet. Running through both versions are hints of the mutual attraction that each is too scared to acknowledge. For all the tension, twists and terror, it is a thoroughly charming character piece about one of television’s great couples.

“I don’t imagine that Chris Carter ever sold the show on that romance,” Shearman says, “but there is such wonderful chemistry between those two actors. The characters essentially redeem one another. The purpose of Mulder is to open Scully’s eyes to other possibilities and Scully is there to humanise Mulder. They bring out the best in each other, which is a terribly romantic thing to happen, regardless of whether there is a sexual relationship going on. It’s almost like the Doctor and his companion in Doctor Who. It’s an unspoken love.”

The Mulder-Scully dynamic also functions as a sort of pop-culture Socratic dialogue, an ongoing debate between faith and doubt. “I see them as a yin and a yang,” says Sharon Yang. “They contain elements of one another. So Scully is very rational but she has enough imagination that she can perform an alien autopsy with an open mind. Mulder represents faith and belief but the famous poster on his office wall says ‘I want to believe’ – not ‘I believe’. He still has some reserves of scepticism. There’s a Watson-Holmes thing going on. Watson roots Holmes in the prosaic and translates the mundane world for him, much as Scully does for Mulder. They’re complementary.”

Towards the end of the original run of The X-Files, however, faith conquered doubt in a not entirely satisfactory manner. Scully was finally won over to Mulder’s point of view but the increasingly convoluted mytharc began to strangle the show. The feature of endlessly deferred closure turned out to be a fatal bug after all, burying The X-Files much as it had Twin Peaks. Duchovny left after the seventh season following a contract dispute, returning only intermittently. And finally the events of 11 September 2001 – when the hideous consequences of a real-life conspiracy made the subject feel toxic – left the final, largely Mulder-free season seeming dated and lacking in relevance.

“Later on in the run, when they were trying to find new characters to replace Mulder and Scully, they could never find the same dramatic purpose for them,” Shearman says. “The new FBI agents Doggett and Monica Reyes are great but you don’t see what they’re doing dramatically for each other. The show couldn’t pull away because the spine of The X-Files was the interaction between Mulder and Scully.”

After Duchovny returned for a two-part finale (somewhat misleadingly entitled The Truth), the show ended on 19 May 2002, with only a very odd feature film entitled The X-Files: I Want to Believe (2008) since. Yet in the intervening years the dangling threads, unfinished business and still-dedicated fan base of The X-Files have made it ripe for revisiting. The landscape has changed but that just means different stories. “Since the show has been away, cameraphones have pretty much killed off flying saucers and Bigfoot and every other cryptid,” William Gibson says. “But somehow,” he notes ironically, “those same phones have caught police all over the world violently attacking people, which apparently they had never done before. I mean, who would believe that? The reality around the show has changed in ways we least expected. But it’s still a weird reality.”

Will The X-Files be as successful in this new world? Initial reports about the first episode (scheduled to be broadcast on Fox in the US on 24 January – the date of the UK broadcast on Channel 5 is yet to be confirmed) suggest that Carter has modernised it only slightly. The miniseries will mix such staples as alien abduction and a “were-monster” with contemporary elements such as Federal Emergency Management Agency camps and a new character, the right-wing internet news anchor Tad O’Malley (played by Joel McHale).

Watching the trailers, one is struck by how much Duchovny’s and Anderson’s age changes the scenario. In the original series, they were in their thirties and twenties respectively, upstart Young Turks of the FBI. Now Anderson is 47 and Duchovny 55 and they project a new gravitas. These are no longer wayward kids but experienced old hands coming out of retirement for one last job. They still look attractive, although the years have not been as kind to one of them. “Mulder’s not in a great place,” Duchovny told Entertainment Weekly during filming. “He’s wearing bad jeans, so you can just extrapolate from my wardrobe. He’s in a dark, dark place.” Uneasy lies the head that wears the tinfoil hat.

The X-Files, Robert Shearman argues, was a masterclass in ambiguous storytelling. “Some people hated that aspect of it but I thought it was fantastically innovative. When most American TV drama was about making very clear points and resolutions, The X-Files completely reinvented the way you tell stories.” Its heirs, he thinks, were not in science fiction but in novelistic TV epics such as The Sopranos, The Wire and Breaking Bad. Here is the reason we could see a talking fish tell Tony Soprano the name of the traitor, or have Walter White’s world framed by songs and hallucinations.

The X-Files is also the reason that every ­alternative news medium – from right-wing sources such as Infowars and the Drudge Report to the left’s Indymedia and Raw Story – now styles itself as an outsider voice, a Mulder and Scully of the digital realm. The show persists because it satirised the conspiracy mindset as much as it depicted it. “People find conspiracy theories fantastically comforting not because they’re more frightening than reality,” William Gibson says, “but because they’re less frightening than reality.” In a relativist age in which Putin pays sock puppets to fill newspaper message boards with pro-Russia comment, in which nothing is true and everything is permitted, we would love nothing more dearly than for the truth to be out there. Somewhere.

Andrew Harrison is a contributing editor of Esquire magazine