They say never go back, and in the case of Alien, why would you? The 1979 original, unusually for a cult movie, was a massive hit in its day, and its appeal has not only endured but grown stronger with time – rare for a science fiction film, since nothing dates so quickly as a vision of the future. Unlike with his 1982 masterpiece Blade Runner, Alien's perfectionist director Ridley Scott has pretty much left the movie alone, releasing only one mildly tinkered-with Director's Cut. Which, for him, shows quite an element of restraint.

Of course, the world of Alien was returned to, first with James Cameron's 1986 Aliens, not only a great sequel but one of the greatest action movies ever. After that, though, it was a case of diminishing returns, and not even such latter-day visionaries as David Fincher and Jean-Pierre Jeunet, who tackled parts three and four, could bring any life to the franchise. It didn't help that the first attempt to reboot it had Scott's "perfect organism" battling it out with the dreadlocked monsters from Arnold Schwarzenegger's 1987 B-movie Predator in 2004's daft Alien Vs Predator (and 2007's even dafter Alien Vs Predator: Requiem).

So when Scott announced his desire to return to the world of Alien – with the massive-budget, 3D Prometheus, which he said would not be a prequel as such, even though that's what everyone, even its cast and crew, considers it to be – there was as much surprise as excitement. Why go back there? Even though he co-wrote the script, screenwriter Damon Lindelof – a smart, self-deprecating New Jersey comic-book writer who was also one of the key creatives on Lost – can't quite put his finger on it.

"My sense of it was that he felt like Alien and Blade Runner were enormously satisfying to him – so satisfying that he didn't need to do sci-fi any more," says Lindelof carefully. "If he did it again, it was going to have to be different and special. But something happened over the course over the last 10 years, particularly with Alien. There was an itch he needed to scratch."

That itch didn't come from a loose end in the original script. Nor – given that the series' strongest aspect was its kickass heroine Ripley, played by Sigourney Weaver – did it come from a desire to reinvent that iconic character. Instead, it came from an image, glimpsed at the beginning of the film, when the crew of the Nostromo board a derelict cargo ship.

'It's very different to the original Alien, where essentially the characters wanted to just not get killed'

Charlize Theron. Photograph: Allstar

"Ridley always says that he was always very interested in the derelict spaceship and the space jockey that was piloting it," says Lindelof. "It was just something that sat in the back of his brain and never went away. So when the chance came up, he couldn't resist it. I wouldn't say he's a particularly spiritual, touchy-feely guy, but in talking to him I was surprised by the fact that he wanted to talk so much about what Prometheus would be about and what the characters wanted. Which is very different to the original Alien, where essentially the characters wanted to just not get killed."

Lindelof had just taken leave of Lost when Scott's people first called him. Within a few hours, the first draft of the ultra-secret script was delivered to his house, which Lindelof read while a courier waited outside. "It was a very good script by Jon Spaihts," he recalls. "Essentially, what Jon had done was a much more dyed-in-the-wool Alien sequel, in that it had all the things in it that we would expect to be in an Alien movie, from face-huggers to chest-busters to eggs. But there were also some original ideas in there, too. And what I pitched to Ridley was, 'I don't think this movie needs the face-huggers and chest-busters and all the things people are going to be expecting. I think these other ideas are original enough to power the whole thing.'"

These "other ideas", hinted at in the title, refer to a literal mythology within the Alien world. "What I can say," he reveals cautiously, "is essentially in line with what we've seen now in some of the promotional material and trailers." The idea is that 60 years from now, two brilliant scientists will unearth evidence of an ancient alien visitation, including signs on the Isle of Skye that seem to invite humanity out to a distant star system. "Somehow," says Lindelof, "they convince others that this is a worthwhile adventure, so they go out there to find answers to the most fundamental questions we've had since we've had cognitive thought. Like, who am I? Who made me? What is the purpose of my life? That is the jumping-off point. And in all great science fiction and even myth, characters who try to cross a line that should not be crossed often pay very harsh consequences."

It sounds pretty metaphysical for a film entering a marketplace currently dominated by Marvel's Avengers Assemble and Battleship. Was it a tough sell to the studio? "This is gonna sound like blatant ass-kissing but I was amazed and impressed by Fox," he says. "Did they challenge our ideas? Of course. But never from a sense of, 'We don't want you to do that.' It was more like, 'Help us understand this.' So never once did I ever receive any blowback or concern that the film was too heavy, or that it was delving into areas not suitable for a summer tentpole."

With Lost, Lindelhof learned the harsh lesson that when fan expectations soar high, there's a long way to fall. "I always want high expectations," he says, "but it is a double-edged sword. There are really only two possible outcomes for Prometheus. One is that it lives up to people's expectations, which are incredibly lofty. Or it falls below. There's really no area whatsoever for people to go and see it and think, 'Wow, that was so much better than I thought it would be!' This film is really making a promise to the audience, so they're not wrong to have those expectations. Certainly, if I had no involvement in this project, and I can remember a time when I didn't, I would have very high expectations for this thing too."

'People are still talking about Blade Runner, 30 years after it came out because some things are unclear. And that's cool'

Photograph: Allstar

As a huge fan of science fiction and the Alien movies, Lindelhof admits that it was sometimes tough to maintain his objectivity. "It's essentially fan fiction that I happen to be getting paid for," he says. "I would be doing this for free if I wasn't in this position. So growing up on this stuff, getting suddenly invited to play in that sandbox, is something I would never have imagined in my wildest fantasies. I never thought I'd ever sit in a room with Ridley Scott, let alone collaborate with him. So, speaking from the epicentre of that incredibly surreal scenario, it's like I'm constantly leaving my body and looking down, thinking, 'You are talking to Ridley Scott right now and he is listening to what you are saying.'" He laughs: "He thinks you're an idiot. But he is listening."

Yet after so many big, dumb success stories, will audiences actually want a highbrow blockbuster? "My hope is that it functions on two levels. First, I hope it is entertaining. I hope that people can enjoy it, whether that means providing scares, thrills or even humour in places. That's all great. But I also hope that, when the movie's over, there's still stuff to talk about. There is an attempt in Hollywood film-making to have a character come out and explain everything. I didn't want to do that, and I don't like doing that."

He laughs, perhaps thinking of Lost's cryptic/awful series finale. "Even though people think I ask too many questions and don't give enough answers! I have a lot more faith in the imagination and intelligence of the audience over my own – they come up with things that are infinitely more interesting. That's why people are still talking about Blade Runner, 30 years after it came out – because some things are unclear. And that's cool."

So is this a one-off? The end of the line? Apparently not. "One of the things we've always said, that makes Prometheus not a direct prequel of Alien, is that if there were a sequel to Prometheus – and, God willing, there will be if people like it – then it will not be Alien. This film ends in a way that goes off at a different tangent to the original Alien. If you see Alien straight after, you will see it in a context that didn't exist for you before, but it's not going to do anything to change the story of Alien. Prometheus has to go off in its own new bold direction, and hopefully it does so.

"Ridley," Lindelhof says finally, "certainly wants to do more." This time, though, the provenance will not come from the stars, or an itch, but from the box office. And it definitely won't take another 30 years.

Alien timeline

Michael Fasbender. Photograph: Allstar

It's 2093, and Michael Fassbender jets off into space in search of the origin of humanity …

Twenty-nine years later, an evil Alien bursts out of John Hurt's chest …

Come 2179, Sigourney Weaver is revived from hypersleep and gains a child sidekick …

… before crash-landing on a prison planet, with another psychopathic alien in tow.

For some reason in 2381 she's brought back to life …

… only to discover that the franchise has gone to pot, leaving our Alien queen to duke it out with a Predator.