Last year, most of the excitement at the Clarke award had happened long before the ceremony. The great Christopher Priest (himself a past winner) had declared the shortlist "dreadful", called for the resignation of the judges and described Charlie Stross, one of the shortlisted authors, as "writing like an internet puppy: energetically, sometimes amusingly, sometimes affectingly, but always irritatingly". It was really most interesting.

This year there were no such broadsides. There was some muttering about the fact that there was an all-male shortlist for the second time in the award's history – but since women have won in the two previous years, and because four of the five judges were female, the complaints didn't grow too loud. And that was about it. Such lack of dissent was disappointing from a journalistic point of view, of course, and for fan's of Priest's merciless wit. There was, however, some advantage to the focus remaining on the event, rather than the gossip surrounding it – not least because it was that rare thing, an awards ceremony that contained plenty of interest aside from the few seconds when the magic envelope was opened.

The event was held in the Royal Society, which organiser Tom Hunter described as "a geek's dream", and began with a panel discussion about the breakthroughs that are going to influence the sci-fi writing of the not-too-distant future. This took in mathemetical models of the human heart that can be used to trial medicines, time travel, faster-than-light drives, synthetic biology and, yes, a giant natural computer for a space station that is being designed for construction 100 years from now. (Project Persephone – "It's real, not fiction," said panellist Rachel Armstrong).

And I suppose all that does sound like a geek's dream – but surely, also, of interest to anyone and everyone. One of the panellists, Adrian Hon, helped create Zombies Run, which now has more than half a million users jogging faster because they can hear zombies at their back. That's a lot of people. And these ideas are not underground. We're living in a world where everyday technologies, such as smartphones, are so advanced that they might as well be magic, for all we know about how they work. Meanwhile, millions of us watch Game of Thrones, a series based on a book that only a few years ago was regarded as niche fantasy. More and more writers such as Lauren Beukes appear in the main literary pages of most newspapers. Five years ago, when I started writing about the Clarke awards, I approached it as if I'd been invited into a strange new world. Now that just seems absurd.

It's telling that Kevin Duffy from Bluemoose books says that when he took on one of the books on last night's shortlist, Nod by Adrian Barnes, he didn't stop to consider what genre it belonged to. "When I took it on, I wasn't really thinking of it as science fiction. The only category I was thinking of was 'good fucking books'."

I'd like to think that most people probably react in a similar way when they pick up a sci-fi title in a bookshop nowadays; that they think about the idea, the story, the quality of writing, and don't wonder if it belongs to a genre they regard as somehow substandard. I'd also like to think most people are happy to accept that sci fi isn't a poor second cousin to other types of literature.

So I liked to think, anyway. But I was brought down to earth when I spoke to Chris Beckett, this year's Clarke award winner. As you might expect, he was delighted with the way things were going.

"I used to be a solitary child," he said when he picked up the award. "On sunny days, when everyone else was playing outside, I would be found in my room, flat on my back staring at the ceiling, engaging in daydreams about imaginary worlds. What might then have seemed dysfunctional or unhealthy now starts to seem like a rational career move. Sure other kids had fresh air and companionship, but have they got a Clarke award?"

More seriously, he also later told me: "I've been waiting a long time, so it's a lovely feeling. It's wonderful when all that work actually pays off – because when you commit yourself to writing fiction it's a huge gamble, especially when you get to my age."

Beckett is 57. Wikipedia currently claims he only started writing science fiction in 2005 – although he told me his first short story was published in 1990. Either way, you can understand why a Clarke award might feel vindicating. But when I aired my theory about sci fi's escape from the genre ghetto, there was a brief flash of anger.

"It irritates the hell out of me when people won't touch my book because they see it's science fiction," he said. "Of course some sci fi is crap. But it can also be so many other things. It's such a wonderful mechanism for thought experiments …"

Dark Eden, the winning book, is a case in point. Beckett describes the premise as the "hoary old sci-fi cliche of Adam and Eve in space", but his story of the incestuous offspring of two astronauts stranded on a sunless planet, written in an "addictively odd vernacular" is evocative, strange and, as Stuart Kelly's Guardian review put it, "properly alien and properly probable". As with most recent Clarke award winners, the question isn't why anyone would want to read science fiction, or even how to get people to understand it has much to offer; the question is why anyone would want to deny themselves such an interesting experience.