Though there’s plenty of fun to be had from this latest instalment of Terry Pratchett and Stephen Baxter’s Long Earth series, I read it in a rather mournful frame of mind. Our supply of original Pratchett is running dangerously low. Since he continued working almost to the end, there are several posthumous titles in the offing: one more Discworld novel (The Shepherd’s Crown, due at the end of August) and two Long Earth books – this one and the series finale. After that, having been so busy a feature of the literary landscape for so many decades, and having inspired a devotion in his readers unparalleled in contemporary writing, Pratchett’s voice will finally pass into silence.

Something of that melancholy seems to have worked itself into the fabric of this novel, too. Earlier Long Earth books possess various degrees of whimsical warmth and inventive charm. A more autumnal breeze blows through The Long Utopia. It’s a book much concerned with things coming to an end, with cosmic-scale disease and with the limits of knowledge. A premise that started as an infinite number of open doors is starting to close them around its characters.

That premise: our planet is one of an apparently infinite chain of Earths, all superposed in quantum-displacement upon one another. Travel from our Earth into these alt-Earths is easily effected with a simple piece of technology (powered by a potato, in fact). Some individuals don’t even need that, possessing the natural ability to “step” in one of the two directions available, up-chain or down, arbitrarily designated “west” and “east”. The only constraints are that most people suffer debilitating nausea with repeated steps, and that it’s impossible to carry iron from world to world. This means that most migrants up and down the Long Earth end up making old-fashioned 18th- or 19th-century-style settlements, having to mine their own iron, make their own nails and so on. The other oddity about the Long Earth, also as yet unexplained, is the complete absence of humans anywhere up or down the infinite chain of worlds except (of course) on ours, so-called “Datum Earth”.

If you go to these books looking for the rich comedy of Discworld, you will be disappointed. It’s worth remembering that hilarity isn’t Pratchett’s only mode. He started out as a science fiction writer (and fan), and jotted down the conceit for the Long Earth before he wrote the first Discworld novel. Indeed, one of the things that made his fantasy writing so distinctive was the scientific rigour with which he pursued even the most absurd of his premises. Baxter, similarly prolific, is Britain’s leading writer of “hard” SF, a seemingly inexhaustible fount of thought-provoking, imagination-tickling and sometimes mind-blowing ideas. Their collaboration is more a hymn to the joys of unfettered world-building than it is to story or character. But if the pace of plotting is gentle, the restless inventiveness more than compensates.

It’s not that nothing happens in the Long Earth series. The narrative is enlivened by terrorist plots, a new generation of superintelligent “homo superior” children, attempts to build space elevators, trips to Mars and a cataclysmic disaster on Datum Earth, plus we also get the more soap‑like digressions on family secrets and dramas of the main characters. But nothing really disturbs the broad, tranquil flow of the novels, plainly written, expansive, taking the reader slowly through a cornucopia of alternate possible Earths. Some volumes work better than others. The first was buoyed along on its basic voyage of discovery; The Long War (not, despite its title, an especially martial text) lost some momentum; The Long Mars added some interesting speculations and alternate versions of the red planet. The Long Utopia is a better novel than these previous two, largely because it uses a compelling central mystery to thread together its various interesting extrapolations, cool ideas and YA-level storytelling. On the 1,217,756th Earth west of ours, some settlers stumble on something that’s never before been encountered in this vast chain of planets: alien life. Insectile cyborg creatures are burrowing under the world, having, it seems, “stepped in” from an impossible direction. What they’re up to, and what humanity can do about it, gives the novel more narrative backbone than earlier instalments.

The story builds steadily to a vividly rendered apocalyptic scenario, and this proves to be the last showdown for several characters. The titular reference to “utopia” looks almost ironic in the face of this. The post-scarcity spaciousness of the series has an obvious utopian vibe, but this penultimate volume is haunted by very non-utopian concepts: the costs of things; the past’s inescapability; the encroaching end. We’re not there yet, but it won’t be long now.

• Adam Roberts’s Bête is published by Gollancz. To order The Long Utopia for £13.99 (RRP £18.99) go to bookshop.theguardian.com or call 0330 333 6846. Free UK p&p over £10, online orders only. Phone orders min p&p of £1.99.