There are hard acts to follow, and then there’s Arctic Monkeys’ last album. 2013’s AM was, in so many respects, the perfect rock record of our times. With a few exceptions, mainstream indie guitar music often seems an exhausted idiom, trying on fifth-hand poses to diminishing creative returns.

On AM – a record about lust in LA – the Monkeys’ carnivorous riffs and piledriving drums exchanged body fluids with the slink and anomie of contemporary R&B and west coast hip-hop. AM’s qualitative brilliance was matched by its quantitative uptake, earning the Sheffield outfit No 1 spots internationally, and a massive new audience in the US. Was it the band’s best since their 2006 debut, Whatever People Say I Am, That’s What I’m Not? In all likelihood.

Five years on, Tranquility Base Hotel & Casino totally strips out the rock from the Arctics’ offering. Sure, there’s a fuzz-toned guitar on Golden Trunks. The low-slung She Looks Like Fun opens with a very rock band-y “one, two, three, four”. Guitarist Jamie Cook, drummer Matt Helders and bassist Nick O’Malley are here, but they are muted, repurposed. All contribute to the vibe – not least, in soulful falsetto backing vocals – but their instruments are not slamming up against your chest cavity and dislocating your spleen.

Instead, singing guitarist Alex Turner has taken his muse off into space – the moon, specifically. It’s an awe-inspiring place, somewhat despoiled by humans and their need to build hotel-cum-casinos near the Sea of Tranquility. If you’ve been to Niagara Falls, you’ll have a flavour. There’s a taqueria on the roof, too, scoring four stars out of five (“and that’s unheard-of”, vamps Turner).

Give yourself up to one of our very greatest songwriters, and then it’s a riveting and immersive listen

This is another record, tangentially, about coupling and uncoupling in LA, as songs like Science Fiction (“I must admit you gave me something momentarily in which I could believe”) and Batphone (“I’ll be by the batphone if you need to get a hold”) attest. But it is set, sonically and aesthetically, in a cocktail lounge, looking back on the home planet with a seething croon.

Turner is the louche, troubled guy at the piano, trying to say something of existential worth in a devalued, cheesy, light entertainment medium. In this, he is echoing Sheffield’s other two big-beast bards, Jarvis Cocker and Richard Hawley, who have paved the knowing road to the ivories before him. But Turner has also arrived at a place very similar to where fellow LA over-thinker Father John Misty resided on his last album. “I swing with the economists,” sneers Turner on One Point Perspective.

Just as most sci-fi is, yes, really about our own world, Tranquility Base is about the last few years in the US (“Breaking news: they take the truth and make it fluid”, Turner husks on American Sports), the warping effects of tech, and fame itself. If the frontman was illustrious enough before, in the wake of AM he only becomes more recognised; there’s the possibility here of a move in which the band might be trying to shake off some fans in the way Nirvana did after Smells Like Teen Spirit. So Tranquility Base is, at least initially, a frustrating listen if you joined Turner and co for the pugnacious guitars, or hold Arctic Monkeys up as the Oasis or Libertines for discerning rock fans with wide musical tastes.

Once you ditch the notion that AM’s successor should rock like it, and give yourself up to rolling around in the psyche of one of our very greatest songwriters like an olive in a martini, then it’s a riveting and immersive listen – an album-bomb dropped without preceding singles, re-emphasising the importance of a cohesive work, rather than a shuffled, Spotified deconstruction.

Other artists have laboured the fame/space metaphor before; the Weeknd is only the most recent. Turner is, obliquely, dealing with being a “motherfucking starboy”, doing some work on himself, and writing about writing; his gift is such that he can carry this solo-album-under-another-name. Hilariously, the album opens with the line “I just wanted to be one of the Strokes”; a jawdropping admission from one of music’s best beaters-around-the-bush. Even better is Batphone’s imagined scent. “I launch my fragrance called ‘Integrity’,” intones Turner, eyebrow making a parabola. “I sell the fact that I can’t be bought.”

The Starboy analogy works because, just as AM fed rock through the west coast genres, this album feeds lounge music through it too. One Point Perspective starts with dink-dink-dink keys, whose vibes recall Dr Dre on Still DRE. There are breakbeats here and there, and subtle funk. The new ingredients, though, are soul and 60s film soundtracks. The vintage loveliness of Curtis Mayfield and his ilk hits you from the off on Star Treatment; retro keyboard sounds abound. The amazing Four Out of Five partially recycles Cook’s bejewelled riff from Do I Wanna Know? and elides it with the memory of Lou Reed’s Satellite of Love (“take it easy for a little while”). The album exists in a narrow bandwidth of sound but that strip reveals depths and textures over time.

Buried inside scenarios, allusions and lunar perspectives are disarming moments of what you might laughably call “realness” in the hall of mirrors that is art. “So I tried to write a song to make you blush,” sings Turner, “but I’ve a feeling that the whole thing may well just end up too clever for its own good, the way some science fiction does.” There is a risk that this atmospheric record, one that wrong-foots expectation, might not land well. But this voyage into themed purgatory – what one song calls the Ultracheese – is worth it.