"You know, this might be my last public statement", Iain Banks said to me on the phone when I was setting up this interview, and at the time that simply didn't seem likely: he was too full of ideas and opinions and schemes. He emailed me a fortnight ago, saying that he was hoping to be out walking around the village again by the end of the week. In fact, he died on 9 June. Nevertheless, the plans and hopes he had capture his quicksilver, optimistic personality, regardless of what transpired. To be robbed of 30 years he thought he might have had is one thing: to lose the few months he was cautiously anticipating seems especially cruel.

I was dreading the end of the interview, when I would say goodbye to him in the full sense of the word. Banks – 59 years old, the author of 29 books, someone I'd chaired at a Scottish book festival six months previously and across the country for years, and whose work I'd long admired – was dying. He would remind the reader that we all are: likewise, after his announcement, in April this year, that he had inoperable cancer of the gallbladder, he knew he had a radically foreshortened perspective on that inevitability. His public statement of the fact was a model of decorum, tempered by his distinctive wit. Beginning "I am Officially Very Poorly", he mentioned he had asked his partner, the Dead by Dawn film festival director Adele Hartley, to "do me the honour of becoming my widow". It came about sooner than anyone imagined.

❦

But back then, Banks answers the door, and looks thinner, less jaundiced than I'd expected ("At least I don't look like Grandpa Simpson any more", he says later), just as fizzing with energy. As we walk through the house, into his study, he laments that bookshelves never keep up with the amount of books. "That pile there" – he gestures towards a neat 20-by-four stack on top of a shelf – "are all unread. And, sadly, likely to remain so!" We go through his music studio ("It's a good acoustic space; no standing waves, apparently, and the books make for wonderful noise insulation") and, as I scan the shelves I see a tatty proof, of his first novel, The Wasp Factory. He lunges towards it to show how often he's repaired the pages with sticky tape. "Much restored," he has written inside.

The Wasp Factory

He has already addressed the unspoken topic. As we sit over tea and biscuits, I prepare to ask the first formal question (is "So how are you?" just dumbly insensitive? Is "Tell me about the new novel" too much as if this were just a standard, publicity circuit chat?). But Banks is off already, talking 19 to the dozen, brandishing a handkerchief with a Cyberman on it (a present) and saying, "You know, I've fallen out of love with Doctor Who, at least in its present incarnation. I just can't get along with it. People have suggested I should write for the programme, but, ach, I just couldn't. I might have been hopelessly naive but I hadn't realised there are just so many rules when you write a Doctor Who story, like the monster has to go back in the box at the end."

I suggest that what most readers thrill at in Banks's science fiction is exactly that sense of risk.

"Well," he says, "if you are going to write what a friend of a friend once called 'Made up space shit', then if it's going to have any ring of truth that means sometimes some of the horrible characters get to live, and for there to be any sense of jeopardy, especially in future novels, the good people have to die. Sometimes."

Banks freely admits that he enjoys writing his SF novels more than his "literary" novels, and the Culture novels more than his other SF. The hedonistic, anarchistic, post-scarcity series is "a hoot. It's my train set. I adore the freedom and the size of the canvas," even though writing them "requires a greater degree of concentration. And there's so much baggage with the Culture now that I have to get each new novel aligned with earlier Culture history. I don't have the same leeway to make things up compared with when I started."

He has no desire to "tie everything up with the Culture"?

"No. And it's a self-conscious decision; just like the Culture itself is determined to keep on going, refusing to sublimate or disappear off stage, so I think it would be too easy for me to lob in a series-ender. To that extent, 'destroying the whole universe' – an always tempting scenario when you realise in SF you can do anything – just seems too easy."

The Crow Road

That sense of stretching and twisting conventions and confuting the expectations of the reader applies just as much to Banks's "literary" novels, from the macabre fantasia of The Wasp Factory to the gothic profusion of The Crow Road to the baroque bildungsroman of Stonemouth. Again, though, narrative drive has to be balanced with plausibility: "It's a writerly truism – probably best illustrated by William Goldman in Adventures in the Screen Trade – that only real life can get away with the really outrageous stuff. The trouble with writing fiction is that it has to make sense, whereas real life doesn't. It's incredibly annoying for us scribblers. A lot of the time you're simply deciding how far down the path of unlikeliness you can go while still retaining the willing suspension of disbelief in the reader. You can't go too far – 'With one mighty bound he was free!' – because it just becomes ridiculous. Readers will start to feel that it's all too coincidental, too easy, too contrived and convenient for the writer's purposes. You're trying to decide how much you can get away with."

Of his new book, The Quarry, the publication of which has been brought forward, he says: "Quite realist. It's a fairly simple book as well; not many characters, there's only really one location and it doesn't muck around with flashbacks or narrative order." He adds: "If I'd known it was going to be my last book, I'd have been quite disappointed that I'm going out with a relatively minor piece; whereas something like Transition, a wild splurge of fantasy, sci-fi and mad reality frothed up together … now that would have been the kind of book to go out on. I'm still very proud of The Quarry but … let's face it; in the end the real best way to sign off would have been with a great big rollicking Culture novel."

The Quarry is narrated by Kit, a typically precocious and alienated Banksian teenager, who finds people stupidly predictable and yet infuriatingly random. He lives with his father, Guy, in a dilapidated house on the edge of a quarry, and exists really in an online game called HeroSpace. Kit describes himself as on "a spectrum that stretches from 'highly gifted' at one end to 'nutter' at the other, both of which I am comfortable with". Guy's student-day friends – film critic Hol, prospective MP Paul, dot.com power couple Alison and Rob, and former couple Pris and Haze, a care services manager and general drifter – all descend on the house, with the covert aim of finding a video of a film they made together at university that could compromise several careers. But the ostensible reason is that Guy is dying of terminal cancer.

Eerily, the book doesn't derive from Banks's current experiences but predates them.

"God, I'd nearly finished the book when I found out. It was bizarre," he says with bewildered candour. "Guy was always going to be dying of cancer; the book was always going to be predicated on that, and nothing really changed because of my own bad news. The initial story came about really quickly. Some books take a lot of teasing out and the coming together of previous ideas, but this one jumped in there fully formed in the course of a couple of days back in October 2012; so much so that once I'd had the idea I just left it for the best part of two months because it was ready to go at that point. It's purely experience that lets you know when a novel's ready to go. You know when not to overwork it before you start."

the quarry

Banks followed his usual schedule of writing in the early months of the year. He went to the doctor thinking his sore back was most likely due to having been sitting at a desk writing The Quarry. "On the morning of 4th March" – after he had been sent for a CT scan – "I thought everything was hunky dory except I had a sore back and my skin looked a bit funny. By the evening of the 4th I'd been told I had only a few months to live. By that time I'd written 90% of the novel; 87,000 words out of 97,000. Luckily, even though I'd done my words for the day, I'd taken a laptop into the hospital in Kirkcaldy, and once I'd been given the prognosis, I wrote the bit where Guy says, 'I shall not be disappointed to leave all you bastards behind.' It was an exaggeration of what I was feeling, but it was me thinking: 'How can I use this to positive effect?' Because I was feeling a bit kicked in the guts at this point. So I thought, 'OK, I'll just give Guy a good old rant.' Like I say; that's reality for you, it can get away with anything."

Banks revealed his illness a month later, and the book world was stunned by his lack of bitterness, the dignity of the statement. "Yes!" he explodes with laughter. "I know; it's not like me, is it?" Guy isn't the therapeutic residue, the lees of unexpressed anger. "I'm not Guy – for example, he deeply resents that life will go on without him. I think that's a stupid point of view. Apart from anything else, I mean, what did you expect?" The Quarry, nevertheless, is full of unsentimental, furiously exact unpickings of the cliches surrounding terminal illness. Guy skewers sloppy thinking, describing nostrums and alternative cancer treatments as like "running into a burning building and trying to put the fire out by means of interpretative dance". Banks agrees that "it would have been therapeutic, if most of the ideas hadn't already occurred to me".

So how does he account for the peculiar synchronicity of The Quarry and his life? "Chance. I was just imagining what it would be like. As a writer – it must be the same for actors – you're used to dealing with the idea of death and all the big questions. Unless you're writing purely for five-year-olds, about bunnies, you're going to have to think about death. Your characters will die and people will live on afterwards who cared about them. You need to be able to empathise with them. Of course, we all go through it; we all have people close to us die. But as a writer you really have to think it through properly, or it'll all ring false. It's almost one of the perks of the trade that you're forced to think about that stuff fairly deeply. So maybe when it comes along in real life, you're slightly better prepared to deal with it." He breaks off, laughs uproariously again and confesses: "No, I'm not sure about this. I'm just flying a kite here. I could be completely wrong, but it's got to be true for some writers."

Banks may have displayed a lack of anger at his diagnosis, but that does not mean that his righteous ire is extinguished. As we chat, he frequently loops off into hilarious denunciations. "I can understand that people want to feel special and important and so on, but that self-obsession seems a bit pathetic somehow. Not being able to accept that you're just this collection of cells, intelligent to whatever degree, capable of feeling emotion to whatever degree, for a limited amount of time and so on, on this tiny little rock orbiting this not particularly important sun in one of just 400m galaxies, and whatever other levels of reality there might be via something like brane-theory [of multiple dimensions] … really, it's not about you. It's what religion does with this drive for acknowledgement of self-importance that really gets up my nose. 'Yeah, yeah, your individual consciousness is so important to the universe that it must be preserved at all costs' – oh, please. Do try to get a grip of something other than your self-obsession. How Californian. The idea that at all costs, no matter what, it always has to be all about you. Well, I think not."

His political zeal burns equally ardently. He confesses that "for half a second", as he and Adele travelled across the Alps from Venice to Paris on honeymoon, he was "elated" when he heard that Thatcher had died. "Then I realised I was celebrating the death of a human being, no matter how vile she was. And there was nothing symbolic about her death, because her baleful influence on British politics remains undiminished. Squeeze practically any Tory, any Blairite and any Lib Dem of the Orange Book persuasion, and it's the same poisonous Thatcherite pus that comes oozing out of all of them." He didn't watch the funeral on TV – "It was just like a royal wedding."

We reminisce about other significant turning points. Blair entering Downing Street: "Watching the helicopter shots of his car journeying from Islington to Buck House was like witnessing the liberation of a city … yet almost immediately he was having tea with Thatcher. My injured self-respect can at least fall back on the fact that I never voted for New Labour – Labour yes, and nothing but Labour for as long as it existed and I could vote, but not for a party that embraced privatisation and refused to scrap nuclear weapons; not for a party slightly to the right of Ted Heath's government." As for the war on terror, there is palpable fury when he discusses "the great lie that our boys are fighting, killing and dying in Afghanistan to keep us safe. It's 180 degrees off the truth. They're dying worse than needlessly; they're dying to save political face, and for every grieving or just aggrieved Afghan family we create the conditions for further atrocities to be visited on us."

The Bridge

Having both seen and chaired Banks at various events I already know the answers to some of the questions most frequently asked of him. He thinks his best book is The Bridge, his worst, probably Canal Dreams. He'd love there to be a film of Consider Phlebas. He doesn't do Facebook or Twitter. (Over the course of our chat, he launches into a riff on inventing a Scottish version of Google called Guddle.) But, I wonder, as he thinks back over his writing career, which novel does he harbour a secret affection for? Which one didn't get the public and critical reception he thought it deserved? After pausing, he jokes: "I suppose I thought that about The Bridge itself at the time! At that point I hadn't even committed the cardinal sin of writing science fiction, so my nose was still relatively clean. But I suppose … A Song of Stone. I think it's up there with The Bridge and Use of Weapons. As always when I mention Use of Weapons I have to mention Mr MacLeod here: Ken saved that novel and came up with the absolutely brilliant idea of the two time streams going in opposite directions. With A Song of Stone, there's an elegance to it, I'd claim. I think it's my most poetic use of language. I come back to Mr MacLeod; I can't remember where the hell it was, but there was him, me and someone else in a pub in London. We were talking about A Song of Stone and this other person said he hadn't read it. He asked Ken, 'What do you think of it?' and Ken sat and thought – you know the way he does; disconcertingly, he's almost the only person I know who sits and thinks before he says anything – well, he said 'you know that bit you get at the end of one of Banks's novels where the relatively clear prose falls away and you get a really intense part where he brings out all his adjectives and big guns? With A Song of Stone the whole novel's like that.'"

He mentions that A Song of Stone was originally a poem. I hadn't realised he even wrote poetry. "Ah yes," he beams, "and thereby hangs a tale. There are bits here and there – poems top and tail the story in Use of Weapons for example" (I check back at home, and there they are: there is something unheimlich about reading, from 23 years ago "The catchment of these cultured lives / Was not in flesh, / And what we only knew, / You felt, / With all the marrow of your twisted cells" and "The bomb lives only as it is falling".) "The poems are a part of the desperate urge to get things that were supposed to be long-term projects out the way. I'm going to see if I can get a book of poetry published before I kick the bucket. I've got about 50 I'm proud of. I've been trying to convince Ken MacLeod that he should come in with me on this as I've always loved Ken's poetry. That, and it gives me cover. It stops the book being what it really is, which is a bit of a vanity project. If Ken comes in it will look more respectable, but I don't think he's falling for it. We'll see if it happens; I just don't know. I think my poetry's great but then I would, wouldn't I? But whether any respectable publisher will think so, that's another matter. I'll self-publish if I have to; sometimes I have no shame."

So neither the "I Am Officially Very Poorly" letter nor The Quarry represents the Last Writing of Iain Banks. A writer as prolific and prestigious as Banks is well used to critical commentary and review coverage, but none of that prepared him for the response to his public announcement. "Overwhelming," he says, looking genuinely moved, "just overwhelming." Then the grin switches back on. "The smartest thing we did was delaying putting anything out in public until we knew we'd be sitting in the transit lounge in Gatwick waiting to fly off to Venice. It meant the very worst of it would have been and gone. I kind of wanted to be old news by the time I got back. Plus, it was a quiet news day when my news came out – supposing Thatcher had died that same day, I'd have been nowhere near the front page. Nevertheless, yes, the response was astonishing."

Are there any other positives, I ask? "You know, I go and see my now very old Mum in her old folks' home, and I saw my Uncle Bob after getting my bloods done this morning and he's not in a good way," he says. "You look at the decrepitude of the very old and ill, and at least I'll be spared years of that. It'll be over fairly quickly. There's a wee bit of what Guy says in The Quarry about being well rid of some of the more ignoble aspects of the present day." Warming again to the theme, he continues: "I think there will be a political rebalancing and we'll stop swinging round to the right. In the unlikely event that I'm around for the referendum on Scottish independence I'm definitely voting 'yes'. I was saying last year that if we don't get it in 2014 we'll get it in my lifetime and now it turns out my lifetime might not extend as far as the first referendum and that just seems wrong – a Scotland still shackled to a rightwing England, especially with the rise of the bizarrely named Ukip (I think they'll find their acronym should be EIP actually) – I won't be sorry to be missing that. I won't miss waiting for the next financial disaster because we haven't dealt with the underlying causes of the last one. Nor will I be disappointed not to experience the results of the proto-fascism that's rearing its grisly head right now. It's the utter idiocy, the sheer wrong-headedness of the response that beggars belief. I mean, your society's broken, so who should we blame? Should we blame the rich, powerful people who caused it? No let's blame the people with no power and no money and these immigrants who don't even have the vote, yeah it must be their fucking fault. So I might escape having to witness even greater catastrophe.

"I think the referendum is going to be defeated, though I don't accept the argument that the whole question will just go away. It'll be back in five, 10 years again depending on the complexion of the next Westminster government. So, yes, I'm annoyed I won't get to vote in the referendum. I'm annoyed I won't get to ride an Edinburgh tram and I'm annoyed I won't get to go on the new Fife crossing.

"And," he sighs, "just not seeing so much of the near future. I'd love to see what's going to happen next, what's happening in the oceans of Jupiter's moon, Europa, and what else we'll find out just in our own solar system. And we're not far from being able to analyse the atmospheres of planets around other stars and maybe spotting the signs of life there. There's so much I would have loved to have seen. The positives? I've been lucky in that I've had such a good life. Simple as that. My first 30 years were pretty damn good and the last 30, since I got published, have been absolutely brilliant. I've so many good friends and been part of a wonderful extended family and I'll leave behind a substantial body of work."

There is a mordant clarity about Banks: his future is mapped out with military detail: "The only other thing I want to do is get as much of my music fettled as possible, and get a way to distribute that – a website where people could download it – as ever I don't know how good it is, and as ever I think it's brilliant. The music has to be brought to some level of respectability and not be embarrassing. That's the main thing, with the poetry, to be working on. It's going to be a really busy summer. Usually I'd be relaxing. But not this year, because it might be the last few months I have. I'm going to be unaccustomedly busy; death may come as a welcome release, just for the rest. Also, just in case there's some highly unlikely good news, I want to make sure I've got the next Culture novel ready to go if I can." I'm momentarily staggered. I don't know if it's the optimism or the Stakhanovite work ethic that is more impressive.

As we walk to the door, Banks pulls one final, left-field surprise. "Do you know that I know what caused the cancer?" I think I pull a face like Macaulay Culkin in Home Alone. "Cosmic ray," he says. "I won't brook any contradiction; it was a high-energy particle. A star exploded hundreds or thousands of years ago and ever since there's been a cosmic ray – a bad-magic bullet with my name on it, to quote Ken – heading towards the moment where it hit one of my cells and mutated it. That's an SF author's way to bow out; none of this banal transcription error stuff." Then the moment comes that I was dreading … but he says "See you soon" instead.

Iain Banks by the Forth railway bridge, which is close to his home. Photograph: Murdo MacLeod for the Guardian

On the train home, crossing back over the Firth of Forth, two things strike me. Firstly, the public is yet to decide if the Forth Replacement Crossing should be called Caledonia Bridge, Firth of Forth Crossing, Queensferry Crossing, Saltire Crossing or St Margaret's Crossing. Whichever uninspired choice is made, the old Forth Road Bridge should be immediately renamed the Iain (M) Banks Other The Bridge Bridge. Second, it wasn't that Iain was still Iain, despite an illness that was as unexpected as it was tragic. It's that in his last days he was more witty, more impassioned, more imaginative, more kindly, more caustic and even cleverer, as if concentrating and distilling the best of himself into the small time he had left. It was humbling to have been there.

❦

I listened to the recording of our conversation after the inevitable became actual, and what struck me was that most of the time we were just laughing. That laughter is gone. But its echoes are headed to the edge of the known universe.