When Paul Auster was 14, a boy just inches away from him was struck by lightning and killed. “It’s something I’ve never got over,” he tells me. He was at summer camp: “there we were, nearly 20 of us caught in an electric storm in the woods. Someone said we should get to a clearing, and we had to crawl, single file, under a barbed wire fence. As the boy immediately in front of me was going under, lightning struck the fence. I was closer to him than you are to me now; my head was right near his feet.”

Auster didn’t realise the boy had died instantly. “So I dragged him into the clearing. And for an hour, as we were pounded by intense rain, and attacked by lightning spears, I was holding on to the boy’s tongue so he didn’t swallow it”. Two or three other kids nearby had also been struck and were moaning; “it was like a war scene. Little by little, the boy’s face was turning blue; his eyes were half open, half shut, the whites were showing.” It took Auster a little while to absorb that, had the strike occurred just a few seconds later, it would have been him. “I’ve always been haunted by what happened, the utter randomness of it,” he says. “I think it was the most important day of my life.”

A similar incident occurs in Auster’s new novel, 4321. Archie Ferguson, a 13-year-old full of promise, enthralled by The Catcher in the Rye and his first kisses, runs under a tree during a storm at summer camp. When lightning strikes, he is killed by a falling branch: “as his inert body lay on the water-soaked ground … thunder continued to crack, and from one end of the earth to the other, the gods were silent”.

But this is the fate of only one of four Archie Fergusons in the novel. Auster’s fiction has always explored the moments in which lives, thanks to chance and circumstance, take different directions, and in 4321 this idea is presented in its purest form. The novel begins with the birth of Ferguson on 3 March 1947 to Stanley, who runs a furniture-appliance store in Newark, New Jersey, and Rose, who works for a photographer. What follows is four versions of Ferguson’s story. The four Archies have the same starting point – “the same parents, the same bodies, and the same genetic material” – but, as they gallop through childhood and adolescence, they take divergent paths. Each Ferguson lives in a different New Jersey town and has a different configuration of family and friends. As their stories unfold in rotating chapters, they become increasingly distinct people: the influence is felt of money, or the lack of it; divorce; education; and all the other factors that shape early lives. Auster presents four lovingly detailed portrayals of the intensity of youth – of awkwardness and frustration, but also of passion for books, films, sport, politics and sex.

All the Archies are bursting with intelligence, and all are aspiring writers. All fall for the captivating Amy Schneiderman, though each relationship plays out in a different way. One Ferguson has a car crash and loses fingers; one is bisexual; one has a friend who dies suddenly; one lives in a garret in Paris rather than going to university; the father of one dies in a fire. It will already be clear that some of the four lives are shorter than others: after the storm in the camp, four Archies becomes three and, as the reader looks ahead, the title of the book takes on a more definite meaning.

Trump’s election is appalling. I’ve been struggling to work out how to live my life in the years ahead

“As far as I know, no one has ever written a novel with this form,” Auster says. Talking in his Brooklyn townhouse, we try to think of comparisons: I come up with Kate Atkinson’s Life After Life; he mentions a film by Krzysztof Kieślowski. But neither are exactly right. “At first, I didn’t know how many Fergusons I wanted to have,” he continues, “I just knew that it was an idea I have been puzzling over all my life.” What he is driving at is not only the role of contingency and the unexpected, but the “what ifs” that haunt us, the imaginary lives we hold in our minds and that run parallel to our actual existence. How might things have turned out had I gone to a different school, or had I not run into the person I married? These are the shadows of our other possible lives (and deaths). “It is a very powerful notion,” Auster believes, “and it drove me through the writing of the novel.”

4321 is published to coincide with Auster’s 70th birthday. He regards it as “the biggest book of my life” – and not just because, at 900 pages, it’s three times as long as any of his other 16 novels (it’s “an elephant”, he admits, “but I hope it’s a sprinting elephant”). In terms of his reputation, he is convinced, it “is going to dominate everything. I feel I’ve waited my whole life to write this book. I’ve been building up to it all these years.”

Its writing became urgent to him. “I stayed downstairs in my bunker” – the basement of his brownstone – “and worked almost seven days a week. I wanted to live to finish it.” He pauses to suck on an e-cigarette: two years ago he abandoned the small cigars he had always chain-smoked, and which have given him his much admired raspy voice (“like a piece of sandpaper scraping over a dry roof shingle” he has said). “I started the book at 66, which is the year my father dropped dead of a heart attack. And once I passed that boundary, I began to live in a very creepy world. I’ve settled into it now, but early on, there was a thought of sudden death in my head.”

Auster has been a starry presence on the international literary scene for decades, ever since his New York Trilogy in the mid-1980s established him as a fashionable author who could deliver pacy plots with a dash of existentialism and literary theory. The first novel in the trilogy, City of Glass, features a writer, Quinn, who is mistaken for a private eye called Paul Auster: it is a postmodern tale of urban alienation, summed up by an editor as “Kafka goes gumshoe”. With his black clothes and expertise in French poetry, his love of baseball and Samuel Beckett, Auster offered a stylish and accessible intellectualism, East Coast meets Left Bank. He became the best example of an avant-garde writer who had found a mainstream audience.

Facebook Twitter Pinterest Paul Auster interviewed at the BFI

Venerated in France and a bestselling novelist in the rest of Europe, he was less celebrated in his home country, though this changed when, in the mid-90s, he made, with Wayne Wang, the voguish film Smoke, and was involved in other movies. More attention then began to be paid to his delicate, thoughtful works of autobiography, and to such novels as The Music of Chance with its desolate solitary male, hardboiled thrills and its swerve into fable and absurdism. He published frequently, and began to amass a body of work distinctive in its themes and playfulness with form (the nesting of texts within texts, self-referentiality, and so on). His close literary friends – Don DeLillo, Salman Rushdie, Peter Carey and also JM Coetzee, with whom he has published an exchange of letters – are from the premier league, and he is married to the writer Siri Hustvedt. The couple – Auster soulful and sunken eyed, Hustvedt blonde and elegant – were once asked to appear in a Gap advert as the embodiment of metropolitan literary cool. These days, Auster is more of an old-timer, a Brooklyn institution, but his stature is unquestioned.

He has also been unafraid to make his voice heard politically, as a member of literature’s left-leaning establishment, and it’s hard at this moment to avoid the subject of America’s new president: “It’s all I’m thinking about right now”. Auster has in the past stood against the Iraq war and George W Bush, and he got into a public spat with the Turkish prime minister Recep Tayyip Erdoğan over imprisoned writers. On the eve of the recent American election, he described himself as “on the verge of a nervous breakdown”. Donald Trump’s message of “Make America Great Again” was, he argued, “really Make America White Again … I’ve never been in more despair about who we are and where we’re going.”

In the wake of Trump’s victory, he says, “I feel utterly astonished that we could have come to this. I find his election the most appalling thing I’ve seen in politics in my life.” The Russians hacking the Democratic party is “almost like a declaration of war, without bullets”. “I’ve been struggling ever since Trump won to work out how to live my life in the years ahead,” he says. And he has decided to act: “I have come to the conclusion to accept something that has been offered to me again and again over the years – to become president of PEN America. I have been vice-president, and secretary, but I’ve never wanted to take on the full burden. I’ll start early in 2018. I’m going to speak out as often as I can, otherwise I don’t think I can live with myself.”

In 4321 the young Fergusons react to landmark events of 1960s US history: the civil rights movement and JF Kennedy’s assassination, the Vietnam war and the student protests at Columbia University in 1968. I ask Auster if there any connections to be made between then and now. “Tumultuous as those times were, they weren’t as depressing as what’s going on today,” he reflects. “How little has changed in American life since then. Race is still a very big problem. Stupid foreign policy decisions are still being made. And the country is just as divided now as it was then. It seems as though America has always been split between the people who believe in the individual above everything else, and those people who believe we’re responsible for one another.”

Auster has spent much of the last decade thinking about his childhood and the America he grew up in. When in his 50s, and after suffering a few bouts of ill health, he wrote a series of novels that centred on debilitated men (Timbuktu, The Book of Illusions, Oracle Night) and the presence of the dead in the thoughts of the living. During his 60s, however, Auster has gone back in time. (He has often mentioned a line from the poet George Oppen about growing old: “what a strange thing to happen to a little boy”.) His 13th novel, Invisible, featured a student at Columbia in the late 60s – when Auster studied there. And the author’s two recent fragments of autobiography, Winter Journal and Report from the Interior, are lyrical attempts to recall the sensations and thought patterns of his childhood self. “I think those two books laid the groundwork for this novel,” he says. “Without having dwelt in that land of long ago, I don’t think 4321 would have occurred to me.”

***

Ferguson, as Auster did, begins his teenage years in 1960: “I wanted to give a sense of what it felt like to grow up then”, the writer says: the new novel is a “story of human development … and I worked hard thinking about the different stages” of a young person’s life. “His mother’s name was Rose,” Ferguson 1’s story begins, “and when he was big enough to tie his shoes and stop wetting the bed, he was going to marry her.” 4321 is intent on conveying the way Archie, in all his incarnations, is formed both by personal drives and public events. The day Kennedy is shot is also the day Ferguson 1 has sex with Amy for the first time: they watch the coverage for hours on TV and then tumble into bed. (Any conjuring of a teenager’s life has to engage with the “obsessive thinking about sex”– Auster recalls that “it’s hard to get another thought in your head”.)

History unfolds: one Ferguson reports that in Alabama state troopers have attacked civil rights demonstrators in Selma, and that the Vietnam draft quotas have been doubled. A stepfather arrives on the scene, sport takes centre stage, cars are driven for the first time, Candide is devoured, Lyndon Johnson’s record is assessed. All the Fergusons are in a state of plasticity, on the way to being formed. They are all notably precocious, sensitive, likable and right-thinking: Auster wants to capture “the nature of the havoc roiling inside” Archie, “the contradictory muddle of hard, unforgiving judgments and raging contempt for big-dollar American greed, combined with an overall gentleness of spirit … his good-boy rectitude and out-of-step clumsiness with his own heart.”

Auster’s urge to convey youthful intensity in 4321 has induced him to change his style (in the past his work has been criticised for being too formulaic). He describes it as the “most ‘realistic’ novel I’ve written … The structure has a speculative feel to it, but it’s very down to earth.” There is nothing noirish about the book, there are no borrowings from genre fiction, and there is no evidence either of what used to be his trademark minimalism: 4321 not only teems with detail but is written in long, breathless sentences – some spilling over pages.

Facebook Twitter Pinterest ‘She’s an ardent feminist and I agree with her in all her positions. They are mine as well’ … Auster with his wife, Siri Hustvedt Photograph: Carlos Alvarez/Getty Images

“I’ve been building up to this in my recent books,” he continues. “And I felt a certain liberation in writing a sentence that goes on for three pages. It creates such a forcefield of energy.” It’s not stream of consciousness, but “as a reader you are following the thought processes of the characters”; the aim is to be propulsive. He has Ferguson 4 dismiss the usual writing advice of “show not tell” in favour of “tell and tell and tell”, and Auster says 4321 is itself a case of “telling and telling”. As such, the novel “contains a lot of the things I’ve been thinking about over all these years, and yet presented in a wholly different way”.

This is not to say 4321 abandons his themes or all of his metafictional trickery. In the detailing of Ferguson’s involvement in the Columbia University sit-ins, for instance, other characters from Auster novels who are graduates from his alma mater make an appearance – Marco Stanley Fogg from Moon Palace, David Zimmer from The Book of Illusions, Peter Aaron from Leviathan, Adam Walker from Invisible. The reader, as so often with Auster, steps momentarily into a hall of mirrors. “Yes, they are all there,” he confirms. “I wanted to bring back all my boys and have them there at the same time … just for fun. It’s a link in my work.”

Similarly, in the novel’s final pages, a crucial sentence refers to the “endlessly forking paths a person must confront as he walks through life”. This is a nod to Jorge Luis Borges – a writer often cited in discussions of the New York Trilogy – and his story “The Garden of Forking Paths”, at the centre of which is, appropriately enough, a novel where all possible outcomes of an event take place simultaneously. For those so inclined, there are plenty of other references to spot (for example the Princeton professor Nagle is an homage Auster’s friend, the celebrated translator Robert Fagles). More significantly, 4321 ends with a characteristic piece of illusionism that changes the nature of the novel entirely.

4321 is also typical in drawing on Auster’s own experiences. We know from reading his memoirs that he, like Ferguson 3, lived in a top floor maid’s room in Paris as a young man; that he, too, visited prostitutes – the parallels are too numerous to mention. In fact for readers familiar with Auster’s work, the novel is almost an echo chamber, with familiar themes and episodes – including the lightning story, which he has told elsewhere – reverberating within the multiple Ferguson narratives. The author has, it seems, poured his whole life into this book.

“I borrowed some things from my own life,” Auster says, “but what novelist doesn’t?” Unlike other writers, however, he rarely shuts down such conversations with a weary reminder that fiction involves making things up, but tends instead to volunteer exactly what in each book has been lifted from his own life. One instance from 4321 is a basketball match, played by Ferguson 4, which ends with a miraculous fluke of a shot and a fight between black kids and white kids. He was at such a match, and “it was very demoralising for me,” he remembers, “I was 14 and filled with idealism.” He mentions a character who is a direct representation of a friend’s father – a man full of wonderful stories of “sea voyages and women’s stockings and wireless, and his first martini”. And “I used my grandparents’ apartment,” he adds, “in a building on the corner of Central Park that wraps around to 58th St right to Columbus Circle.”

I am trying to represent the world that I know – the reality that I have experienced, which is so full of surprises

The borrowings go beyond incidents and places to include enthusiasms. Auster is able to indulge his well-known love of Laurel and Hardy when a troubled Ferguson 2 watches their films repeatedly at home on a projector screen. The novelist resurrects his own past as a student translator of French poetry (Ferguson 1 has a similar inclination) with a new rendering of a poem by Apollinaire. He even inserts into the narrative a text he wrote aged 19 called “The Droons” – described as Ferguson 4’s “most crackpot effort so far” – which includes the incomparable line: “After three days and three nights, I arrived at the village of Flom”. It is “pretty much word-for-word,” Auster says: “I thought: this is what I sounded like at 19, so why meddle with it?”

The narrative of the Columbia sit-ins is “accurately told” – done as “straight history”. In 1967 Auster himself took part in the protests, “got arrested, got kicked by the cops: I’m very glad I did it”. At one heady moment of student insurrection, he knew seven out of 10 men on the FBI’s most wanted list.

The retelling of anecdotes in Auster’s different books and the repetition of episodes from his own life have attracted some flak. Given Ferguson’s intellectual sparkle and progressive views, the author might be said to have opened himself up to the charge, made by an early reviewer of 4321, that he has “written a very long chronicle of his own genius”. Another review has referred to the “magnetic pull of Auster’s fascination with his own biography”. Readers of the new novel who don’t know his work simply won’t care, and the novelist, who always has mischief on his side against the critics, knows there is no simple correspondence between himself and Ferguson, and cares little about any overlappings in his work: “I am trying to represent in my fiction the world that I know – the reality that I have lived through and experienced, which is so full of surprises, and befuddling, and just not what one expects at all.”

***

Auster likes to pinpoint his beginnings as a writer to the day when, aged eight, he met his baseball hero Willie Mays at a New York Giants game and, mustering all his courage, asked him for an autograph. But neither his father nor his mother had a pencil, and eventually the player shrugged and walked away. Auster cried, and hated himself for crying, but from that day on – so the story goes – never left home without a pencil: “If there’s a pencil in your pocket, there’s a good chance that one day you’ll feel tempted to start using it” (52 years after the game, Mays gave him a signed ball).

Auster’s breakthrough with the New York Trilogy came when he was in his late 30s (and even then City of Glass was rejected by 17 publishers). He has written engagingly about the long years before that success, particularly in the memoir Hand to Mouth, which is subtitled “A Chronicle of Early Failure” (his early jobs included working on an Esso oil tanker). From 1971, he lived in France with the writer Lydia Davis, whom he had met in college. They eked out an existence as critics and translators and shared a belief that their poverty was romantic – until the situation grew desperate. They eventually returned to the US, with nine dollars to their name, and were married in 1974. The following year, expecting a child – their son, Daniel – the couple bought an old house in Duchess County, New York. On their arrival, Auster knew they had made a mistake. On the back porch were old pro-Nazi pamphlets and a copy of The Protocols of the Elders of Zion, and when moving a wardrobe Auster found a desiccated crow – “a classic omen of bad tidings”.

The following years were the bleakest of his life. He was so hard up, he touted around a baseball game he had invented using playing cards, and considered responding to an ad that promised “Make Money Growing Worms in Your Basement”. “I had spent my whole life avoiding the subject of money,” he writes in Hand to Mouth, “and now, suddenly, I could think of nothing else.” His turbulent marriage to Davis ended in 1978, and Auster faced what he has called a “very bad crisis”: “the ground was opening up … the things you clung to were no longer there”.

The death of his father, Sam, the following year (he had a heart attack while having sex with his girlfriend) triggered a change. Not only did a small inheritance enable Auster to keep writing, but he immediately embarked on a book of prose written in search of his remote, absent father, which became the superb memoir The Invention of Solitude. Most shocking was his discovery that in 1919 his grandmother had shot and killed his grandfather. She was acquitted on grounds of temporary insanity and her five children never mentioned the scandal; Sam Auster was eight years old at the time: “A boy cannot live through this kind of thing without being affected by it as a man.”

In 1981, the year before The Invention of Solitude was published, Auster met Hustvedt at a poetry reading. “The family joke,” she has said, “is that it took me about 60 seconds to fall really hard, and it took him several hours. It was a really fast bit of business.” Auster has often said that she saved him: “It sounds sentimental, because we’ve been together now 36 years,” he tells me, “but she is far and away the most intelligent person I’ve ever known.” She is always his first reader, and “hasn’t made a suggestion that I haven’t followed”. Hustvedt has recently published a collection of essays entitled A Woman Looking at Men Looking at Women, and I ask Auster whether she has ever picked him up on his representations of women. “Never,” he replies. “I’ve learned so much from her over the years. She’s an ardent feminist and I agree with her in all her positions. They are mine as well.”

***

Auster’s life-changing meeting with Hustvedt is, for him, a perfect example of the “befuddling” workings of contingency. In the same vein, he says that had he not received a wrong-number phone call (twice) from a man asking for the Pinkerton’s detective agency, he would never have written City of Glass. Such an interpretation of events can be pushed too far, or seem routine, but Auster has a deep affinity for tales of coincidence and the uncanny. “People who don’t like my work say that the connections seem too arbitrary. But that’s how life is.”

As if to prove it, between 1999 and 2001 he took part in the National Story Project on American public radio, in which he read out yarns submitted by “ordinary people” across the country – “true stories that sounded like fiction”. His original call was for tales “that defied our expectations about the world, anecdotes that revealed the mysterious and unknowable forces at work in our lives”. It was a success; thousands of stories were submitted and a selection published as True Tales of American Life. Auster found confirmation that “reality is truly as strange and incomprehensible as I thought it was”, and that others too felt the pull of improbability: “I’m happy to report that I’m not alone,” he told the Paris Review. “It’s a madhouse out there.”

Siri Hustvedt: ‘Trump was elected because misogyny is alive and well’ Read more

At the very beginning and end of 4321 is a joke about chance. It’s an adaptation of an old joke about a Jewish immigrant to the US that is apparently used by tour guides to Ellis Island. Before being interviewed by the immigration official, Archie’s grandfather, Isaac Reznikoff, is advised by a fellow Russian Jew to choose a new, American-sounding name, such as Rockefeller. But when the interview takes place, he forgets the name, slaps his head in frustration and blurts out in Yiddish, “Ikh hob fargessen (“I’ve forgotten”). The official thus writes his name down as “Ferguson” – a single moment with major consequences. (Auster says he originally intended to call the novel “Ferguson”, but had to change the title following the controversial shooting of Michael Brown in Ferguson, Missouri – “Now it’s a name that’ll be in American history for a long time”.)

Auster produced 4321 keeping to his celebrated old-school habits – in particular his dedication to writing in longhand, and his use of the trusty Olympia typewriter that has been on his desk since 1974. He has even published a book, with the artist Sam Messer, called The Story of My Typewriter, some of the original artworks from which hang above Auster as he talks to me. He likes the sound the keys make, he has said, but turns to the Olympia only once a paragraph he has worked on in his quadrille notebooks seems finished. He dislikes computers and thinks Amazon is “the enemy”. Each day, having worked for six hours on the new novel, he felt “depleted”: writing books is “exhausting, physically and mentally”. With Hustvedt, he would usually unwind by watching a classic film.

According to Auster, “only a person who really felt compelled to do it would shut himself up in a room every day … When I think about the alternatives – how beautiful life can be, how interesting – I think it’s a crazy way to live your life.” Dwelling again on Trump and the state of America, he remarks that he has often been tormented by the question he puts into the mouth of Ferguson 4: “If the world is on fire”, what use are works of fiction? “When you have a social conscience, there is a great push and pull inside of you about how to spend your time” and he has “never really come up with an answer”. But there remains the “hunger to write”, he insists, “to keep doing it”, even if the good sentences refuse to come. The “excitement, the struggle, is emboldening and vivifying. I just feel more alive writing.”

• 4321 is published by Faber on 31 January. To order a copy for £15 (RRP £20) go to bookshop.theguardian.com or call 0330 333 6846. Free UK p&p over £10, online orders only. Phone orders min p&p of £1.99.

• This footnote was added on 23 January 2017. After publication of this interview, Auster clarified that he has not yet received the role of president of PEN America, which will be decided by vote in 2018.