Novelists might appear to be in charge of their invented worlds, but they often have to wait a surprisingly long time to do what they want; fiction isn’t quite as malleable as it may seem. Kazuo Ishiguro – for all that tight authorial control he is associated with – is no different. For a long time, he tells me, as we sit in his Cotswolds cottage on a bright, wintry afternoon, he’s wanted one of his novels to feature a man and his horse. Now, with the publication of his seventh, The Buried Giant, he has finally had his way. “That lone rider figure has always done it for me,” he laughs.

Ishiguro’s prototype was the character familiar from the westerns he loves, the John Ford and Sam Peckinpah movies that so frequently, as in Ford’s The Searchers, open with a distant rider crossing a vast landscape, “like a self-contained little community of just a man and a horse, a lonely community that moves from place to place”. What particularly resonates with Ishiguro is the idea of a man somehow adrift from history: “There’s a real sense that there’s a whole world travelling there in that man … he’s out of time, somebody who belongs to a more violent world. And peaceful people need him when violence is needed, but he’s not really welcome in a peaceful community.”

But it was not to the American frontier that Ishiguro looked for the backdrop for his own lone rider. Instead The Buried Giant, his first novel since 2005’s Never Let Me Go, takes us back to Britain after the Romans have departed, and the Anglo-Saxons are on the brink of fully occupying the island; the moment, in other words, “when England is created”. Its horseman is none other than Sir Gawain, the youngest of King Arthur’s knights, a figure of myth and legend, of chivalry and poetry; and in Ishiguro’s imagining, now an old man responding – just like John Wayne in The Searchers or James Coburn in Pat Garrett and Billy the Kid – to one last call to arms. “Dressed in rusted chainmail and mounted on a weary steed”, Gawain’s “sacred mission” is to slay the she-dragon Querig.

The novel inhabits the gap in the historical record, argued over by archaeologists and historians, about what happened after the Roman occupation. The hypothesis Ishiguro brings to life is that waves of Anglo-Saxon migrants from the Germanic countries landed in the eastern part of Britain, “and at a certain point they massacred the people who were living here. There was what today we’d call ethnic cleansing – and they just vanished. The Britons, basically, were slaughtered.”

In support of this theory – another view suggests that the different groups interbred and assimilated – is that so little survives, in terms of place-names or the language that we use today, of the Romano-Celtic spoken at the time. It’s possible that it endures, in remnants, in Cornish or Welsh; that the survivors of the genocide fled to the westernmost parts of the country. By alighting on the moment before this brutal drama begins, Ishiguro captures a country on the brink of seismic change. As he puts it: “It’s not any old conflict that’s about to happen. This is the Anglo-Saxon settlement.”

The cover of Kazuo Ishiguro’s The Buried Giant. Photograph: Faber

It was both the uncertainty of what actually happened and the distance from contemporary events that attracted Ishiguro. His choice of setting was sparked by a reading of the 14th-century poem Sir Gawain and the Green Knight or – more accurately – a tiny moment near the beginning when Gawain, “quite a pampered guy”, has to travel between castles, in a land without comfortable inns or courtly protection, constantly being chased out of villages by wolves and up hills by ogres: “And there’s a series of all the irritating things that happened in the countryside, how freezing cold it was, and rain, nowhere to shelter, and then he gets to the other castle and the story continues. And the way these things are mentioned, particularly the ogres, as if they’re just like boars or something in a field, I suddenly got this vision of a landscape … I thought, that’s quite a fun place to put something. And things like ogres and elves could be completely banal.”

But it wasn’t just the prospect of “fun” that got Ishiguro going – and neither has writing the novel been plain sailing. He was some way in, he explains, when his wife Lorna, who is always his first reader, told him: “‘This just will not do. I don’t mean you have to change a few scenes here and there. You’re going to have to start again if you want to do this. Because the language is all wrong … some of it’s laughable.’” It was, he admits, a “horrible moment”, but he also points out that he started Never Let Me Go twice during the 1990s, “so I don’t panic”. And on the bright side, the break gave him time to write the five musically themed stories in 2009’s Nocturnes. He also saw the film of Never Let Me Go through production. Ishiguro is supportive of the movie adaptations of his books, another of which now looks likely, with Scott Rudin having acquired the rights to The Buried Giant. More generally, Ishiguro believes in the potential harmonies between film and cinema; he also writes screenplays, including 2005’s The White Countess.

Returning to The Buried Giant after completing the other projects, he made the language much plainer – less of the “behold this” and “pray that” – and wrote it much more quickly. Ishiguro had been looking for a vehicle in which to explore a specific set of ideas about societies and historical events, but didn’t want it to be easily and obviously relatable to any particular place or time. “It seemed to me wherever I put this story down, there was a danger that it would be seen as about that,” he says, but he was adamant that he didn’t want to look “peculiarly interested in a particular crisis in history. And so I was trying to find something that would be obviously fictional.” Ogres, elves, knights and a quest-style narrative would at least partially solve the problem. “A landscape like that would clearly signal that this was fantasy and you’re supposed to apply it to many situations.”

Illustration by Jonathan Edwards

Not that he went the whole hog. “I didn’t want a fantasy world where anything weird could happen. I went along with what happened in the Samurai tales I grew up on. If it’s conceivable that the people of the time had these superstitions or beliefs, then I would allow it.” In other words, he wouldn’t have suddenly allowed a flying saucer to appear in the sky.

For the reader, though, the language and the appearance of the kinds of creature more usually associated with the fantasy genre is something of a shock. Ishiguro is moderately sympathetic, although to him, of course, it makes perfect sense. “I often don’t appreciate the extent to which it might be a surprise,” he concedes. “I’ve come at it from the inside. What you see at the end is my arrival point. I’ve got there with what to me are fairly logical steps, but if you encounter it cold you think, ‘God, this is odd.’ This is quite a departure.”

Then again, he’s used to it by now. “I had this moment very early in my career when I went from writing Japanese books, or at least books set in Japan with Japanese characters, to writing something like The Remains of the Day. At the time, people were startled by this. They were saying, ‘That’s a bit odd, what does he know about English people? Well, I suppose he’s lived here for a while.’ But if you look at the reviews that came out at the time, and the comments made on review shows, this is what people talked about more than anything else – isn’t it extraordinary that this young Japanese guy should know so much about English butlers, or English high society between the wars. There’s a kind of surprise; perhaps people are slightly uncomfortable that I made what they see as a jump, from being someone identifiably writing about my ethnic background, if that’s the word, to someone who wasn’t.”

In fact, Ishiguro often maintains – perhaps slightly teasingly – that he writes the same book over and over again. But he also writes in the gap between two cultures – or, more precisely, between our perceptions of what those cultures might represent, or what properties they might display. He was born in Nagasaki in 1954; his mother, Shizuko, had survived the atomic bomb attack there when she was a teenager. His father, who was brought up in Shanghai, was an oceanographer, and it was his job that led the family to move to Guildford, where his mother still lives, when Kazuo was five. In a 2008 interview with the Paris Review, Ishiguro recalled how struck he was by the quietness of England, the sudden diminution in noise and images, the abiding sense of greenery. From his grandfather in Japan he still received colourful, busy books and comics; finding their English equivalents rather dull, he developed instead a great enthusiasm for Sherlock Holmes, even coming up with his own detective stories.

But more than writing, his early creative impulses found expression in the song lyrics he still writes; during a gap year in 1973 between Woking grammar school and a degree in English and philosophy at Kent University, he hitchhiked around the US with a guitar over his shoulder. His musical heroes are Bob Dylan, Leonard Cohen and Joni Mitchell; a recent favourite film was the Coen brothers’ spirited reimagining of the Greenwich Village folk scene in 1961, Inside Llewyn Davis. Although his career as a musician never took off, he believes this early experience helped to form his distinctive prose style, to which the words “austere”, “reticent” and “pared down” are often attached. By the time he signed up for the creative writing MA at the University of East Anglia in 1978, this sensibility was already ingrained enough for course tutor Angela Carter to describe his work as “very grown-up for a young lad”.

Malcolm Bradbury, his other tutor, perhaps got close when he noticed the “quality of irrealism” and the “texture of oddity” in Ishiguro’s first novel, A Pale View of Hills, which was published in 1982 and won the Winifred Holtby memorial prize. The remarks came during a fascinating conversation between the two at the ICA just after publication, in which Ishiguro, who grew up speaking “pidgin Japanese” at home, talked about his dual‑culture upbringing. “It’s very difficult for me to distinguish how much Japanese influence I’ve actually inherited naturally, and how much I’ve actually generated for myself because I felt I ought to,” he explained. “I think I certainly do have a tendency to create a Japaneseness about my writing when I do write books in a Japanese setting.”

Emma Thompson and Anthony Hopkins in The Remains of the Day (1993). Photograph: Everett Collection/Rex Features

But his books didn’t confine themselves to that background. After A Pale View of Hills and 1986’s An Artist of the Floating World, which won the Whitbread book of the year award and was shortlisted for the Booker prize, Ishiguro turned to England between the wars and to Stevens, the emotionally repressed butler who some years later reflects on his unquestioning service to his former employer, the pro-German Lord Darlington. “It is sometimes said that butlers only truly exist in England,” confides Stevens. “Other countries, whatever title is actually used, have only manservants. I tend to believe this is true. Continentals are unable to be butlers because they are as a breed incapable of the restraint which only the English race is capable of.”

The Remains of the Day won the Booker prize in 1989, and was later filmed by Merchant Ivory, from a script by Ruth Prawer Jhabvala (Harold Pinter had worked on an earlier version), with Anthony Hopkins taking the role of Stevens. Both commercial and critical success meant continuing freedom to experiment, and in 1995 Ishiguro published The Unconsoled, which could hardly have been more different. At 500-odd pages, it was set in an unnamed city in central Europe, and narrated by a concert pianist whose every effort to fulfil his obligations is thwarted by a series of bizarre obstacles. Its aesthetic was informed above all by the logic of dreams. Critic James Wood said – a much quoted comment – that it “invented its own category of badness”; Anita Brookner described it as “almost certainly a masterpiece”.

His next novel, When We Were Orphans, also had a dreamlike quality. Focusing again on the interwar period, it was the story of Christopher Banks, a famous society detective whose life has never really risen clear of the early trauma of his parents’ disappearance in Shanghai, and who believes that he can still crack the case – which, it gradually becomes apparent, operates as some kind of mental stand-in for the approaching global conflict. It is a deeply mysterious, upsetting novel, and also confirms Ishiguro as a master of psychogeography; his landscapes, whether they are bits of half-remembered English countryside or cities in the midst or aftermath of conflict, are suffused with mental imagery that potently blurs the distinctions between what characters – or readers – see and what they imagine, between what might have been and what will perhaps come to pass.

The nightmarish final section of When We Were Orphans sees Banks struggling through chaotic, contested back alleys during the battle of Shanghai, far from the safety of the international settlement, searching for his long-vanished parents. Sustained over the course of 50 utterly gripping pages, it is studded with abandoned buildings, impassable tunnels and dispossessed people of uncertain identity, and with sudden changes of scene: “I would often stumble into a barely damaged chest of drawers or shrine, giving the impression the family had just gone out for the day. But then right next to such places I would discover more rooms utterly destroyed or flooded … Once I came upon three dogs savagely tearing something apart, and drew my pistol, so convinced was I they would come for me; but even these animals meekly watched me pass, as though they had come to respect the carnage a man was capable of wreaking.”

When We Were Orphans was shortlisted for the Man Booker prize, as was his next novel, Never Let Me Go (2005), which lost to John Banville’s The Sea in an exceptionally strong shortlist that also included Ali Smith’s The Accidental, Zadie Smith’s On Beauty, A Long Long Way by Sebastian Barry and Arthur and George by Julian Barnes. The dystopian, science-fiction element of the story – cloned children are bred for organ donation and must nurse one another through their “donations” until, at a horribly early age, they “complete”, or die – meant that for a long time, Ishiguro would be invited to comment on TV and in newspapers every time there was a “weird science” story. It too was made into a film, with a screenplay by Ishiguro’s friend, the novelist turned film-maker Alex Garland.

Keira Knightley, Carey Mulligan and Andrew Garfield in Never Let Me Go (2010). Photograph: FoxSearch/Everett Collection/Rex Features

Of course, the book is not really about cloning, but about mortality, which Ishiguro believes links it to his new novel: “That sense of what happens to love between people when the proximity of death is felt is something that both books have in common. And that is quite an emotional subject. It would be odd if it wasn’t.” (Nonetheless, he adds, he doesn’t like to have extraneous emotion in his novels. “It has to be about the big thing. I tend to avoid incidentally sad scenes. I don’t like orphans and dogs dying in my books.”)

As The Buried Giant opens, we are introduced to an elderly couple, Axl and Beatrice, who, impelled by some vague sense of unease, decide to leave their small, warren-like community and go in search of their son. Their journey is made more precarious both by the numerous dangers that might assail them, including the dark forces of the “Great Plain”, and by their own tenuous grasp on reality: for snaking in and out of the narrative is a nebulous mist that has rendered the population strangely forgetful, aware that certain episodes in their past have taken place, but mystifyingly unable to access them.

It is this mist that is The Buried Giant’s ideological centre, the nub of what Ishiguro wanted to explore. He had previously concentrated on how individuals struggle with painful memories, caught between a desire to confront them and a fear of what might be unearthed; sometimes past trauma is so deeply embedded, as in A Pale View of Hills and When We Were Orphans, that it warps perspective, giving rise to fractured narratives in which both the protagonist’s and reader’s sense of what is real is brought into question. Now, though, he wanted to explore how that might work at a societal level. And while his incidental reading of Sir Gawain and the Green Knight provided him with the setting, his interest in the theme went back much further.

He recalls being asked, over a decade ago, to chair a panel at the World Economic Forum and settled on a topic that was preoccupying him greatly: how societies remember and forget, when and how we decide it is time to move on from the past, and what implications this has for the pursuit of justice. His panel members included Holocaust survivor and political activist Elie Wiesel and a then deputy mayor of New York, Dan Doctoroff, at the time engaged in working out how the city should commemorate 9/11.

If I had been a generation older, what would I have done? Would I have also been a fascist?

For himself, says Ishiguro, referring to The Remains of the Day and An Artist of the Floating World, “I’d always been slightly uneasy about the idea that you could make an individual like Stevens the butler, or a Japanese artist of a certain period, representative of the whole of English society between the wars, or Japanese society after the second world war.” Those characters were pictured looking back at lives devoted to what turn out to be dangerous principles – in both cases, fascism. They were explorations of “what happens if you do your very best, but because you don’t have an extraordinary perception of the world, and you can’t help but be a part of the society you live in, you invest all your talents in something that later you think you shouldn’t have”. They are elegies for lives not merely wasted but fundamentally misguided.

This was the key territory of Ishiguro’s first three books, and he says it was “very, very clear” to him as an area he was determined to explore: “Because I was a young guy at the time, I was aware of my parents’ generation, and the generation in Germany, who because they’d lived when they did, they had been caught up in that sort of fervour. I felt if I had been a generation older, what would I have done? Would I have also been a fascist?”

It also chimed with his own experiences as a young man, when he was working for the housing charity Cyrenians, where he met his wife; the couple married in 1986 and have a daughter, Naomi, who is in her early 20s. Back in the 1980s, Ishiguro found himself a frequent participant in political debates on key issues of the day, such as the campaign for unilateral nuclear disarmament. As time went on, he says, he began to ask himself: “How sure are we that we’re backing the right cause? We’re well-meaning, but most of us are just being swept along. We’re not sitting down and really examining why we’re taking certain stances. We belong to certain tribes, and there’s always the correct slogan and position on every issue for our little tribe.”

Having explored the intersection between politics and personal belief, Ishiguro wanted to move on to examine how communities, societies and nations deal with memories, and whether it is crucially different from the ways that individuals do. He wanted to know who controls a society’s memories, who mobilises them and to what ends; whether they can be used, as he believes Slobodan Milošević did in the former Yugoslavia, to awaken dormant ill-feeling between people.

Ishiguro in London in January. Photograph: Graeme Robertson for the Guardian

“What are the main mechanisms by which a country like Britain or France or Japan remembers? Is it by means of the literature, is it by means of museums, is it official history books? What is it? It’s some mixture of all those things, but in the end it comes down to what ordinary people actually have in their heads about what happened in their country,” he argues. Along with the rest of the world, he had watched as Yugoslavia disintegrated, and as genocide claimed at least 800,000 lives in Rwanda; as well as the obvious horror, he experienced “a profound disappointment, because after the Berlin Wall came down, and we were supposed to be at the end of history, we had this idea that things were going to become peaceful”. Then, suddenly, “you had concentration camps, death camps, massacres like Srebrenica, right in the middle of Europe. And the astonishing thing then – and I suppose looking back it shouldn’t have been so astonishing – was the fact that people, neighbours who’d been living with each other for decades, just turned on each other and massacred each other. It is well known that in those villages, certain houses were marked with crosses the night before, so that the militia could go through and just kill the people in those houses. And these were people who’d been eating in each other’s houses, and looking after each other’s children.”

He is interested, too, in how countries such as Rwanda and South Africa are able to move on from such widespread violence and trauma. “To some extent,” he notes, “you’ve got to abandon justice and grievance to break the cycle of violence.”

But also feeding into Ishiguro’s thought processes were examples of a collective reaction to trauma from further back. Japan in the immediate aftermath of the second world war gave him the setting for his first two novels; he has clearly been long preoccupied by what he calls the country’s “really strange version” of what happened during the conflict: “Japan as a nation wiped out the fact that they were aggressors in the second world war and that they rampaged through a lot of China and east Asia slaughtering people. Most people in Japan remember the second world war as a great tragedy which climaxed with two atomic bombs.” He adds that they were encouraged in this view by the American authorities, who needed Japan to be a sturdy ally against communist Russia and China; but that the consequences of that version of events having taken such firm hold creates problems between Japan and China to this day.

Ralph Fiennes and Natasha Richardson in The White Countess (2005), scripted by Ishiguro. Photograph: Allstar/Merchant Ivory/Sportsphoto

Vichy France, too, is an area of fascination, not merely because of its own interpretation of its wartime history – “They’ve all convinced themselves that they were resistance fighters rather than that it was a country that sent its own Jews to Auschwitz, and the Vichy regime was happily collaborating, and in every small village there were people who betrayed other people to the Gestapo” – but because of the way that interpretation shaped popular culture. To say that Ishiguro and his wife are film fans is something of an understatement; they have a home cinema in their north London house and organise their own mini-seasons (recent examples are screwball comedies and war films, which they’ve temporarily shelved after the distress of watching Elem Klimov’s drama about the Nazi occupation of Belarus, Come and See). They also, clearly, think very deeply about them; and one of their theories about French postwar popular culture – both the Nouvelle Vague in film and the nouveau roman in literature – is that “there was a real desire to put an end to the kind of realist narrative fiction tradition, because nobody wanted a Balzac to write about what the hell happened. That’s the last thing you want.” In movies, the argument continues, film-making became ever more personal, more inward – “a way of creating what seems to be serious art without tackling this huge thing that’s staring you in the face. What the hell happened to us? How did we face up to the challenges of Nazi occupation?” Meanwhile, the films that were actually addressing the legacy of war-time France were the genre films, the gangster stories of honourable thieves and corrupt policemen and politicians.

Alain Delon is keeping a watchful eye as Ishiguro is telling me all this – a large portrait of the French film actor, specially commissioned after the couple saw a vast picture of Claudia Cardinale by the same artist, occupies one wall, with another on the wall opposite. “We haven’t quite figured out how to use this place yet,” he says of the cottage they acquired a couple of years ago. It sits in a quiet corner of a village in the north Cotswolds which, he assures me, is a world away from Chipping Norton, associated with the likes of Jeremy Clarkson and Rebekah Brooks. “We still think of it rather like a holiday place. I come with a lot of good intentions, with lots of books and stuff to do, and I never do it. We spend all our time wandering about local cafes. I’m a bit bad about tea and cake.”

I’ve managed to stay relatively pure. I stick to what I really want to write about.

Does he just have a sweet tooth, I wonder, or is there something about the ritual of afternoon tea that he likes? The question is prompted by the meticulously described upper-class ceremonials in The Remains of the Day, the constant tea-making of A Pale View of Hills, the grand 1930s hotels in When We Were Orphans. A bit of both, he thinks. “I’ve never really liked sitting in pubs in the evening drinking ale, but I do like sitting around chatting with tea and scones. But you’ve got to have the right setting.” When I drove into the village that morning, I tell him, I saw lots of togged-up ramblers, setting off with walking poles and charts. Is he likely to do that? He gestures, half-hopefully, half the opposite, towards a shelf holding books of walks and Ordnance Survey maps. “I think Lorna thought we would go for bracing walks and things, but we haven’t done that very much. We’ve just sat eating coffee-and-walnut cake.”

That is, of course, not “just” what he does. Most of the time, he is writing. But, I point out, he is conspicuous by his absence from public debate, from the writer-as-commentator position occupied by novelists such as Ian McEwan or Martin Amis or, in her essays, by Hilary Mantel.

“I did it once, right at the beginning of my career,” he replies. “It was the anniversary of one of the atomic bombs, and the Guardian asked me to write a piece about the atomic bomb’s relationship to literature. This was back in 1983 or something, when the cold war was still on, and people were much more preoccupied about nuclear weapons.” He thought he should do it; his first novel hadn’t even come out in paperback and he’d probably get a whole page in the paper. “I sat down and I thought, well, I don’t really feel strongly about anything, but I’d better work myself up into some position, and write a piece as though I do feel very strongly about something to do with it. I came up with this concept of the pornography of seriousness, that some people would often bring in issues like the Holocaust or the atomic bombs into otherwise fairly ordinary stories, so that the stories would be given a serious dimension.” It sounds pretty plausible and, indeed, he thinks it was pretty well written. “But this was something I hadn’t been going around fuming about, or even thinking about until I got asked.” He laughs. “But if you read that piece now, you’d think I had; a guy who was born in Nagasaki who felt the whole nuclear thing had been exploited by people just trying to lend spurious weight and seriousness to their otherwise rather banal piece of fiction.”

After that experience – quite the reverse of what happened when he wrote fiction – he realised that “if I keep doing this, I won’t know who the hell I am. I’ll just be a sum total of these positions that I’ve taken up to fulfil commissions.” Instead, he decided that he’d be better off spending the time working out what really interests him – and has spent the following 30 years doing precisely that. “I’ve managed to stay relatively pure. I stick to what I really want to write about,” he says, although he thinks others should feel free to air their views and be given a platform to do so. For him, though, “that’s not my job. I write novels. I try and write films. I write songs. That’s all I can do.”

The Buried Giant is published by Faber on 3 March