Midway through our interview, James Kelman is relaxing into an acutely knowledgable exposition of 19th‑century land clearances across the north-west of Scotland, and the resulting suppression of local language and culture, when it strikes me that the role of schoolmaster suits him far better than that of guarded, watchful subject.

He has recently returned from a term’s teaching at California’s San José State university. Does he enjoy it? He takes a long pause: “It depends,” he says. “Usually it’s upsetting for students.” There is a hint of the imp in his tone.

“When I was teaching creative writing in Texas, for example, the students found some of what I was doing difficult, asking ‘why are you giving us Van Gogh’s diaries?’” Or he would challenge why they had adopted a certain accent rather than speaking in their own vernacular. “Young people get angry when you point out they are fighting for the right to conform.” Surely that’s the point, to be the grit in the oyster? “If your students are upset it’s no longer a theoretical position, if they get angry and walk out of your tutorial.” He roars with laughter.

Kelman is so much more than a professional irritant, and yet it is now near impossible to consider his work without some reference to this perception – it is at the heart of how he describes his creative development. Although he’s been the recipient of a tranche of literary awards throughout his 40-year career, it is the furore around his 1994 novel How Late It Was, How Late that still defines his biography to the extent that it is possible to forget the key fact that it won the Booker prize. Describing one desperate day in the life of ex-con Sammy, and told in broad Glaswegian idiom, it was “a disgrace”, according to Rabbi Julia Neuberger, one of the judges; dismissed as “literary vandalism” by one broadsheet critic; and apparently contained 4,000 uses of “fuck”. Kelman is one of the most influential writers of his generation, described by Amit Chaudhuri as “the greatest living British novelist”, and an acknowledged influence on Alan Warner, Kirsty Gunn and Irvine Welsh; yet questions still abound as to whether his literary merit extends beyond the skilful rendering of a Glasgow accent.

At 70, Kelman ought surely to be enjoying financial and creative security. But he says his wife, Marie, a recently retired social worker, “basically kept” Kelman and his two children. And, after praising his new publishers, Canongate, for their “tremendous endeavours to market [his new novel] properly”, he adds with lugubrious intent: “I’m used to my work disappearing without trace”.

Dirt Road, Kelman’s ninth novel, tells the story of a father and son road trip through the American south, accompanied by the rhythms of Cajun and Creole music as well as Kelman’s more familiar regional lilts. It offers his closest examination yet of the father-son relationship, which has been a constant in his work, perhaps out of necessity since he examines masculine rituals and rhythms as thoroughly as he does their language. After the teenage son’s thwarted devotion to his chaotic father in How Late it Was, How Late, and the bigoted, distant dad in Kieron Smith, Boy, Kelman conveys with great poignancy the intractable silences and clumsy negotiations of intimacy between Murdo and his father Tom, both wounded by grief and loss following the death of Murdo’s mother.

Kelman signing books at the Hay festival, 2004. Photograph: David Levene/The Guardian

The novel is told from Murdo’s point of view, and although the publicity blurb rather cheekily describes it as Kelman’s “most accessible novel for years”, it continues in the tradition that he has made his own: disposing of the omnipotence of third person narrative, and offering a demotic and sometimes mercilessly mundane stream of consciousness that has attracted comparisons to Joyce, Camus and Beckett. According to Warner, another writer who has written from the perspective of youthful Scots: “Kelman brings alive a human consciousness like no other writer can.”

The story is also partly inspired by Kelman’s own travels around the US as a young man. The second of five boys, growing up in solidly working class areas of Glasgow, first Govan and later Drumchapel, he left school at 15 to begin an apprenticeship as a compositor. But at 17, visiting family who had emigrated to the US, he found himself too young to work under the country’s labour laws. “All I did for about three to four months was walk about LA.”

He also calls the book “a portrait of a young artist”. Murdo is a budding musician; Kelman himself wanted to be a painter before he was drawn to writing and, formidably well-read and intellectually agile across a number of art forms, he suggests it can be more useful to see writing through the prism of, say, music or visual arts.

“I didn’t start writing until I was 21 but I kind of knew I was going to be doing something. At that age, I read a lot of biographies of artists and that took me into literature. An early hero was Cézanne, but he took me to Zola.” Kelman went on to read French, American and Russian writers, but “I wouldn’t have been reading English literature, because of the class barrier.

“Why would you want to read things that were treating you as an animal? The Scottish voice was equated with being working class. If you weren’t working class, you would have lost it and assimilated to the bourgeois, as people still do whether it’s Morningside or Kelvinside [upmarket areas of Edinburgh and Glasgow] – an approximation of Hampstead Heath is what they attempt.”

Morningside, Edinburgh. Photograph: Suzanne Plunkett/Reuters

The limiting conventions of English literature have been felt keenly by Kelman all his life: “The problem is that, formally, there’s nowhere to go. The only way is to enter from within, and it’s treated as a hostile attack. If you feel: ‘I need to break down narrative structure here, I need to be involved in oratory devices as well as literary devices,’ then right away – as I’ve experienced for 45 years, with the Booker and so on – there is utter hostility. The red mist descends. Eventually you have to work out why you’re being atacked.”

Kelman insists that it was never a choice to be a sweary, unpunctuated breaker of literary convention. “Most of my stories were written from within my own culture, so you use the language as people use it. If you’re writing a story about a man in a pub, why can’t you use the language he speaks? It becomes a case that you can’t write that story, or can only write it from the outside, and most English literature is from the outside.

“It’s not that you make a decision to challenge convention because it’s really irrelevant to you as an artist. All you do is explore, and take it as far as you can. And it’s in the act of doing that that someone will say, ‘Well, what are you doing here? You can’t use the word fuck.’ So I can’t write about that character? That area of male working-class community cannot exist within literature?” I ask if he feels there has been any evolution since he won the Booker in 1994: for example with the awarding of last year’s prize to Jamaican author Marlon James, whose novel A Brief History of Seven Killings was heavy with patois.

He doesn’t know James’s work, he says, nor will he be drawn on his opinion of younger Scottish writers –many of whom credit him as a significant influence. As I throw names at him, and he looks blank, I can only conclude that – rather than not wishing to play favourites – he simply hasn’t read any. At another point, while talking about female and gay writers who have tried to transform dominant male values, I expect to hear a name such as Jackie Kay, but instead he offers Virginia Woolf and Jean Genet.

I feel that there are all these areas in Scottish culture that are still not properly explored

What he will say is that younger writers must discover the distinctive Scottish tradition themselves. “Writers like myself or Tom Leonard are working from that tradition, but maybe [younger writers] don’t realise that was what it is.” He points out that Justified Sinner, for example, the brilliantly eerie but neglected novel by Scottish author James Hogg, was published a couple of decades before Emily Brontë’s more celebrated Wuthering Heights. “I feel that there are all these areas in Scottish culture that are still not properly explored.”

One tradition that Kelman has previously been happy to place himself within is that of Kafka: in his writing he is drawn to scenarios that depict the petty humiliations and frustrations of bureaucracy – in hospital, a job centre, or the first day on the factory floor.

“I think that is an essential working-class experience,” he says. “Intimidation, provocation, sarcasm, contempt, disgust and so on. You learn how to cope with it as a young person, because you see your parents in that situation, for example if they’re dealing with the doctor or the headmaster, you know they’re watching their Ps and Qs.” We’re sitting in a cafe in Partick, which serves a decent egg and chips alongside your skinny latte. “You live in Glasgow around here and walk to the cafes across the road,” – Kelman nods to the more salubrious environs of Byres Road – “you’ll be aware of the language and social bearing that is used to intimidate. Which kind of tea?”

Kelman was one of very few senior Scottish creatives not to campaign during the referendum of 2014, although he did write in an essay that he cautiously supported independence. We met before the EU referendum, but he was keen to note that many of those supporting Scottish independence considered themselves internationalists rather than nationalists. In terms of an artist’s response to the recent upheavals across the Scottish political landscape, he argues that “much of the mood for self-determination came about through the work of the artists”, and adds, “I think what will happen is more of a resilience and awareness of why we’re not nationalists, but why we’re in favour of self-determination. So what you will get is an exploration of not so much Scottish identity, but the reality of where it is you come from as a Scottish person.”

Following their San José sojourn, Kelman and his wife have settled back into Glasgow life. They are now grand-parents. He avoids contemporary forms of communication, though his family uses Facebook for the usual stuff. He still writes every day: “Five o’clock this morning,” he says. “It is second nature but it’s also what I’m comfortable doing. Sometimes you would be better talking to writers the way you would talk to a musician or a visual artist. You have to always be practising, in that sense. You always want to go further than you’re going.”