‘I’m just borrowing Leonard,” says Jim Kay as he carries a skeleton from the dining room into his studio. Kay’s world, ostensibly a small Victorian house on a quiet street in Kettering, is in fact a realm as fully imagined as that of Hogwarts School of Witchcraft and Wizardry. Spread across the hedge outside the house is a large spider’s web with a healthy-looking inhabitant. Kay has been feeding it flies, because he knows he’ll want to draw it later. On the stairs, stuffed chickens peck at invisible seeds; the living room is governed by silk-feathered crows and one-eyed porcelain dolls. In the studio, clocks are set to the time in Narnia and Mordor.

Kay is the uncommonly gifted illustrator of A Monster Calls, the dark, award-winning children’s book written by Patrick Ness, and of an elaborate glossy pop-up book about bugs. More recently, he produced haunting monochrome drawings for a collection of stories about the First World War, and did some concept work for the television production of Jonathan Strange & Mr Norrell. Now that A Monster Calls is being turned into a film starring Liam Neeson, Kay is helping the film-makers, the monster being based very much on his own gigantic, creaking creation – one of the most spectacularly imagined nightmares in children’s literature. Still, no one was more surprised than Kay when JK Rowling’s publishers asked him to illustrate not one but all seven of the Harry Potter books, for glorious new large-scale editions, over the next seven years.

“I’d not really drawn children,” he says quietly, as if still stunned. “And I’m not known for a cheerful style of illustration.” Then there was the fact that the Harry Potter films had already visualised that universe so fully – why do it again, he wondered. And, of course, there was the pressure. As Kay puts it: “You don’t want to be known as the person who ruins the most popular children’s book in history.”

But after almost two years of work, seven days a week, Kay’s illustrated edition of Harry Potter and the Philosopher’s Stone is a triumph – a book so alive it seems to jump, explode and slither out of your hands as you read. Rowling has given it her public seal of approval: “Seeing Jim Kay’s illustrations moved me profoundly,” she wrote for the dust jacket. “I love his interpretation of Harry Potter’s world, and I feel honoured and grateful he has lent his talent to it.” She also wrote to Kay privately. “She sent a really lovely letter, and that’s the first time it hit me that this was real,” he says. “Imagine you’re a vicar and you find a Post-it note from God on your fridge. It was like that.”

Kay is genial, agile and occasionally as bashful as a young boy. He often holds his forehead and grins in embarrassment as he looks back over his work, and he describes almost every illustrative technique as having emerged from anxiety or failure. The happy accidents in A Monster Calls came about because he was too scared to draw (he thought the manuscript so wonderful it shouldn’t be illustrated at all) and some of the ink splotches he’d made by printing with old breadboards looked like hedgerows or trees.

As he shows me some of the work stored in his studio, he points out that most of his preparatory sketches for Harry Potter have gone. Not long ago, he says, he took a full carload to the rubbish dump.

Though Kay claims he hasn’t yet arrived at a style – meaning that one day, when he stops trying to do new things all the time, work might get a little bit easier – there is a telltale love of creatures in his rendering of Harry Potter. Owls take centre stage – magnificent clawed or droopy eyed, they all have character – and there are exceptional pages from a textbook on trolls and a guide to dragon eggs. The dog with three noses is rendered as if it were a much-loved and oft-drawn pet. A vivid green toad, keys with bright wings and a few hilariously expressive chessmen based on the 12th-century Lewis set are all full of Kay’s personality and humour.

But it’s true that he uses a wide range of techniques, from Ralph Steadman-like splats to Holbein-esque portraits to ghostly watercoloured landscapes, richly textured backgrounds and gnarly, impacted details that are all his own. Kay pulls out of a drawer an early sketch for his phenomenally finicky Diagon Alley panorama. Drawing it, he says, “was almost like knitting – you start at one end and move along”. But others were much trickier. The Astronomy Tower was a building he could never get right – at first he thought perhaps a serpent could pierce the tower, but he still couldn’t detach himself enough from reality. “I had this note for a while that said, ‘It’s fantasy, stupid.’ Because I kept bringing everything back down to physics and logistics and you don’t need to do that because it’s supported by magic, you know. It took a long time to get rid of that – it’s so ingrained.”

Draco Malfoy Credit: Jim Kay/Bloomsbury

Kay says that behind every final drawing, there are several attempts. “I was drawing a hand for book two,” he says, “and I realised I’d done 14, 15 attempts at this hand.” And even when he’s happy with the drawing, he adds layers of colour and detail in separate sheets.

“It’s partly because I had a terrible crash of confidence while I was doing it,” he says. “Once I started drawing something, I was convinced I was going to ruin it, so I’d go on to a separate sheet, and another sheet, and another sheet… So I’d have 10 different bits where ordinarily you should have this nice finished painting. But I got really bad shakes when I was doing this. I think because of the size of the project. It’s the first time I’ve worked on a project where everyone knows what the story is. Ordinarily you’re working on something that’s unpublished. And the enormity of it… I had to change the way I work, because I couldn’t paint the way I used to. When I get my confidence back, I’ll start painting in one go and it’ll be much quicker. But it’s shot at the moment.”

Telegraph logo This video content is no longer available To watch The Telegraph's latest video content please visit youtube.com/telegraph

Some of Kay’s work can be quite fraught, either through intricacy or density. That’s part of what gives it its strength. I suggest to him that if you tried to translate the temperament in the illustrations to the person behind them, you might find a lot of energy, or a lot of worry.

He laughs, shyly. “Well, I was diagnosed with various… problems, yes… so that might be a reason for it,” he says, before censoring himself.

But this density of the drawings, I persist: is it a symptom of, or a solution to, whatever’s in his head?

“It would be therapy if it went right more often,” he says sombrely. “But it doesn’t, so it can actually be the catalyst for it being a difficult day. It’s quite antagonistic.” He adds that his old university lecturer used to advise him to try things outside his comfort zone.

He tends to do that anyway, but the Harry Potter project has taken it one step further. “This is like wearing nettle pants standing on a prickly mat,” he says.

A panel from 'A Monster Calls', 2011 Credit: Jim Kay

Kay was born in a small town in Derbyshire in 1974. He is one of four children, whose father worked in insurance. His grandfather loved to draw – a sketch of the Pink Panther drawn by the six-year-old Kay at his grandfather’s side is pinned to the wall of his studio. But what he mostly remembers, when asked about his background as an illustrator, is all the acting he and his siblings were encouraged to do as children.

“Because we were studying drama we were exposed to a huge variety of literature,” he says. Stories, more than pictures, were planted in his head – and it’s his imagination rather than his technical skill that makes him an illustrator. “People confuse the craft with the art,” he says. “I’ve been to schools where children at a very young age discount the idea that they could be an artist or an illustrator because at that age they can’t draw particularly well. And yet their ideas are fantastic.”

Kay did, however, draw wildlife constantly. He describes David Attenborough as “a third parent”, and still now will have Attenborough’s documentaries playing in the background while he’s working in the studio.

As a child, he’d cover huge sheets of paper in detailed coral reefs: “I’d start in one corner really small. I realised the smaller I drew the more I could fit on the paper. My friend used to draw alongside me. We’d cover these sheets of paper with intricate things.” Even now, he says, “landscapes are like self-portraits, for me. There’s more of who I am in drawing trees and the natural world than there is in drawing people.”

Ron Weasley Credit: Jim Kay/Bloomsbury

Although he studied illustration at university, and worked as an illustrator for a brief period (his first job was for the now defunct Soroptimist Weekly), Kay struggled to make a living. So he gave up, and drew nothing at all for 10 years. “It’s my great regret,” he says. Instead, he took jobs stocktaking carpets, packing calendars, filing medical records in a hospital. That led to a job in the archive at the Tate gallery, where he looked after the papers of British artists. He realised he’d always read about artists through the words of others, and this was the first time he was formulating opinions first-hand. “I remember going through Stanley Spencer’s correspondence,” he says. “Beautiful letters. You get this insight into artists’ lives.” From there, he was hired as a curator of the illustrations collection at the Royal Botanic Gardens at Kew. And although this suited him, given his long-standing love of botany, none of the curatorial or archival work involved drawing. One day, he was talking to a friend, criticising some artist or other, and his friend said: “Well, at least they’re trying.” He realised she was right.

Shortly after that, the director of the Riverside Gallery in Richmond, who had seen some of his old work, offered Kay an exhibition. Though he hadn’t picked up a pencil in a decade, Kay said yes. “Don’t ever try and fill two rooms of a gallery when you’re living in a tiny flat in Twickenham and working full-time,” he advises, grinning. “I had to frame everything myself because I had no money – I made the frames, cut the glass… It was a nightmare.” He showed ominous collotype illustrations (a raven, a crocodile) for a story that existed only in his mind, and built strange models and automata. A worm embedded in a giant book popped out at visitors in response to a motion sensor. It turned out to be the most highly attended show in that gallery’s history.

Thanks to his partner, Louise Clark, whom he met 13 years ago when she was a librarian at the Tate, he was able to make the leap back into illustration. “The only way I could start was because my partner supported me,” he says. That first year, he worked 360 days out of 365, yet when he filed his tax return he realised that his profits from illustration came to less than £1 per day. While he illustrated A Monster Calls, he took a part-time job at Paperchase in Edinburgh. “Apparently JK Rowling used to go into that Paperchase,” he says, though he never saw her.

Clark, who is as serene as Kay is wired, has made us an orange and chocolate madeira cake. She now works at home as well, making hats, among other things, in a small, neat study upstairs. She helps Kay too, and in their home it’s hard to delineate where his imagination ends and hers begins. There’s something about the house that seems not so much decorated as narrated, each object pulled from a story that doesn’t actually exist. (Kay once showed some sketches to a publisher, who lighted upon his cat and rat characters Scribble and Smudge and said: “I bet there’s a funny story behind those two.” There wasn’t.)

Hogwarts Credit: Jim Kay/Bloomsbury

Clark is also the model for Professor McGonagall in the new book. Kay had to age her artificially – her real hair is ringleted and blonde rather than a harsh grey – but it was helpful to have her as a basis. “We’re quite isolated here,” Kay says. “I use mirrors a lot. I don’t know any children. This book forced me to interact with people. I needed young models that I could refer to over seven years.”

By the time he came to illustrate the books, he’d forgotten how he’d imagined the characters when he first read The Philosopher’s Stone, years ago. The film actors had supplanted them in his mind, but once he went back to the books, he found that he could make his own interpretation separate, though there are enough resonances not to upset fans.

He looked for real-life models wherever he could. Hagrid is, in part, an old biker who lives in Kettering, with the eyes of Winston Churchill. He was the first character Kay drew, and he set the viewpoint for the rest. “I see a lot of children’s books where the eye level is set at an adult’s,” Kay explains. “Which I find odd, because children see the world from a lower perspective. It’s nice drawing giants because it reminds you of being a child again. The illustration of Hagrid is that perspective, looking up.”

Mr Dursley is based on the local butcher. Hermione is based on Kay’s niece. Harry was a boy he spotted swinging from the bars on the London Underground, and Kay had two stops to introduce himself to the boy’s mother and persuade him to pose for him – for a book whose subject he could not yet disclose. Draco Malfoy and Ron Weasley both came from a school in Burford, to which he’d been invited by a pupil who’d loved A Monster Calls.

Hagrid flies through the sky Credit: Jim Kay/Bloomsbury

“I’ll be honest,” he says, “it’s the first time I’ve properly looked at people. I’ve never appreciated how varied we all are before.” He looked round the school, and round another one he visited in Corby, and was astonished by the children. “I mean, even at the same age there can be two or three feet difference. I just wanted to draw them all the time.”

He couldn’t draw them all the time, though. He couldn’t even get any of them to sit for long enough to draw them from life – that, he reasoned, was too much to ask of children. So he took photographs of them, in the positions he needed according to his preparatory sketches, and altered their features to fit Rowling’s descriptions. It wasn’t easy. “Children are mercilessly unforgiving to draw,” he says. “If you put a single line wrong on a child, you age them by about five or 10 years. One under the eye and they look like an adult.”

Tracking the way in which the characters age over seven years is not something he wants to risk faking – already, in six months, the slight boy who became Malfoy has developed an athletic build and grown taller than Kay. But when he met them, Kay couldn’t tell the children what the project was, so there was no way to ask them to commit to a seven-year modelling arrangement. Now they know.

Listening to Kay speak, you might think some uncertainty would be traceable in his work. It isn’t. The result is fearless, exuberant and focused. But that won’t prevent him from feeling anxious. “I can’t tell you how worried I am,” he says. “It’s so public, Harry Potter.” But then, he thinks, if it doesn’t work this time, “we’ve got six more books to get it right”.

Harry Potter and the Philosopher’s Stone: the Illustrated Edition is published by Bloomsbury on October 6, priced £30. To order your copy for £24 with free p&p call 0844 871 1514 or visit books.telegraph.co.uk