Not long ago, an associate of Matt Smith’s – “a man of prominence in the film world,” the actor says gnomically – was at a dinner hosted by the Queen and Prince Philip. There was just a handful of guests and he was sitting next to the Duke of Edinburgh. “What do you do?” asked Philip, as the first course arrived. “Are you involved in this… Crown thing?”

The Duke was referring to the Netflix drama created by Peter Morgan that gives us supposed fly-on-the-wall access to Buckingham Palace and the longest-running soap opera in British public life. The first season of 10 episodes, released last November, covered from 1947 to 1956; it introduced us to a young Elizabeth (played by Claire Foy) and followed her as she married Philip Mountbatten (Smith) and became our monarch. Like Morgan’s other work – notably the 2006 film The Queen and The Audience, a 2013 stage play – The Crown skilfully skips along a line between salacious and sympathetic, revealing but also respectful. Morgan always reiterates that his work is free from external interference, but has also said that the royal family is “very, very aware” of the series.

I grew up surrounded by lads and I like the camaraderie, the way you take each other down. I think Philip did too

At the dinner, the man replied to Philip that no, he wasn’t involved in the show, and conversation moved on. But the idea of Liz and Phil sitting down for a Netflix night in with the corgis warming their feet was too irresistible. So, as the evening wound down, he said: “Philip, I’m just wondering, because I have some friends who made The Crown, have you watched any?” The Duke stopped and glowered: “Don’t. Be. Ridiculous.”

Smith explodes in laughter as he tells the story; he pronounces Philip’s riposte with an impeccable haughtiness, so posh that it sounds like he’s being slowly throttled. “Whether it’s true or not, I don’t know, but I just think he’s a bit of a cool cat,” Smith goes on. “And that’s what I love about him: he’s done what he wants, when he wants, how he wants, with whom he wants. He hasn’t asked permission. And his wife’s the Queen.”

“This Crown thing” has caused a bit of a stir. Much of the attention, initially, was on its lavish budgets: most estimates said $100m for 10 programmes, which would make it the most expensive TV show in history. This being Netflix, they didn’t need to confirm or deny, but Morgan has suggested it was closer to $130m for the first two series, so nearly $7m an episode. Initially, the suspicion was that The Crown would be catnip for anglophile American audiences; in the event, its meticulous research and attention to detail have given it considerable traction over here, too. Jennie Bond, for many years the BBC’s royal correspondent, confessed that she binge-watched the 10 episodes in a day and had to remind herself it was not a documentary.

The first season of The Crown concentrated on Elizabeth, a performance that earned the immaculate Foy a Golden Globe, with scene-stealing turns from John Lithgow as Winston Churchill and Vanessa Kirby as Princess Margaret. With its return, which tracks the House of Windsor from the Suez crisis in 1956 to the Profumo affair in 1963, Philip comes to the fore. For the modern audience, the Duke of Edinburgh tends to be viewed as a politically incorrect liability: “HRH Victor Meldrew” as the historian David Starkey put it. In The Crown, he is a castrated alpha male; an irascible silverback who has been stuck in a zoo and prowls his cage, furious.

“Rightly, as a society, we’ve celebrated Elizabeth as a wonderful example of a powerful, stylish, brilliant woman,” says Smith, over a cup of tea in a photographic studio in east London. “But in many ways, what an example of a roguish, brilliant man. Why aren’t we as men allowed to celebrate that, fictionally or not? And I just found a lot to celebrate in Philip.”

At a glance, the 35-year-old Smith would not seem to have much in common with the Duke of Edinburgh. Today he is dressed all in black and navy save for a grey Whovian scarf made by Louis Vuitton and pink hearts on his socks (“That’s our one flourish a day, see. Women get underwear, we get socks”). He was born in Northampton and went to the local state school; his father ran a plastics business. “Yeah, I’m not of that class, nor is Claire,” he says. “We’re lowly middle-classers.”

There’s not much of a physical resemblance to Philip, either. Smith is very obviously handsome, but there’s nothing fine-featured about him. With his craggy brow and jutting jaw, there’s a hint of that Philip Larkin description of Ted Hughes: “a Christmas present from Easter Island”. “I’m not great casting for Philip really,” Smith accepts, “but actually when I got into it, there’s an interesting synergy here.”

One area of crossover was a love of sport, and the male companionship that often comes with it. For Philip, it was mainly polo, sailing and carriage driving, in which he competed for Britain and which was a great passion into his 80s; for Smith, it was football. As a teenager, he played central defence in the youth teams for Northampton Town, Nottingham Forest and Leicester City. Then, aged 16, he had a back injury and Leicester released him. Smith has scarcely kicked a ball since, and it’s clear that he misses not just that, but the sense of belonging he felt in the dressing room.

“I grew up surrounded by lads and I like that culture,” says Smith. “I like the camaraderie and the way you can take each other down, and that still exists between me and my mates. We get together and we tear each other to bits and we laugh. And in the tearing each other to bits, there’s something about it that entertains me, I suppose. But also, at the end of that, you have a sporting endeavour or whatever it is together and I like that sense of a team. I think Philip liked all that, too.”

Facebook Twitter Pinterest Left: Matt Smith with Claire Foy as the Duke of Edinburgh and the Queen in The Crown, season one. Photograph: Alex Bailey/Netflix

Smith realises that, in the age of Harvey Weinstein and Donald Trump, these are precarious waters to dip a toe in. “Maleness,” he notes sadly, “has been perverted and misconstrued.” It is also complex with regard to The Crown. The first season touched on the emasculation that Philip felt when Elizabeth became Queen, many years earlier than they both expected, and his discomfort at doing her bidding; he once complained that he was nothing but “a bloody amoeba” and that he was the only man in Britain who wasn’t allowed to pass down his name to his own children.

Season two of The Crown returns to this theme with relish. The new series opens with Elizabeth informing Philip that she would like him to – that is, he has to – take a five-month solo tour of the outer reaches of the Commonwealth aboard the Royal Yacht Britannia. He would open the 1956 Olympics in Melbourne and drop in on exotic locales such as Papua New Guinea and the Antarctic. For company, he would have Mike Parker, his private secretary, and 240 men and officers from the Royal Navy. The trip has gone down in legend: “It’s not a royal tour,” snipes Princess Margaret in The Crown, “it’s a five-month stag night.” There are resilient rumours that Philip had numerous affairs around this time.

Smith again has a measure of sympathy for the Duke of Edinburgh. When he retired from royal duties in August, he had been dispatched on 22,219 solo engagements since 1952. “If you strip away the royal family-ness of it, it’s two human beings,” he says. “I challenge anyone if their partner said, ‘By the way, you’re going off for five months,’ to not go, ‘Whoa, hang on!’ And I defy anyone to be married as long as they have and it be plain sailing the whole way. As human beings, they’ve struggled.”

Here’s another parallel between Prince Philip and Smith: both have lived their early adulthood in the public glare, and both seem to have been deeply conflicted about the lack of privacy. The scrutiny of the press is certainly not a modern invention. Peter Morgan is insistent that Princess Margaret received more attention from the paparazzi when she was seeing Peter Townsend and others in the 50s and 60s than Diana ever did. The main difference was that the film photographs were slower to process and the coverage was, in the main, more respectful.

For Smith, the day his life changed for ever came in early January 2009, when he was unveiled as the 11th Doctor, and aged 26, the youngest actor to play the 1,000-year-old time traveller. It wasn’t just that he was a long shot to replace the very popular David Tennant; he didn’t feature on anyone’s list outside the BBC. More than one publication announced the news with the headline: “Doctor Who?”

The response was immediate, and it was especially confounding for Smith, who was far from a rabid fan of the show. “No, I’d never seen it,” he says. “Nothing can prepare you. Nothing. And literally overnight. As soon as it’s announced, five reporters turn up to your mum and dad’s house, with cameras. Overnight. They then go to your grandad’s house. They then approach every one of your best friends. It was that intrusive.

Facebook Twitter Pinterest Photograph: Phil Fisk/The Observer

“Yeah, I was 26 and I didn’t claim to be a saint,” Smith continues. “I was a 26-year-old lad, and I thought about saying no. And my agent quickly dispelled that idea. But I did, I genuinely thought, ‘Is this where I want to take myself?’”

Smith’s casting was a success, both with hardcore fans and in the ratings, and he is happy with his decision, broadly speaking. Doctor Who was demanding hours – “Undoubtedly the greatest workload you will ever take on, bar none” – but it provided an unbeatable education in leading a show. Plus, the role is a gift for any actor: there’s a reason why no recurring part, even James Bond, attracts such frenzied speculation when it is up for regeneration.

“The part’s sooo good,” says Smith. “He can do anything. He can go from A to Z. Like any other character you play, Prince Philip for instance, you have go through at least D, F, G, P, before you get to Z. But the Doctor goes A to Z to B to Y, flips it up, and you don’t have to explain that because he’s an alien who’s 1,000 years old: he’s seen more than you have, he’s done more than you’d do. He’s seen so much tragedy that it’s made him really funny.”

When Jodie Whittaker was announced earlier this year as the 13th Doctor, Smith rang her up and sang the theme tune to her voicemail. Does he have any more useful advice in advance of her Christmas debut? “Oh, I just think she’ll have a wonderful time,” he says. “But the shift in your life is extraordinary, because it crosses generations. So when she goes to a wedding, she’s not going to the wedding any more as Jodie Whittaker, she’s going as the Doctor. When she goes to a funeral, she’s not going to a funeral as Jodie Whittaker, she’s going to a funeral as the Doctor. At her local cafe, they’ll talk to you about that. Because people can’t help but attach that to you; something changes in the perception of everyone else around you.”

Smith, who stepped down from Doctor Who after the 2013 Christmas special, feared that he would never outgrow the role. Or, worse, that he would be defined by the celebrity it imparted. In the event, he is making a decent stab at writing an alternative script. Right after he wrapped Doctor Who, he signed up for a musical version of American Psycho, directed by Rupert Goold at the Almeida theatre. His Patrick Bateman was required to appear onstage in just “tighty-whities” and an eye mask and, more terrifyingly, had to belt out 80s pop. Next year, he will appear in the lead role in a biopic of the photographer Robert Mapplethorpe, whose work on the underground BDSM scene in New York in the 60s and 70s caused national outrage in America.

“I did Mapplethorpe,” Smith explains, “as an endeavour of, well a) he’s a fascinating character, and b) there’s something about his photos that is just… who knew, as a straight man, that pictures of penises were so utterly compelling? But they are! And you don’t know why.

Facebook Twitter Pinterest ‘The part’s sooo good’: Matt Smith with Karen Gillan in Doctor Who in 2012. Photograph: Adrian Rogers/BBC

“I like to make myself feel uncomfortable. That’s why I did a musical. That’s why I did Mapplethorpe, because there was an unknown entity to those projects. David Bowie said to the guy in LCD Soundsystem: ‘Go and make yourself feel uncomfortable again.’ And it’s really important.”

Off-screen, Smith still remains a Sidebar-of-Shame standby. There’s not much to report, really: for a couple of years, he went out with the model Daisy Lowe; since 2014, he’s been dating the Downton Abbey actor Lily James. Smith’s policy is to give interested onlookers as little to go on as possible. “I’m not on Facebook, not on Twitter, not on Instagram,” he says. “I just don’t really care. I don’t care where you are. I don’t care what you’re doing. I don’t really expect you to be interested in where I am or what I’m doing. Also, I don’t really want you to know where I am or what I’m doing. That is probably a reaction to Doctor Who if I’m honest, in some way. But it’s one that I’m quite pleased that I had.”

In the same way that many of us fail to interrogate our parents about their lives (before we came along), The Crown proves how little we really know about the royal family. This is Peter Morgan’s great skill. He takes a subject that feels as familiar as your bedroom curtains, and shows that you’ve really not been paying attention for years.

Where Prince Philip is concerned, there are positive and negative aspects to this reappraisal. In many ways, he is the most likable of the older generation of the royal family; he’s certainly the least stuffy. He’s fond of slapstick and is said to call the Queen “Cabbage”. “When you investigate him, he’s very bright, he’s funny, he’s hugely popular in the royal house,” says Smith. “All the research I did with people who have worked there, he was the guy they all really loved.”

On the other side, there is what looks like damning evidence that he had a wandering eye. Just as the Queen and the Duke are celebrating their 70th wedding anniversary, both The Crown and a new biography suggest that he was serially unfaithful, especially in the early years of the marriage: some of the women were well known, others he met through the raucous Thursday Club, a gentlemen’s lunch society that met weekly in Soho (regulars included David Niven, the Kray brothers and Stephen Ward, the osteopath at the centre of the Profumo affair).

If the first series of The Crown felt broadly compassionate to the royal family, even humanised them in many eyes, the second definitely has a sharper edge. If the Queen and the Duke do ever tune in, there might be some awkward moments. “It’s quite tough on them,” agrees Smith. “But we can’t be reverential. I was always quite conscious of that. With any biopic, you’ve got to show the person at their ugliest, because otherwise, what’s the point really?”

As for the affairs, Smith goes momentarily coy. “Did he or didn’t he?” he muses. “As an actor, I made a choice on did he or didn’t he, but I don’t want to give that away. The facts are the facts, but is it ever explicit? Do I ever wave a flag about it? I will let you decide.”

Facebook Twitter Pinterest A scene from season one of The Crown. Photograph: Alex Bailey/Netflix

This is Smith’s final outing as the Prince; for the next series, he’ll be replaced by an older, as-yet-unnamed actor (Claire Foy’s role is being taken on by Olivia Colman). He will be sad to leave the world, in part because of the exotic locales The Crown films in: “We went to Africa and Scotland and Surrey. Totteridge.” But, perhaps after the experience of Doctor Who, he accepts that career-wise it is a good time to move on. Besides, Smith has been realising that he was finding it hard to leave Prince Philip on set, which isn’t easy when you don’t live in Buckingham Palace. “You expect your toothpaste on your toothbrush in the morning,” he sighs.

How’s that going down at home? “Yeah, there’s none of that,” he replies. “Although the last couple of days my girlfriend has brought me a cup of tea in bed because I’ve been poorly, which has just been triumphant! Small victories.”

Smith, though, is excited about what comes next. He says, “I think my 40s will be quite interesting as an actor.” Because? “Dunno. I’ve got a weird face. Charactery. I think if I keep getting better hopefully there could be some interesting parts out there.”

Netflix crowns Olivia Colman as next Queen Elizabeth II Read more

This is true: in person, Smith is fun, funny and a little bit eccentric; his best screen performances – especially the Doctor and Prince Philip – transmit these raffish qualities. He would also like to direct again, having dabbled with a half-hour drama called Cargese for the Playhouse Presents strand on Sky Arts in 2013. And, at some point, he wants children. “Yeah, definitely, one day, 100%,” he says. “I’m really close to my family and if I can have the relationship with my son that I’ve had with my dad, that’s what life is about ultimately.”

Son? He sounds like he might have been hanging around the royal family too long – an heir and a spare and all that. “Son,” he confirms, laughing, before adopting a Henry VIII-ian bellow. “I’ll have a son and if not, I will chop off her head. Give me a son, dammit!”

What does Smith specifically want to do with a son? “Everything,” he replies. “I want to watch football with him. I want him to be my son, I want him to be my boy. That said, either way it’s a blessing.”

Then, momentarily, he becomes serious. “What’s happy? Am I content? Probably not. There has to be more to life than there is at 35 for me. Everything: look, I’m lucky I have a great family and great friends. Touch wood, that is the most important thing but I’m definitely not content with whatever position that I’m in. I want to keep going.”

Season two of The Crown begins on Netflix on 8 December