If the 15-year-old Rory Stewart could see himself today at 40, "he would think I was a bit pathetic". He would see at once "all the ways in which I've compromised, and sold out. And he would be absolutely right." What would he have made of his decision to be a Tory MP? "Really confused, I think," Stewart smiles. "Yes. Really, really confused."

A lot of other people have been, too. Stewart is a Scot born in Hong Kong, raised in Malaysia and educated at Eton, who studied PPE at Oxford while tutoring Princes William and Harry in his spare time. On graduating he joined the foreign office, posted first to Indonesia to help sort out East Timor, and then to Montenegro to deal with Kosovo. Between 2000 and 2002 he walked 6,000 miles through Afghanistan, Pakistan and Iran, staying in villagers' houses, before being dispatched to Iraq to take charge of two provinces and to help write the country's new constitution. He wrote two bestselling memoirs about his experiences in Afghanistan and Iraq, Harvard made him a professor, and he founded a charity in Afghanistan at the request of its president and the Prince of Wales.

By 35 he had led so many adventures that Brad Pitt's production company bought the rights to a biopic of his life. And then he came home to become the Tory member for Penrith.

We met once before, about 10 years ago, and he struck me then as a character from another century, or possibly a Flashman novel. He remains hugely appealing: self-deprecating, funny, open, curious and kind. So what was he thinking to give up his former life for the tedium of the backbenches? His explanation turns out to be the most convincing analysis of foreign interventions and domestic politics I can remember ever hearing from a Conservative MP.

Stewart came home when he realised that even the least-educated Afghan housewife in a mountain village knew more about the country than he did. Fluent in Dari, along with nine other languages, he'd thrown himself into the coalition mission with great conviction, but had to conclude that: "In the end, the basic problem is very, very simple. Why don't these interventions work? Because we are foreigners. If things are going wrong in a country, it's not usually that we don't have enough foreigners. It's usually that we have too many."

Ten years ago he would have listed 10 things Afghanistan needed to build a new state: rule of law, financial administration, civil administration and so on. "And, then you would say, well, how do you do that? Well, I'd say, by a mapping of internal and external stakeholders, definition of critical tasks – all this jargon talk. And I've only now just begun to realise these words are nonsense words. I mean, they have no content at all. We should be ashamed to even use them."

They are nothing more, Stewart now acknowledges, than tautologies. "They pretend to be a plan, but they're actually just a description of an absence. Saying 'What we need is security, and what we need to do is eliminate corruption' is just another way of saying: 'It's really dangerous and corrupt.' None of that actually tells you how it's done."

Stewart is one of those rare people who talk in perfect sentences. He goes on: "Our entire conceptual framework was mad. All these theories – counterinsurgency warfare, state building – were actually complete abstract madness. They were like very weird religious systems, because they always break down into three principles, 10 functions, seven this or that. So they're reminiscent of Buddhists who say: 'These are the four paths', or of Christians who say: 'These are the seven deadly sins.' They're sort of theologies, essentially, made by people like Buddhist monks in the eighth century – people who have a fundamental faith, which is probably, in the end, itself completely delusional."

Rory Stewart in Iraq in 2004 during his time as deputy governorate co-ordinator of Maysan province. Photograph: Dan Chung/The Guardian

Whenever Stewart took one of these ideas, such as rule of law, to an actual Afghan village, it became meaningless. "None of the things that I'm looking for exist. There obviously isn't police, or a judge, there isn't a legal code, there isn't a prison. There's a bunch of guys with white beards sitting around, and their system of doing that might be quite different from the next-door village. So then how do you get from there to here? Well, it can be done, but it's not going to be done by a foreigner who barely understands any of that."

But if he were an MP in his own country, he figured he'd at least understand what he was doing. What he hadn't anticipated was the conclusion he would reach after four years in Westminster. "I think British democracy at the moment is really struggling to work."

Part of the problem is the unprecedented nature of the problems facing us today. "You have to ask yourself what a country that was the first to industrialise, and the first to de-industrialise, does with itself. What is our civilisation? What is our democracy? Who do we want to be?" But almost no one else in parliament appears at all interested in these questions. His colleagues tease him, telling him they're for thinktanks, not politicians, and that "people will think you're a sort of nutty professor".

Westminster works much better for career politicians, Stewart soon found, than for a newcomer with intimate experience of the world it legislates on. "It's such a weird profession, with such specialised rules and such a strange anthropology, that people who've been in it for a long time have a huge advantage. This is just such an eccentric institution that it's difficult for an outsider to really understand what we're doing." Does he? "No, not at all. Not at all, because a lot of what we do day-to-day is very difficult to make sense of."

When the house first sat after he was elected, he remembers everyone waving their order papers and jeering and cheering, and he said to Chuka Umunna, a newly elected Labour MP: "We're not going to get pulled into that, are we?" It seemed plausible that the new intake could rewrite the rules. And yet MPs' obsession with "who's up and who's down" is contagious. "You can't spend three and a half years in here without being changed profoundly. I mean, I'm not who I was when I first came here."

Cumbria is a long way from SW1, which may help to explain why Stewart is convinced that a radical new localism is the only way to revive democracy. "We have to create a thousand little city states, and give the power right down to all the bright, energetic people everywhere who just feel superfluous." A huge fan of the Big Society, he calls it "the fundamental insight" and "the big idea", and valiantly maintains that it has not been ditched by Downing Street. He also believes what we need now is a brand new written constitution.

"In some sense I'm a romantic. I like the idea of organic history and tradition. But I think Britain is such a different place now, and changing so quickly, that I'm coming slowly, painfully, to accept that we need to start again."

Stewart would separate the legislature from the executive, slash the number of MPs from 650 to 100, introduce powerful locally elected mayors, and impose greater transparency and controls on the security services. He thinks the US public have been much more upset than the British by Edward Snowden's revelations because of the cultural legacy of the first amendment, and he would introduce something similar here.

Rory Stewart in Kabul, Afghanistan in 2009. Photograph: Jason P Howe

"We've relied for 400 years on an informal faith in our own common sense and sanity and Britishness, and thought that would all be all right. But those things are very fragile in a new world, and so you need to begin to write things down." But of course, all this is a tall order. "This is where the gap between my theoretical desire and practical politics comes in," he chuckles. "How do you get 650 people to vote to lose 550?"

It strikes me that being a backbench MP in Cumbria is probably the least powerful job Stewart has ever done. He laughs. "I like that, yes, that's true. Anybody running a small pizza business has more power than me. I mean, in four years, what have I done?" He says he has "maybe influenced, in a small way" the troop withdrawal from Afghanistan; led a Commons motion obliging mobile phone operators to increase rural coverage from 87% to 98%; had some minor influence over the technical details of the way rural broadband is rolled out; "might" have changed the way the foreign office analyses language skills when it runs its promotions boards; and saved the local cinema. "And I might, if I'm lucky, have got a lift at the train station in Penrith. But that's about it."

In a way, he says, ordinary Afghans are far more powerful than British citizens, because at least they feel they can have a role in one of the country's 20,000 villages. "But in our situation we're all powerless. I mean, we pretend we're run by people. We're not run by anybody. The secret of modern Britain is there is no power anywhere." Some commentators, he says, think we're run by an oligarchy. "But we're not. I mean, nobody can see power in Britain. The politicians think journalists have power. The journalists know they don't have any. Then they think the bankers have power. The bankers know they don't have any. None of them have any power."

And this from a man who only two years ago attended the Bilderberg conference, a highly exclusive and secretive gathering of the world's most powerful bankers, politicians and businesspeople?

"Well there we are, you see," he smiles. "I can tell you, there is nothing there. It's like the wizard of Oz. This is the age of the wizard of Oz, you know. In the end you get behind the curtain and you finally meet the wizard – and there's this tiny, frightened figure. I think every prime minister has sort of said this since Blair. You get there and you pull the lever, and nothing happens."

But that doesn't mean he thinks he's wrong to be an MP, and he doesn't for a minute regret it. In fact he is remarkably cheerful, plans to do the job for at least a decade, and hopes for a ministerial post. "I'm not depressed or disillusioned – I want to be here to see if I can change it. I'm desperate to try to use my life to engage with the spirit of the age, and in the end the thing I grumble about – powerlessness – is the essence of the spirit of the age. So the thing I'd be really proud of would be to change the British constitution in a way that unlocked all that untapped energy in this country."

He caused a bit of a stir recently with an article arguing that children have become "the opium of the masses", worshiped to the exclusion of all others. Stewart is still childless, having recently married an American he met through his Afghan charity. Does he think he might change his mind? "As every one of my friends who complains about kids behaves exactly the same as everybody else as soon as they have kids, I presume I'm going to be the same," he grins. What will he make of his article when that happens? "I anticipate thinking that I was probably right, and I could see more clearly before – just as I think I was probably more right about parliament before I was in parliament, and just as I think that when I was 15, and was very judgemental of 40-year-olds, I was right."

As we're saying goodbye I ask what became of the biopic. Rumour had it that Orlando Bloom was lined up to play the part of Stewart. "I think," he says, "becoming a Tory here didn't help." Did that spell curtains for the project? He bursts out laughing. "Yes, I think it's just a phenomenally bad end to a film."