When Prince Charles becomes king, will he be able to stop his compulsive ‘meddling’? And if he can’t, what will it mean for the monarchy and the United Kingdom?

On 15 September, while President Obama was meeting with his advisers in the White House and deciding how to unleash the world’s most powerful military machine on the Islamic State in Iraq, his ambassador to Britain, Matthew Barzun, was spending the day in a field in Gloucestershire, learning about nitrogen-fixing plants and the dangers of sub-clinical mastitis in cows’ udders. The reason was simple: Barzun was visiting Prince Charles’s organic Home Farm. Wearing boxfresh Hunter wellies, Barzun picked his way around some cowpats to take a close look at a field of organic red clover. He snapped a photo on his smartphone.

For the past 34 years, the farm has been one of Charles’s chief passions. It has become the agricultural embodiment of his beliefs about everything from the natural world to the globalised economy. On winter weekends, he can be found – wearing his patched-up tweed farm coat – laying some of the farm’s hedges to keep alive one of his beloved traditional farming techniques. (Charles is such an enthusiast that he hosted the National Hedgelaying Championships here in 2005.) The farm closely reflects Charles’s likes and dislikes. In one field, there is a herd of Ayrshire cattle. Charles bought them after he declared that he didn’t want yet more common “black and whites”.

That morning, the ambassador was not the only influential figure invited for a private tour of the royal farm. Alongside Barzun was Professor Ian Boyd, the chief scientific adviser at the Department for Environment, Food and Rural Affairs (Defra), George Ferguson, the elected mayor of Bristol, and Sir Alan Parker, the chairman of Brunswick, the public relations company that advises Tesco. They were accompanied by civil servants from Defra and the Department for Business, Innovation and Skills, and shown round by the Prince’s friend, Patrick Holden, an organic agriculture campaigner, and Charles’s farm manager, David Wilson.

The day was organised by Holden’s Sustainable Food Trust, but the talking points faithfully echoed Charles’s view that industrialised agriculture is a big, dangerous experiment with our environment and a threat to the livelihoods of small farmers. Here was a branch of Prince Charles’s power network in action. Away from the public glare, issues that matter intensely to him were being discussed in front of some of the most powerful people in Britain. In an echo of his famous comment of 1986 that he talks to his plants – he joked more recently he actually “instructs them” – there was even a brief exchange on whether oak trees communicate with their relatives through the soil. Holden and Wilson raised a few eyebrows with some of their scientific claims, not least about the danger of antibiotics in meat. On the whole, though, the guests seemed receptive.

Facebook Twitter Pinterest Prince Charles at Chester Cathedral. Photograph: David Levene

Over the past four decades, Charles has carved out a unique position for himself as an elite activist, tirelessly lobbying and campaigning to promote his concerns. From farming to architecture, medicine to the environment, his opinions, warnings and grumbles are always heard. He spreads his ideas through his writings and speeches, his charities and allies and, behind the scenes, in private meetings and correspondence with government ministers. His interventions matter. Peter Hain, the former cabinet minister who lobbied with Charles for NHS trials of complementary medicine, summed up his influence in this way: “He could get a hearing where all the noble, diligent lobbying of the various different associations in the complementary medicine field found it hard.”

Letters, written in black inky scrawl, are a key part of his lobbying arsenal. His carriage on the royal train is fitted with a desk, blotting paper and a stash of red-crested HRH notepaper on which he scrawls his “black spider memos”. He writes the memos whenever he can – late at night, after dinner guests have left, even at 35,000ft on the royal jet. “I have travelled with him and within five minutes of takeoff, he is doing his letters,” said Patrick Holden. “[At home] he goes back to his desk after dinner. How many of us do that?” Sometimes, the late-night letter writing is so exhausting that the prince, who turned 66 last week, is found asleep at his desk.

It is a habit that has put him in a precarious position. On 24 and 25 November, the supreme court will be asked to decide whether Charles’s letters to ministers should remain private. This may be the final chapter of a nine-year legal battle between the Guardian and the government over freedom of information laws. In 2005, this newspaper asked to see letters Charles had written to ministers in 2004 and 2005. The government refused but revealed that Charles had sent 27 letters to several departments over eight months. In October 2012 Dominic Grieve, then attorney general, again vetoed release of the letters, arguing that the public might conclude Charles had been “disagreeing with government policy”, which “would be seriously damaging to his role as future monarch because, if he forfeits his position of political neutrality as heir to the throne, he cannot easily recover it when he is king”.

A former high-ranking government official, who is experienced in handling the prince’s interaction with ministers, described the risk to Charles’s kingship posed by publication as “quite large”. There are, he said, “quite a lot of letters and they say some things that are quite zany”.

Facebook Twitter Pinterest What are the ‘black spider memos’?

One letter from February 2002, which was leaked to the Daily Mail, revealed Charles’s strident approach. Writing to Lord Irvine, then lord chancellor in the Labour cabinet, he rubbished the Human Rights Act, suggesting that it was “only about the rights of individuals (I am unable to find a list of social responsibilities attached to it) and this betrays a fundamental distortion in social and legal thinking”. In another letter to Lord Irvine, written in June 2001, he expressed his worry that the act “will only encourage people to take up causes which will make the pursuit of a sane, civilised and ordered existence ever more difficult”, and added that “I, and countless others, dread the very real and growing prospect of an American-style personal injury culture becoming ever more prevalent in this country.”

Friends admire him as intelligent and conscientious but, perhaps unfairly, the death of the Queen is a day many dread

Even if the letters remain private, many are concerned at the prospect of Charles continuing his activism as king. His record suggests that he will find it hard to abandon his campaigning approach. Charles has used the phrase “mobilising” to describe his activities; his critics call it “meddling”. They view his involvement in political matters as an abuse of the unspoken understanding that the royal family should merely symbolise power, not wield it. Charles has waited longer to succeed than any previous heir to the throne. As his wait reaches its final stages, the question of how he will channel his political instincts when he finally becomes king is becoming a matter of debate in his court, in Whitehall and among his friends. Preparations are being made for a very different monarchy to that of Queen Elizabeth, who has secured acceptance of the constitutional monarchy in part through her strict silence on political affairs. Charles’s friends admire him as intelligent, caring and conscientious, but, perhaps unfairly, the death of the Queen is a day many dread.

“A quiet constitutional revolution is afoot,” his friend and biographer Jonathan Dimbleby said last year. “I predict that he will go well beyond what any previous constitutional monarch has ever essayed.”

* * *

Charles’s life, up to now, has been about how to carve meaning from his seven decades of waiting. “The defining question of his life has been, ‘What good can I make of this ill-defined role?’” said one well-placed source. But this project has also created a dilemma over how he should reign, once the wait is over.

Charles gained a place at Trinity College Cambridge in 1967 after passing two A-levels (B in History and C in French). He graduated with a 2:2 in archaeology, anthropology and history. By 1970 he had met Camilla Shand (later Parker-Bowles). Over the next decade he had several girlfriends and by the time he became engaged to Lady Diana Spencer in 1981, Camilla had married the cavalry officer, Andrew Parker-Bowles. The royal soap opera soon cranked up into a Hollywood blockbuster: the wedding at St Paul’s, the babies, infidelities on both sides, divorce, Diana’s shocking death in Paris, national mourning, Elton John at the funeral. The royal family’s reputation collapsed under an avalanche of negative press coverage.

I predict that he will go well beyond what any previous constitutional monarch has ever essayed Jonathan Dimbleby

All this time, Charles had been fashioning a parallel intellectual life, immersing himself in the world of ideas and spirituality. “I was born in 1948, right in the middle of the 20th century, which had dawned amid the gleaming Age of the Machine, the very engine of colossal change in the western world,” he wrote in his 2010 treatise Harmony: A New Way of Looking at Our World. “By the mid-1950s, a frenzy of change was sweeping the world in a wave of postwar modernism … By the 1960s the industrialised countries were well on the way to creating what many imagined would be a limitless Age of Convenience. Even as a teenager I felt deeply disturbed by what seemed to have become a dangerously short-sighted approach.” By the 70s, this feeling had hardened, and Charles began to speak out: “I could see very clearly that we were growing numb to the sacred presence that traditional societies feel very deeply.”

The ideas Charles set out in Harmony are dizzyingly eclectic, and, at times, verge on a kind of mysticism. He cites the “grammar” of Islamic art “that underpins the whole of life”, the “magical” rhythms of gardens and nature, the timelessness of Christian iconography and the symmetry of 16th-century German astronomy, Thomas Aquinas’s “eternal law”, the Vedic traditions of India, and Chinese Daoism. He is interested by the idea of “a duty to try and achieve an attunement between human nature and the greater scheme of the cosmos”.

“He is smarter than you think,” said one well-placed source. “He reads widely and deeply all the time at night, when he takes time off, when he travels. It is not so much books. He reads papers and is sent them all the time. If a new paper comes in from the University of Georgia on agriculture in the 21st century he’ll read it, understand it and send someone a note about it.”

In 1984 Charles launched a lifelong war on modern architecture by publicly criticising proposals for an extension to the National Gallery that he said was “like a monstrous carbuncle on the face of a much-loved friend”. The plans were dropped. Five years later, he wrote a loftily titled book, A Vision of Britain, in which he explained how the urban reconstruction of the 1960s prompted in him a belief that it was “crazy … to destroy so much of value and by the dictates of fashion throw the baby out with the bathwater”. He quoted GK Chesterton in his call to the quiet majority who, he felt, shared his views: “We are the people of England that never have spoken yet.”

From the mid-70s onwards, Charles began setting up charities – not as a patron like his mother, but as president, leading meetings and directing them from St James’s Palace under the auspices of his charitable foundation, which he launched in 1979. Promoting his beliefs in the fields of health, work, the environment and architecture, his network of organisations came to handle £100m a year in funds. Friends such as the hedge-fund billionaire, Michael Hintze, donated money and the government provided grants. Charles began appointing experts to advise him, such as the former Friends of the Earth leader, Tony Juniper.

St James’s Palace became a kind of grand salon for convening the powerful. Charles’s “rainforest summit” in 2009 – where he proposed schemes to limit deforestation and reduce climate change – attracted the then French president Nicolas Sarkozy, then US secretary of state Hillary Clinton and UN secretary-general Ban Ki-moon. Charles had established his court as a kind of thinktank, a model that may well follow him to Buckingham Palace. According to a source who has known Charles for many years, the charity network, which employs experts on architecture, regeneration, business and the environment, will be slimmed down but will remain “proactive and entrepreneurial”.

Some say Charles’s activism reflects not just his personality but the era he was born into. He will be the first British monarch to have gone to school – Gordonstoun in Scotland, which he later described as like “Colditz with kilts”. The Queen was brought up in an Edwardian household where the women withdrew after dinner so men could talk among themselves, whereas Prince Charles reached adulthood in the late 1960s, and seized the opportunity provided by his unique role to cultivate relationships with the powerful.

Charles has not spoken publicly about how he might approach kingship, and even privately, aides always imply it is a highly delicate issue because it involves talking about a matter of deepest family sadness – the death of his mother. Yet his desire to think of the long-term consequences of human actions, coupled with the ideas expressed in his writings and many public interventions, offer a clue. In 2010, he told an NBC news crew that he felt “born into this position for a purpose”. He explained: “I don’t want my grandchildren or yours to come along and say to me, ‘Why the hell didn’t you come and do something about this?’”

* * *

It sometimes seems that Charles is pushing against the limits of his position, testing what is possible for a constitutional monarch in the 21st century. It is an approach that has alarmed many onlookers. “The main difference [between Charles and his mother] is that the Queen is frightfully discreet about these things and will mention them in private meetings with the prime minister,” said a former senior government official. “Prince Charles is much more pushy and writes letters about his views which are on the edge of the mainstream. He pushes them hard and takes a risk. He is much more activist.”

Since the beginning of 2012, Prince Charles has held 27 meetings with government ministers. The agendas are kept secret by both the palace and Whitehall, but we know that he has previously lobbied ministers over NHS policy, foxhunting, farming policy, grammar schools and human rights laws. Clarence House says the meetings are essential to the Prince “understanding the workings of government, its departments and its senior members”. This knowledge will be necessary, it suggests, when he becomes king.

Over the years, Charles has adopted a number of like-minded politicians as allies. He got to know the former anti-apartheid campaigner, Peter Hain, after he became secretary of state for Wales in 2002. At the time, Hain used to send the prince short briefings twice a year to let him know what was happening in Wales. Later, he began meeting Charles on-on-one at Highgrove, his Gloucestershire mansion, and at his London palace, Clarence House. They discovered a shared enthusiasm for complementary medicine and forged an alliance to introduce it into the NHS. “We just got talking about it,” said Hain. “And, as a result, I tried to get the Welsh health minister to run a pilot where GPs could use certain recognised and established complementary therapists, whether osteopaths, chiropractors, nutritionists, homeopaths and acupuncturists.” The attempt failed, but in 2007, when Hain was Northern Ireland secretary, he launched a trial, which “absolutely thrilled” the prince.

Facebook Twitter Pinterest Prince Charles on his tour of Chester. Photograph: David Levene

Charles took a close interest in Hain’s progress. He listened to Hain’s interviews on Radio 4’s Today programme and gave feedback on his performances. “He wanted to persuade, as I did, the secretary of state for health and colleagues in government to do the same kind of pilot study [on the effectiveness of complementary medicine],” said Hain. “I would speak to colleagues and he would approach it in whatever way he chose. I think it involved letters and meetings. He encouraged me and I encouraged him.” During this period, the pair shared a dinner with their wives in the upstairs dining room at Clarence House. Despite the formality of the protocols, Hain said that Charles was “full of humour” and “easy to talk to”. Charles sipped a soft drink while the others enjoyed wine from the royal cellar.

Whether it was due to his alliance with Hain or his effort through other channels, the prince’s campaign worked. In 2005 his charity, the Foundation for Integrated Health, began receiving £1.1m in Department of Health grants to advise on the regulation of massage, aromatherapy, reflexology and other complementary therapies.

From 1997 to 2003, Michael Meacher, MP for Oldham West and Royton, who was then environment secretary, became another of Charles’s cabinet allies. At that time, a debate was raging over whether the UK should allow genetically modified crops to be grown commercially in the UK. In 2008 Charles told the Daily Telegraph that the development of GM crops would be “the biggest disaster, environmentally, of all time”. Tony Blair was in favour of GM crops and complained to Peter Mandelson that Charles’s lobbying was “unhelpful”. (Mandelson later described it as “anti-scientific and irresponsible in the light of food shortages in the developing world”.)

Meacher, who was sympathetic to the prince’s views, soon received an invitation to Highgrove. “As we went through the Highgrove gardens, I found myself alone with him when we got to a particular place and I think that was probably arranged,” Meacher said. “On climate change and on organic farming, we had shared views. We had agreed objectives and we were both going to do what we could to achieve them.” At receptions with the prince, Meacher would sometimes “get a message from one of his aides” saying the prince wants to have a word with him. Meacher said he was “obviously pleased” with such invitations. During his time in the cabinet, Charles wrote him eight or nine “encouraging” letters about climate change and the environment.

There is no sign that Charles has let up in recent years. Since the beginning of the year he has held meetings with nine ministers in the UK and Scottish governments including David Cameron, George Osborne and Alex Salmond. In just three days in September he met Liz Truss, the environment secretary, Brandon Lewis, housing minister and John Hayes, transport minister. Then there are the dinners at Highgrove, the prolific letter-writing, and campaigning by the 15 charities of which he is president. It is clear that Charles’s “mobilising” machine is running at a high voltage. Will he be willing to step away from the controls when he becomes king? Paul Flynn, a member of the commons political and constitutional reform committee, has predicted that unless he does so, there will be “a big confrontation between the monarchy and parliament”.

* * *

On a warm September day, Prince Charles and the Duchess of Cornwall stepped out of a Bentley on to a council estate in Chester, having hopped over from RAF Northholt in a private jet that morning. On the back seats of the car were “his and hers” cushions in case the creamy leather upholstery was insufficiently comfortable. Charles had come to Chester for a day of walkabouts.

Facebook Twitter Pinterest Charles at a primary school in Lache, which he last visited in 1972, on the outskirts of Chester. Photograph: David Levene

Within seconds of arriving at his first stop, a primary school in the suburb of Lache, Charles was burbling greetings in a husky baritone to a line of dignitaries who wore pinstripes and fascinators. For a man who is supposed to worry a lot, Charles seemed relaxed – his skin was tanned and his suit immaculate. Somehow alert and detached at once, he skilfully picked people out from the crowd and spent a little time with each of them. He looked for moments of humour and broke into an infectious laugh as often as he could.

Every day, Charles must negotiate some extraordinarily banal conversations, which is inevitable when you initiate small talk with 20 people in 15 minutes, as he did here.

Charles: “What are these bulbs?”

Teacher: “Spring bulbs”.

Charles: “Oh, well done.”

If he was bored (and he mostly didn’t look it), he could be forgiven. A plaque commemorates his last visit to the school. It was 1972. He has been a long time in the same job.

Outside the school gate I asked locals about his impending accession to the throne. Suddenly we were back in the 1990s, the royal family’s lowest ebb in recent memory. John Schofield, 74, a retired local government officer was talking to his neighbour Bryan Williams, 47, a gas fitter, before the royal motorcade arrived. Schofield said he liked the family “but I don’t think he should be king because of the divorce”. Williams agreed. “The crown should go down a generation,” he said. “I think Charles is a bit too old and he has been in his mother’s wake so long. And a lot of people thought so highly of Diana and her sons have taken on everything she stood for.”

A popularity poll in June for ComRes suggested there could be a profound slump in public affection for the monarchy when Charles takes the reins. He scored 43% approval against the Queen’s 63%. One former cabinet minister I spoke to agreed with the widespread view that Charles’s relationship with Diana was the biggest factor in public antipathy towards him.

“People outside the metropolitan bubble, the people who really care about the monarchy, are still quite upset about what happened,” said Catherine Mayer, an editor-at-large of Time Magazine, whose biography of Charles will be published next year. “They are inclined to believe a fiction which sees him as a cynic, an experienced older man who married a much younger bride and then treated her badly.”

Even so, Charles strongly believes he has a public mandate to engage with the political side of public life. His allies argue that his right to engage with government is rooted in a profound connection with the British people – not least through hundreds of public engagements each year. He is, they say, “well-placed to relay public opinion [to ministers] on a number of issues”.

In 2010 he successfully blocked Richard Rogers’s £3bn modernist redevelopment of the Qatar-owned Chelsea Barracks site by complaining to the prime minister of Qatar that it was another “brutalist” development of the kind responsible for “the destruction” of London. At that time, Charles’s then private secretary, Sir Michael Peat, said it was his “duty to make sure the views of ordinary people that might not otherwise be heard receive some exposure”. Rogers, among others, disagreed. Mr Justice Vos, presiding over a related high court case, criticised Charles’s intervention as “unexpected and unwelcome”.

But how far should he go in speaking out? On his trip to Canada in May this year, he waded into the Ukraine crisis, telling a 78-year old Jewish woman who had fled the Nazis that Vladimir Putin was “doing just about the same as Hitler”. Putin said the statement was unacceptable and “not what kings do”. Foreign Office diplomats might have been alarmed, but 51% of British people polled by YouGov said the comments were appropriate and only 36% disapproved.

Many of those who know Charles well believe that, as king, he will not adopt the same discreet style as his mother. As one source put it: “The man the public has seen for the last 40 years is the man the prince is.” “He will be true to his beliefs in his contributions,” said another source, who has known Prince Charles for many years. “Rather than a complete reinvention to become a monarch in the mould of his mother, the strategy will be to try and continue with his heartfelt interventions, albeit checking each for tone and content to ensure it does not damage the monarchy.” Another source spelled out the possible new rules of engagement: “Speeches will have to pass the following test: would it seem odd because the Queen wouldn’t have said it – or would it seem dangerous?”

His allies offer assurances that there is no cause for concern. Firstly, they argue that Charles and his officials already have a close working relationship with government and that Charles’s team at Clarence House usually shares with ministerial aides any speeches that touch on policy, so that they are ready to iron out problems in advance. Yet it is hard to believe, for instance, that Whitehall cleared Charles’s comments about the “tragedy” of the government’s slow response, in February, to the floods in southern England.

He will have very firm advice when he comes into office to overcome the habits of a lifetime

What’s more, they say, King Charles simply won’t have as much time to “mobilise”. The monarch’s schedule is weighed down by the daily red box of state papers, investitures and formal meetings with incoming and outgoing diplomats and clergy. That, too, sounds a little unrealistic, given the plentiful evidence of the prince’s appetite for activism. But one source, who has known him for years, dismissed such concerns: “In private moments, at the end of the day and talking freely, he gets frustrated that people don’t think he doesn’t understand it is a completely different job being head of state.”

“He fundamentally gets the role that the monarch plays in our constitution,” one source said. “What politicians are, the monarch is not. Politicians are inherently divisive because they are not standing for the nation. Politicians think in five year cycles, but the monarch thinks long-term.”

Meacher said he believes Charles wants to influence politicians when he is king. But, Meacher argued, there should be more transparency, and the public has “a right to know if the king has taken an interest in [an issue] and has written to the relevant minister”. Others think Charles must desist altogether. “He will have very firm advice when he comes into office to overcome the habits of a lifetime,” said a former top Whitehall official.

* * *

The night before Charles’s trip to Chester, four members of Republic, the national campaign for an elected head of state, were plotting the end of the monarchy over carrot cake and tea in a neat suburban house on the outskirts of Altrincham. The area around Manchester and Liverpool, two cities with radical political histories, is a relative stronghold for Republic, but the group still has only 15 regular local activists. Inside the house, flyers and #bornequal badges covered the dining room table, while a banner demanding a republic flopped over a toy. The republicans were preparing to run stalls at Wigan Diggers, a festival celebrating the life of Gerard Winstanley, a Cromwell-era political reformer, and at the Live a Better Life vegan festival, where they planned to focus on the royal family’s love of hunting.

Republicanism is still a taboo, the activists said. (In polls over the past decade, support for Britain becoming a republic has remained at only 10 to 20%.) They spoke about embracing their beliefs as if they were coming out of the closet. “I’ve had feelings along these lines for a long time without knowing where to go with it,” said Terry Bates. “And then 30 years later I find out I am not alone in feeling like that.” For Bates, the moment of truth came at a packed rugby league stadium in Wigan, when he decided to stay seated as the national anthem was played. “I realised this needed to be my protest in front of 32,000 people in the ground,” he said. “Now if I hear it on TV I walk out of the room.”

Facebook Twitter Pinterest With the Duchess of Cornwall – and Grace the

golden eagle outside Chester cathedral. Photograph: David Levene

Charles, said Bates, is a useful “recruiting sergeant” in the republican cause. “Are people going to be singing ‘God save our gracious king’ with quite as much enthusiasm?” agreed Helen Guest, 36, a former nursing sister. “I don’t think so.”

But if the middle-aged, beige-clad crowd pressed up against the fence of Chester Cathedral at the royal couple’s second event of the day is any indicator, then the answer is yes. Charles and Camilla shook hands, posed for photos and accepted gifts. According to Charles’s allies, talk of William and Kate “eclipsing” the prince underestimates his popularity with the baby boomer generation, which remains a key constituency for Charles. In Chester, he seemed to be shoring up support with every handshake.

Inside the cathedral, the strangeness of Prince Charles’s life came into focus. Around one corner a choir performed a piece by Charles’s favourite composer, CMH Parry. Some modernist choir stalls, installed 15 years ago, caught his disapproving eye. “Doesn’t quite go,” the prince announced, locking eyes with the senior churchman. “It may be time for a review.” Around the next corner were members of the Mercian regiment of soldiers, waiting to shake hands with the prince. Behind them was a group of earnest amateur dramatists, who were poised to enact part of a mystery play. They wore what looked like M&S pashminas as biblical headdresses. Finally, in the cloister, Charles was invited to hold Grace the golden eagle, a magnificent bird who, moments earlier, had evacuated her bowels explosively onto this reporter’s notebook.

The day ended with a trip to see a plane wing being assembled at the Airbus factory at Broughton and a reception for the industrial cadets programme, which Charles supports as part of his aim to encourage more young people to take up engineering. Improbably, the meticulously planned cocktail of engagements reflected his interests. Charles had done his bit for the nation’s education, youth, faith, heritage, and industry – all in the space of three and a half hours, before his jet roared off into the sky to take him back to London. About an hour’s flight – long enough perhaps, for three or four more black spider memos that the rest of us may never see.

Follow the Long Read on Twitter: @gdnlongread