Every week, in a meeting shrouded in secrecy, the PM briefs the Queen about British politics. What influence does she exert? Who has she liked – and who made her nod off? Peter Morgan on how this strange tradition inspired his new play

When I started writing the screenplay for The Queen, about the aftermath of the death of Princess Diana, both Stephen Frears, the director, and Andy Harries, the producer, begged me not to put Tony Blair in it. They felt the presence of a politician, particularly one as divisive as Blair had become by 2004, would diminish it – make it feel more temporal, more journalistic, more TV. And so less filmic.

I consoled myself that there would still be plenty to work with: the death of a princess, a Queen's young deputy private secretary out of his depth, a royal family tucked away on the Balmoral estate, and people out on the streets of London baying for blue blood. But after three months, I had written just 35 pages and wanted to shoot myself. It was awful. I rang Frears and Harries and told them the script was a big fat royalist snooze. Would never work. Everyone gracefully accepted defeat, and Frears went on to direct another movie.

Privately, however, I was deflated. I felt I hadn't given an interesting subject a fair shot. I retreated to the mountains of Austria and, without telling anyone, wrote my own version. With Blair. I didn't care if it felt smaller. I didn't care if it was journalistic, nor if it ended up on TV or radio. I wrote a draft in under three weeks. It was one of those experiences that comes all too rarely, where you hear the voices and write with total certainty. You're not really writing – you're channelling.

Looking back, I realise the reason I was suddenly so unblocked was because I had stumbled on something significant: the relationship between our most senior elected public servant and our head of state. The first minister and the crown. Two human beings, in flesh and blood, but also the representatives of their offices. At some level, just by having them sitting opposite one another, even in silence, one was dealing with the British constitution, the bone structure of our establishment in its most elemental form.

Happily, Frears liked the script and committed to filming it. As I watched Helen Mirren and Michael Sheen at work, I started thinking more about the sovereign and the PM – and the weekly "audience" at the heart of their relationship, and what a unique opportunity it presented a dramatist. Because the meeting is so shrouded in confidentiality, imagining what was discussed felt more valid, somehow, than proving it. I resolved, as far back as 2006, to write something.

In reality, depending on the chemistry between them, the audience can either be no more than an informal briefing – a courtesy, where the PM brings the sovereign up to speed with what happened in the past week and what is expected in the following one – or it can be a great deal more. Some of the pairings have been reluctant; some eagerly looked forward to. Some of the sessions last barely 20 minutes with no refreshment; others stretch out to an hour or two – with drinks. Some PMs are grateful for the breaks provided by foreign travel; others are sure to have their audience by phone even when abroad.

So when did they start? I wrote to Professor Vernon Bogdanor, the closest thing we have to a formal constitutional expert in this country where no formal constitution exists. "I am not wholly clear when they began," he said. "I do not think they existed before the war. But during the war, the practice arose of the King and Churchill meeting for regular lunches. That was a consequence of the particularly good relationship they had built up – after a shaky start. The regular meetings have now become so much of a convention they could be regarded as part of the constitution – in a typically British unplanned way. Were either the PM or the sovereign to discontinue them, this would, I think, be regarded as a breach."

So there it is: the private audience has come in through the back door. And yet the meeting is not minuted or recorded. No one else is present. This is vital – so that PMs can feel free to say what they like, and even make disobliging remarks about colleagues. As a weekly event, it takes up a significant amount of the PM's working life. Allowing for summer breaks, an average four-year term would contain at least 70 meetings. Seventy hours! I dare say there are happy friendships, even marriages, where partners don't sit opposite one another and talk openly, in a spirit of trust and mutual confidence, for an hour a week. Indeed, the very nature of the meeting (one-on-one, confidential, one-sided) reflects another relationship: therapy. And James Callaghan did say it was like talking to one's psychiatrist.

In which case, should we not be entitled to know more about it? How has the meeting worked over the years? What form does it take? Which PMs liked to talk? Who liked to listen? Who was HM's favourite? With whom was there chemistry, laughter, silence? What did they discuss? How many secrets were shared? What advice was given? What influence did it have? The Queen is known to have struggled to stay awake with Heath and Macmillan (both famous bores), actively disliked Blair and Thatcher (though the Palace dutifully denies this), and had a soft spot for losers (two of her favourites, Major and Wilson, regularly come bottom in rankings for most effective PMs of the 20th century).

One popular misconception might be that it's a polite chat, over scones or sherry, with the PM checking his or her watch, wanting to get on with more important business. But as Margaret Thatcher said: "Anyone who imagines that they are a mere formality, or confined to social niceties, is quite wrong; they are quietly businesslike, and Her Majesty brings to bear a formidable grasp of current issues and breadth of experience."

The Queen reads each one of her red ministerial boxes every day, is privy to the minutes from every cabinet meeting, has a generous staff to keep her informed, and is scrupulously prepared for every meeting. In civil service circles, she is known as "Reader Number 1". A meeting with Her Majesty, therefore, is like a meeting with a well-briefed civil servant. If transcripts of the audience were to exist, I'm confident she would emerge with some credit.

It struck me that by being denied the minutes of these conversations, we were being denied a significant part of British history: an insight into the workings of government and state, and the way power – real and symbolic – functions in our name. In the 19th century, the essayist Walter Bagehot argued that the Queen has three constitutional rights: the right to be consulted, the right to advise, and the right to warn. No more. But bear in mind she spends an hour every week sitting one-on-one with the most important politician in Britain. How much has she known over the years that has been denied us? How much has she known that we haven't?

The longer she remains on the throne, the greater her standing on the world stage and the greater the respect for her – and, therefore, the greater her potential surreptitious influence. Imagine you're Ed Miliband: you've narrowly won the election, and you go to the Queen to ask her permission to form a government. The idea that the most instantly recognisable woman in the world – who has sat opposite Churchill, Eden, Macmillan, Douglas-Home, Wilson, Heath, Callaghan, Thatcher, Major, Blair, Brown, Cameron, presided over a Commonwealth and had a ringside seat at the great political events of the second half of the 20th century – would not have influence on you is laughable, simply by virtue of your anxiety to at least leave an impression. You don't want to be the one she forgets.

That gives her influence, if not power. And any influence over our PM needs to be examined closely. So I set about writing a series of what must strictly be called "imagined" audiences (although no shortage of anecdotal information has leaked out over the years) between the Queen and her various PMs – and was immediately presented with a challenge. How could one tell the story without it feeling linear and inevitable? How could one avoid the almost audible ticking off in theatre-goers' heads as PMs came on and off the stage? Nothing is enjoyable if there is no sense of surprise – and everyone knows who they all were, even if they can't quite remember all their names.

So I decided to tell it in a non-linear way, leaving out some PMs. Who to drop and why? I wrestled long and hard with this and, as we head into rehearsals, I have made my decisions. But by curtain-up, it might be totally different. Who knows?

• This article was amended on 15 January 2013. It was 'a Queen's young deputy private secretary out of his depth', not 'a young Queen's DPS out of his depth'. This has been corrected.