Whatever the crises that have come and gone, in the Queen’s eyes, Andrew remains the son who can do no wrong

As 100ft flames licked across Windsor Castle a disconsolate figure in a headscarf stood watching, tense and tight-lipped, as firemen desperately fought to save her home. Acrid smoke billowing over the Queen made her eyes water.

She had rushed from Buckingham Palace on being told that her favourite home was ablaze. In all the confusion, Stephen Matthews, the head coachman, remembers seeing her standing helplessly as the castle burned.

‘Everything was going up in flames,’ he recalls. ‘And then I saw the Queen. She was standing in the middle of the quadrangle watching her house burn down. It just was terrible. She looked very small, very lonely and very sad. Nobody would approach her, though everybody wanted to, but knew it wasn’t their place. That’s the most emotional I’ve ever seen her.’

Being the Queen has never been so isolating, so lonely, as hearts burst for her. No one felt confident enough to put a comforting arm around her and Prince Philip was abroad. But at that moment she saw something that lifted her spirits. It was Prince Andrew marshalling staff into a human chain that was rescuing the castle’s treasure trove of paintings, furniture and artefacts.

Luckily, Andrew happened to be in the castle when the fire broke out. Still in the Navy, he had been carrying out some research there on behalf of the Royal Military Academy. And, to his credit, he took command to save so much that is precious to his mother.

‘Thank goodness at least we had the Duke of York there,’ says Matthews, now 63. ‘He sort of rallied everybody around and organised a hell of a lot.’

It wasn’t the first time the Queen had felt a surge of maternal pride in her second, and favourite, son. She has always regarded him as a ‘hero’ for his bravery in the Falklands War, when he served as a ‘decoy’ helicopter pilot with the dangerous role of distracting Exocet missiles away from British ships.

‘The Queen worried about him and what he was doing, of course,’ says her close friend and cousin Margaret Rhodes, ‘and she was very proud of him when he came back from the conflict with that rose in his teeth.’

In fact, she was there with other proud and relieved families at Portsmouth to welcome home 22-year-old Sub-lieutenant Windsor and hundreds of other Falklands heroes brought back on the aircraft carrier Invincible.

The Queen and Prince Philip, and also Princess Anne, were taken out to meet Invincible in a barge and came down the gangplank one after the other. Andrew had never looked more dashing and handsome — or the Royal Family more joyously reunited. If only Prince Andrew’s life was always as clear cut as that day. But whatever the crises that have come and gone, in the Queen’s eyes, he remains the son who can do no wrong.

Unlike Charles and Anne, both of whom were born when she was still Princess Elizabeth, Andrew was the son who came along when she had reigned for the best part of a decade. He was named after his paternal grandfather, the dissolute gambler Prince Andrew of Greece, who saw little of his son Prince Philip and died penniless on the French Riviera.

‘The Queen knew the ropes by then and was able to give him all the quality time that Charles says he didn’t get,’ says a former courtier. ‘She so loved looking after him she even curtailed her foreign travel.’

At bath time, she put on an apron and bathed him, and she would rock him to sleep. At Windsor Castle he was allowed to race his bicycle down its wide corridors, and play skittles along them while his mother dealt with her red boxes of Government papers in the late afternoon.

Unlike Charles and Anne, both of whom were born when she was still Princess Elizabeth, Andrew was the son who came along when she had reigned for the best part of a decade. He was named after his paternal grandfather, the dissolute gambler Prince Andrew of Greece

The Queen's maternal closeness with her third son (second from left) — and with Edward (second from right), who was born four years later — has never changed. ‘They are just as close today,’ says a former lady-in-waiting

On Saturdays at teatime, he would routinely sit with his parents — Charles and Anne were away at boarding school — watching the BBC’s Grandstand sports programme on television. On Sundays he would watch cricket with them, also on television. Despite this early exposure to our national games, his passion became golf.

The fact is, such maternal closeness with her third son — and with Edward, who was born four years later — has never changed. ‘They are just as close today,’ says a former lady-in-waiting.

Perhaps that hasn’t entirely worked in Andrew’s favour. Rarely chastised as a child, he grew up, says a courtier, ‘with a pompous level of self-importance based on being second in line to the throne. Of course, the arrival of his nephews William and Harry pushed him down the succession, and he felt it.’

The Queen has always tried to help him in this respect, by making sure he has a ‘role’, not always, it must be said, with success.

When Pan Am flight 103 and its passengers were blown up and fell on the luckless town of Lockerbie a few days before Christmas 1988, the Queen’s then deputy private secretary Robert Fellowes urged her to go there. But as with the Aberfan disaster 22 years earlier, when a sliding coal tip crashed onto the Welsh village near Merthyr Tydfil, killing 144 people, most of them children, the Queen decided not to visit immediately fearing she would be a distraction from the desperate recovery work.

She could have asked any of her four children to go to Lockerbie in her stead, but she chose Andrew. He was not a good choice.

Her Majesty has always regarded him as a ‘hero’ for his bravery in the Falklands War, when he served as a ‘decoy’ helicopter pilot with the dangerous role of distracting Exocet missiles away from British ships

She could have asked any of her four children to go to Lockerbie in her stead, but she chose Andrew. He was not a good choice

Not only did he upset the people of Lockerbie — where 11 residents were killed on the ground — by declaring that it was ‘much worse for the Americans’ (259 passengers and crew were on the U.S. airliner) but he added that it had been ‘only a matter of time’ before a plane fell out of the sky.

A few days later she told Fellowes: ‘I was wrong [about Lockerbie] — I wish I had gone.’ Whether she had Andrew’s faux pas in mind we do not know.

Then there was Andrew’s marriage to Sarah Ferguson. Here we have a woman whose adventures with other men while married to the Prince deeply upset the Queen because of the humiliation heaped on her son.

But almost 20 years after their divorce, the couple continue to share not one home but two in a domestic arrangement of almost unfathomable complexity. As well as regularly staying with Andrew at his home, Royal Lodge, in the shadow of Windsor Castle, the pair have jointly bought, for £13 million, a sumptuous ski lodge in that most chi-chi of Swiss ski resorts — and their favourite — Verbier, where Fergie is seeking Swiss residency.

The arrangement baffles the no-nonsense Prince Philip but is accepted without question by the Queen.

Even more puzzling to Philip is the Queen’s apparent continuing affection towards the former daughter-in-law who brought so much embarrassment, and even shame, to the Royal Family.

She was pictured having her toes sucked by a lover when separated — though still married — to the humiliated Andrew. In a frenetic period of marital upheaval involving three of her children, that poolside spectacle, with Fergie’s small daughters Beatrice and Eugenie playing nearby, was the one that wounded the Queen more than any other.

Then there was Andrew’s marriage to Sarah Ferguson. Here we have a woman whose adventures with other men while married to the Prince deeply upset the Queen because of the humiliation heaped on her son

Fergie was pictured having her toes sucked by a lover when separated — though still married — to Andrew. In a frenetic period of marital upheaval involving three of her children, that poolside spectacle, with Fergie’s small daughters Beatrice and Eugenie playing nearby, was the one that wounded the Queen more than any other

And yet, these days, to the bafflement of Prince Philip, the Queen is friendly towards her.

She even gave Andrew permission to help bail out his hard-working but extravagant ex-wife from huge debts with £1.5 million of his own money.

Long-standing Windsor Castle staff remember a Sunday around the time of the divorce when the Queen mentioned to Philip, and to Prince Edward who was with them, that the Duchess was coming to tea. A servant recalls: ‘The Duke said: “I can’t face that, I’m having tea in my study.” Edward then said: “I can’t face it either. I will have tea with my father.” ’

For her part, Fergie knew the Queen was ‘furious’ about her behaviour. ‘Her anger wounded me to the core,’ she has said, ‘the more so because I knew it was justified. I had violated her trust.

‘I’d betrayed the bond that we’d built ever since she had invited me to Royal Ascot in 1985 as one of the younger people she enjoyed having about.’ When Fergie wrote her memoir, My Story, and it was serialised in Hello! magazine, the Queen read it with surprising equanimity. ‘Better now than in five years time when everything is quiet and it’s all dragged up again,’ she told a friend.

Princess Margaret’s attitude to Fergie was unequivocal. Her chauffeur David Griffin was driving her to RAF Northolt when news came through that the Duke and Duchess of York were divorcing. ‘Poor Andrew,’ remarked Margaret to a companion in the back of the car. ‘Thank goodness he’s got rid of that dreadful woman.’

Remarkably, even with the Duchess being ‘non grata’ at court — and excluded from Christmas at Sandringham — the Queen never allowed it to affect their personal ties. So keen was she for Fergie to remain in the family orbit that she even offered her a grace-and-favour house near Sunninghill Park, the marital home which had been her wedding gift.

Andrew still has a flat in Buckingham Palace and whenever the Queen hears he’s there by himself she’ll send him a hand-written note and he will change into a suit and go and see her, greeting her with a bow from the neck, then kissing her hand and both cheeks

These days, she sees Andrew at Windsor most Sundays after church when she joins him for a pre-lunch drink at Royal Lodge, the home he inherited from the Queen Mother. He spent £8.5 million on refurbishments before moving in

It was small and unglamorous, but living there would at least provide security for the erring Duchess and the Queen’s granddaughters Beatrice and Eugenie. It would also mean, as former head coachman Colin Henderson explains, ‘that the Duchess was still in the family fold’.

Why on earth would the Queen want that after the disgrace of the toe-sucking holiday with American lover John Bryan? The simple answer, says a courtier, is that: ‘She was doing it for Andrew. And she was disappointed when the Duchess said “No thanks” to the house. For she knew that despite everything that had happened, Andrew still loved her and felt that he might be able to win her back.’

The Queen has always done ‘just a little more’ for Andrew, says a family friend. ‘It’s ‘second son syndrome’. Andrew was the spare to Charles the heir and the Queen has always tried to compensate by ensuring that he is not left out of things. It’s a mother’s instinct.’

He still has a flat in Buckingham Palace and whenever the Queen hears he’s there by himself she’ll send him a hand-written note and he will change into a suit and go and see her, greeting her with a bow from the neck, then kissing her hand and both cheeks. ‘It’s a little ritual that she adores,’ says a Palace aide.

It’s second son syndrome. Andrew was the spare to Charles the heir and the Queen has always tried to compensate by ensuring that he is not left out of things. It’s a mother’s instinct

These days, she sees Andrew at Windsor most Sundays after church when she joins him for a pre-lunch drink at Royal Lodge, the home he inherited from the Queen Mother. He spent £8.5 million on refurbishments before moving in.

The Queen likes it when Beatrice and Eugenie are there, and even when Fergie is staying overnight and is there to greet her. She and the Queen have sometimes taken a walk in Windsor Great Park together. But these days this is rare as her former daughter-in-law spends most of her time in Verbier.

Meanwhile, Andrew has lurched from the crisis in his marriage to crises that have emerged through him spending so much time in the company of dubious billionaires.

Of particular embarrassment was his friendship with wealthy American paedophile Jeffrey Epstein, who served more than a year in a Florida prison for paying teenage girls for sex. Andrew went to parties at Epstein’s palatial home and accepted £15,000 from him when arranging to pay off Fergie’s £1.5 million debts in 2010.

Soon after Epstein’s release from jail the two were pictured walking together in New York. It was this picture that triggered the end of the Prince’s role as a roving international trade ambassador for Britain. The friendship caused a rift between him and Charles, and upset the Queen who urged her son to end his links with the seedy businessman.

Then there was the curious saga of Sunninghill Park, the house she had built for him and Fergie close to her in Windsor Great Park as a wedding gift — which he subsequently abandoned and sold at an inflated price to a Kazakh billionaire.

Prince Andrew’s friendship with the billionaire sex offender drew scandal and opprobrium. In 2011 this photo of Prince Andrew walking in Central Park with Epstein, shortly after Epstein's release from jail, saw the Duke forced to quit his role as the British Government’s global trade envoy

Just the other week it was bulldozed to the ground. In the eyes of some friends, this is Andrew displaying a cavalier attitude.

The Queen, however, is said to refuse to criticise him for it. ‘In the Queen’s eyes,’ says a family friend, ‘the Duke’s private life is his private life. She keeps him on something of a pedestal.’

As if to prove the point, at the height of the Epstein affair a private ceremony took place at Buckingham Palace. With astonishing timing, the Queen was presenting Andrew with the gold embroidered insignia of a Knight Grand Cross of the Royal Victorian Order, a mark of her gratitude for his services to the Crown.

How she must wish that she could resolve another troubling issue with nothing more than a ceremonial bauble. But it will take a lot more than that to settle the simmering row that now divides Andrew and his brother Charles.

Even more regrettably for the Queen, it all started at her Diamond Jubilee. At least, that is when the public could see that there was a change among the cast in the lead roles of the royal road show.

As the Red Arrows streaked across the London sky towards Buckingham Palace it became clear that Andrew, together with his brother Edward and sister Anne, were no longer ‘key’ royals. For they were not with the Queen on the Palace balcony.

At her age, it was inevitable that many executive decisions should have been handed to Charles, now 67 and the longest waiting heir to the throne in British history.

But this was a dramatic statement of intent by Charles, who long ago decided that when he is king the Royal Family would be ‘slimmed down’.

As the Red Arrows streaked across the London sky towards Buckingham Palace on the Queen's Diamond Jubilee it became clear that Andrew, together with his brother Edward and sister Anne, were no longer ‘key’ royals. For they were not with the Queen on the Palace balcony

Andrew was furious, Edward dismayed, while Anne, the hardest-working royal, remember, knew that as the only woman in the Queen’s family, she would always have a role.

The Queen was uncertain about the plan at first, recalling how, ten years earlier, at her Golden Jubilee, the family had filled the balcony and the public loved it.

But with Prince Philip in hospital with a bladder infection, she was relying heavily on Charles. At the time Charles was close to her senior adviser, her private secretary Sir Christopher Geidt (though they are close no longer), and had been told he could initiate some of the changes in style he wanted as king.

Reluctantly, the Queen accepted Charles’s view that the changes were necessary because the mood of the public was for what one courtier describes as an ‘economy monarchy’.

For Andrew, being excluded from the balcony — he was also not invited to the livery companies’ celebration lunch for the Queen at Westminster Hall — was, said one of his friends, ‘like a dagger to his heart’.

For the Queen this is a sad development. But in allowing Charles to become effectively the Royal Family’s ‘chief executive’, there was always a risk that Andrew — not to mention Edward — would be downgraded.

Even so, Andrew has had the consolation of spending part of this winter, as usual, with his ex-wife in Verbier, playing house at the Chalet Helora, which they can both call home, and where Fergie lives almost permanently.

When they give dinner parties there, the couple, who have been divorced almost 20 years, refer to each other around the table as ‘husband’ and ‘wife’.

However fondly the Queen looks on her 56-year-old second son, and on Fergie, the re-marriage that some friends think could happen is not on the cards. At least, not while Prince Philip, who will be 95 in June, is still around.