As television interviews go, it was one of the most excruciating – and most sought-after – in British history. But when Prince Andrew’s painstakingly negotiated head-to-head with Emily Maitlis in Buckingham Palace finished, the royal appeared oblivious to the damage that had been done. In fact, he was so pleased with how things had gone that he gave the Newsnight team a tour of the palace afterwards.

On Sunday, as the prince’s team picked up the pieces from an interview widely perceived to have been disastrous for his reputation, the remarkable story of how it came about emerged – from the departure of a key aide to drawn-out discussions and a last-minute message to the Queen.

Esme Wren, the editor of Newsnight, said the interview was secured after six months of negotiations, with Andrew’s team referring the decision up the royal chain of command before finally agreeing to take part. “One assumes that means checking with his mum,” she said.

On Sunday night, though, the Daily Telegraph reported a different account, quoting “Palace insiders” who accused the Duke’s private office of “operating in a silo” and said that the Queen was only made aware of the interview after it had been set up – implying that she had no opportunity to veto the plan.

Friends of Jason Stein, who worked as the prince’s press adviser for little more than a fortnight, confirmed that he told Andrew not to do the interview shortly before leaving his position by mutual consent last month.

People involved in the recording have told the Guardian that after the recording Andrew was pleased with how it had gone. “I think he appreciated the fact that he’d had plenty of time to say his piece and that there hadn’t been interruptions,” one said. “There was a sense on both sides that there had been a generous amount of time and he had been allowed to speak. He was incredibly gracious afterwards.”

Andrew’s private secretary, Amanda Thirsk, is now facing questions over her position after playing a key role in pushing for the interview to take place. Andrew’s team were nervous about how the interview would be perceived, sources said, and they are unlikely to have been reassured by the days of negative coverage that it has created. Others said the general public would appreciate Andrew putting himself up for scrutiny, and his references to a “straightforward shooting weekend” and rare visits to Pizza Express revealed him as he is, without PR sheen.

According to multiple sources with connections to the royal household, Thirsk has always had complete belief in Andrew’s insistence that he is innocent regarding the claims of his involvement with the child sex offender Jeffrey Epstein and sexual impropriety. It is thought she felt the public would come around to his side if they saw the royal explain himself in his own words on television.

A former banker who has worked with Andrew for more than a decade, Thirsk was originally hired to focus on his financial affairs, according to people with knowledge of the household. She worked her way up to the top job of private secretary in 2012, taking responsibility for shaping the prince’s reputation despite not having any background in public relations or media. Former members of the household have talked of being “survivors” of her management style, while also noting her total loyalty to Andrew.

Wren, who became editor of Newsnight last year, said her team had repeatedly requested an interview with Andrew through their planning producer, Sam McAlister, but until recently they had assumed there was no chance of it happening.

Six months ago the programme turned down a proposal to interview Andrew about his charity work, saying they would consider it only if there was no limit to what questions could be asked. Then the palace renewed contact last month.

“A couple of weeks ago they said they were ready to do an interview with no subject off the table,” she said. This culminated in despatching the programme’s lead presenter, Emily Maitlis, and a small team to meet Andrew and discuss a plan for the interview.

“Last Monday they went in to the palace to discuss with his team and him what would be in the interview,” Wren said. “He wanted to find out more about Newsnight, really. He said he was going to refer up – one assumes that means checking with his mum – and on Tuesday they said they were going to do it and they wanted to do it quickly.”

Wren, who was not at the recording, said she was aware of the need to ensure all questions were asked, to avoid any perception of Andrew receiving an easy time. “We had to be absolutely resolute that we had to ask him every single question or else we would be deemed to have not done the job. We already had one of our investigative producers working on a brief and had a lot of work in train.”

Throughout the week Maitlis practised her interviewing approach, with Wren playing the role of Andrew in rehearsals. “We spent time drilling the questions back and forth, back and forth. A lot of this was tone and how we’re going to deliver this. Speak normally, don’t turn away.

“They knew what Newsnight stood for, and they knew that no one would say he was let off. They knew that if they do Newsnight, no one will say they were softballed. To our surprise, he did engage with questions and didn’t turn away a single question put to him.”

Although the programme was originally intended to run in Newsnight’s normal weeknight slot, when Wren realised the importance of the material she called the BBC Two controller, Patrick Holland, who agreed to put out a special one-off Saturday night programme to carry the interview in full and avoid it clashing with Children in Need.

Previous attempts to rehabilitate the prince’s reputation, such as an interview to promote his Pitch@Palace tech startup scheme, also produced mixed results.

Questions remain over the decision to allow one of the UK’s leading political journalists a lengthy sitdown with no vetting of questions. Former royal advisers point out that the story had largely dropped off the agenda and other broadcasters might have been willing to compromise on format and agree to a much shorter interview. American TV stations could also have been willing to agree to a softer style in order to obtain the access.

Allowing the BBC control over the release of information means the coverage has lasted several days, as Newsnight dripped out different news lines day by day before the interview.

Despite the reception in the press, Wren said there had been no complaints over Newsnight’s treatment of the prince. “I haven’t heard back from the palace,” she said. “I believe they feel we were, as we had promised, fair and forensic and robust. We haven’t heard to the contrary.”

• This article was amended on 18 November 2019 to correct descriptions of Epstein’s offending.