London (CNN) Prince Andrew had it all. He is -- reportedly -- the Queen's favorite son. A decorated war hero. A playboy prince once desired by many.

But the prince has also attracted a fair amount of scandals and controversies over the years. The latest one -- his association with prominent sex offender and disgraced financier Jeffrey Epstein -- is casting an especially dark shadow over him.

A BBC interview broadcast Saturday -- the first time he had spoken publicly about the accusations -- sparked near-universal condemnation in the British press and on social media.

The Duke of York told BBC Newsnight's Emily Maitlis he had seen nothing that struck him as suspicious when he was around Epstein, who died by apparent suicide in August while awaiting trial on federal charges that he sexually abused underage girls and ran a sex trafficking ring. Epstein had pleaded not guilty.

He chose to stay at Epstein's home, the prince said, because it was "convenient" and "honorable." Asked by Maitlis whether he felt any "guilt, regret or shame" about his behavior or friendship with Epstein, the prince said only that it was "the wrong decision to go and see him in 2010."

The prince said he let the royal family down by associating himself with Epstein, even after the American was convicted as a sex offender.

"I stayed with him and that's ... that's the bit that ... I kick myself for on a daily basis," Prince Andrew said during the interview, recorded at Buckingham Palace Thursday.

"Because it was not something that was becoming of a member of the royal family and we try and uphold the highest standards and practices and I let the side down, simple as that."

'Spare heir'

Andrew, whose royal title is His Royal Highness The Duke of York, is the second son of Queen Elizabeth II and a brother of Prince Charles

Originally second in line to the throne, he became number three when Prince William was born, and has been slipping lower and lower with each successive royal birth. He is now number eight.

Being the "spare heir" is a notoriously tricky role to fill. Throughout his lifetime, Andrew has only seen his chances of ever acceding diminish -- yet he is still facing all the scrutiny and expectations of being a royal.

Like other spares, notably the Queen's sister, the late Princess Margaret, and more recently Prince Harry , Andrew was at times seen as the "party royal."

Prince and and "Fergie" were among the most popular royals in the late 1980s.

His colorful past is perhaps best illustrated by the number of nicknames UK tabloids have come up with for the prince over the years, from "Randy Andy" to "Airmiles Andy."

"When you think of Prince Andrew, you think of three things," said Richard Fitzwilliams, a royal commentator.

"His loyal service in the Falklands, his ex-wife Sarah Ferguson, who was a catastrophe of a royal, and now of course also the Epstein link."

War hero

The duke is often referred to as the Queen's favorite child, an attribute that Fitzwilliams said can be traced to his early childhood.

"When Prince Charles and Princess Anne were born, the Queen wasn't able to spend the time with them that she would have wished to," Fitzwilliams said.

By the time Andrew was born, he said, the Queen had been on the throne for some years. "She was able to give him more attention and Andrew was someone with whom she's had a particular affinity," Fitzwilliams added.

Unlike his older brother Charles, Andrew didn't go to university. Instead, he joined the Royal Navy. A few years later, he was sailing across the Atlantic on a mission to retake the Falkland Islands from Argentina. He returned a hero -- and a darling of the nation.

Enter Sarah Ferguson.

Prince Andrew is still involved in projects aimed at boosting British business.

The prince and Ferguson, known as "Fergie," were the ultimate "It" couple of the late 1980s. Their wedding drew a TV audience of hundreds of millions. Tens of thousands of people lined the streets on the day, an event the New York Times described as "Fergie fever."

The scandalous years

A commoner, Ferguson was said to bring a breath of fresh air to the royal family. Too fresh, according to some. "He and Sarah, and there's no doubt about this, appear to have a weakness for colorful characters who live in an ostentatious way," Fitzwilliams said.

The honeymoon was over soon. Andrew and Ferguson separated in 1992 and divorced in 1996, following a number of scandals. However, the two maintain a close relationship -- much to the amusement of UK tabloids.

Prince Andrew (left), pictured with his brothers Charles and Edward at Sandringham in April 1969.

After leaving active military service in 2001, the prince became a "full-time royal," carving out a role for himself as a champion of British business. He became the United Kingdom's special representative for international trade and investment, traveling the world selling the British brand.

That career came to an abrupt end a few months after he was photographed with Epstein in Central Park in 2010. At that time Epstein was a registered sex offender who had served 13 months in prison on prostitution-related charges.

The Epstein link

Andrew was one of the many high-profile, influential men Epstein had associated with. The prince said he met Epstein in 1999 and "saw him infrequently and probably no more than only once or twice a year." He also admitted to having stayed at "a number of his [Epstein's] residences."

Epstein, 66, was found dead in his New York prison cell in August. He had been awaiting trial on US federal charges accusing him of sexually abusing underage girls.

The prince has always claimed he never suspected the kind of behavior that Epstein was accused of. But that claim is problematic given he associated with Epstein even after his 2008 plea deal , in which he pleaded guilty to two state prostitution charges.

in fresh allegations that he The 59-year-old prince was named in court papers in New York in Augustin fresh allegations that he groped a young woman at Epstein's Manhattan property. Previously, he was named in connection to Epstein when Virginia Roberts Giuffre claimed the financier kept her as a teenage sex slave.

Andrew, who is not facing legal proceedings, said "any suggestion of impropriety with underage minors is categorically untrue." The prince insists he never witnessed or suspected the kind of behavior that led to Epstein's arrest, but admitted it was "a mistake and an error" to see the disgraced financier in 2010.

The episode is doubly embarrassing for the prince, given his daughter Princess Eugenie's involvement in a charity fighting modern slavery

A photograph appearing to show Prince Andrew with Jeffrey Epstein's accuser Virginia Roberts Giuffre and, in the background, Ghislaine Maxwell.

While Andrew continues to support business in the UK through several initiatives, he doesn't have a formal role.

"The friendship was absolutely disastrous for him, it cost him his job, and even though he can't resign as a member of the royal family, his name is attached to nearly 200 patronages and it's an open question whether they will all continue," Fitzwilliams said.