Cardinal Keith O'Brien had a long-standing physical relationship with one of the men whose complaints about his behaviour sparked his downfall as leader of the Catholic Church in Scotland.

The man left the priesthood in the middle of the last decade but rejoined and is living on the continent in a post the cardinal helped him secure.

The complainant is known to have been in regular telephone contact with Cardinal O'Brien until recently and was a frequent visitor to St Benets, his official residence in Edinburgh's Morningside.

It is understood the cardinal confessed to the relationship after it was recently revealed there had been several complaints to the Vatican about his sexual behaviour towards priests in the 1980s. It is thought to be part of his reference to his sexual conduct as "a priest, a bishop and a cardinal".

It also emerged the dramatic downfall of Britain's leading Catholic cleric was spurred by gay priests angry at his rhetoric and hypocrisy about same-sex marriages.

All those who complained about Cardinal O'Brien and alleged they had been abused by him were known to him for decades. At least two are known to have been in same-sex relationships and had become exasperated at double standards in his statements about gay marriage.

In the six months building up to him being forced to stand down last month, the cardinal had been under some pressure from priests to tone down the rhetoric.

However, his statements, such as describing homosexuality as a "moral degradation", were a tipping point for those previously close to him.

The first complainant alleged an assault in the Vatican on the day Cardinal O'Brien was made a cardinal. He is living outside Scotland, having taken temporary leave from the church. He was given leave of absence from the Diocese of Aberdeen and is understood to be in a relationship with an Anglican churchman.

This complaint, made in September 2012 and known among some members of the Catholic clergy in Scotland beforehand and immediately afterwards, led to the four others lodging their own complaints with the Vatican.

The man was due to speak to The Herald but is understood to have taken advice not to do so by his bishop.

A spokesman for the Catholic Church said: "The priest concerned is a priest of the diocese of Aberdeen currently on a leave of absence from parish ministry."

One senior church figure said that while some fundamentalist Catholic groups had previously linked the priest with Cardinal O'Brien "there were many questions that others were asking about the relationship".

Another source said: "These guys, we now know, were part of an inner circle. In the 30 years since these allegations took place there's been ample time to complain. The Cardinal has had a huge profile for the past decade. But the door wasn't just shut on them, it was bolted in the past 18 months.

"I believe they wanted to silence O'Brien - as he's about to do another conclave, and make a huge deal of it. As he's retiring, a decision's been taken to go public and take him down."

Another said: "If you're asking me to describe what this is about in one word, it's revenge. I've no doubt the allegations did take place in the 1980s but they've come out to - destroy O'Brien."

One clerical source said: "I can't answer for those who have complained and it could well be that their reaction [to the anti-gay rhetoric] was at the heart of this. I thought his words were very harsh and I'm not alone in that. There certainly were those who were close to the Cardinal, an inner circle.

"One particular priest was a very close friend of the Cardinal. It seemed to some to be a very unusual friendship."

A Catholic Church spokesman said: "Some clergy were not in favour of Church efforts to persuade the Scottish Government against same-sex marriage.

"It is also the case that objections were raised to Cardinal O'Brien's robust rhetoric.

"A number of complaints about Cardinal O'Brien were passed directly to the Vatican. Whether they were precipitated by his comments on homosexuality is not known, since the detail and nature of the complaints were not shared with the Bishops' Conference of Scotland."