A potentially explosive report has linked the resignation of Pope Benedict XVI to the discovery of a network of gay prelates in the Vatican, some of whom – the report said – were being blackmailed by outsiders.

The pope's spokesman declined to confirm or deny the report, which was carried by the Italian daily newspaper La Repubblica.

The paper said the pope had taken the decision on 17 December that he was going to resign – the day he received a dossier compiled by three cardinals delegated to look into the so-called "Vatileaks" affair.

Last May Pope Benedict's butler, Paolo Gabriele, was arrested and charged with having stolen and leaked papal correspondence that depicted the Vatican as a seething hotbed of intrigue and infighting.

According to La Repubblica, the dossier comprising "two volumes of almost 300 pages – bound in red" had been consigned to a safe in the papal apartments and would be delivered to the pope's successor upon his election.

The newspaper said the cardinals described a number of factions, including one whose members were "united by sexual orientation".

In an apparent quotation from the report, La Repubblica said some Vatican officials had been subject to "external influence" from laymen with whom they had links of a "worldly nature". The paper said this was a clear reference to blackmail.

It quoted a source "very close to those who wrote [the cardinal's report]" as saying: "Everything revolves around the non-observance of the sixth and seventh commandments."

The seventh enjoins against theft. The sixth forbids adultery, but is linked in Catholic doctrine to the proscribing of homosexual acts.

La Repubblica said the cardinals' report identified a series of meeting places in and around Rome. They included a villa outside the Italian capital, a sauna in a Rome suburb, a beauty parlour in the centre, and a former university residence that was in use by a provincial Italian archbishop.

Father Federico Lombardi, the Vatican spokesman, said: "Neither the cardinals' commission nor I will make comments to confirm or deny the things that are said about this matter. Let each one assume his or her own responsibilities. We shall not be following up on the observations that are made about this."

He added that interpretations of the report were creating "a tension that is the opposite of what the pope and the church want" in the approach to the conclave of cardinals that will elect Benedict's successor. Another Italian daily, Corriere della Sera, alluded to the dossier soon after the pope announced his resignation on 11 February, describing its contents as "disturbing".

The three-man commission of inquiry into the Vatileaks affair was headed by a Spanish cardinal, Julián Herranz. He was assisted by Cardinal Salvatore De Giorgi, a former archbishop of Palermo, and the Slovak cardinal Jozef Tomko, who once headed the Vatican's department for missionaries.

Pope Benedict has said he will stand down at the end of this month; the first pope to resign voluntarily since Celestine V more than seven centuries ago. Since announcing his departure he has twice apparently referred to machinations inside the Vatican, saying that divisions "mar the face of the church", and warned against "the temptations of power".

La Repubblica's report was the latest in a string of claims that a gay network exists in the Vatican. In 2007 a senior official was suspended from the congregation, or department, for the priesthood, after he was filmed in a "sting" organised by an Italian television programme while apparently making sexual overtures to a younger man.

In 2010 a chorister was dismissed for allegedly procuring male prostitutes for a papal gentleman-in-waiting. A few months later a weekly news magazine used hidden cameras to record priests visiting gay clubs and bars and having sex.

The Vatican does not condemn homosexuals. But it teaches that gay sex is "intrinsically disordered". Pope Benedict has barred sexually active gay men from studying for the priesthood.