There has been a lot criticism heaped on four newspapers, the Sunday People, the Sun, Mail on Sunday and the Independent, for not publishing what they knew about John Whittingdale's affair with a sex worker for five months between 2013 and 2014.

The insinuation is that it suited them to have a hold on him, given that he was so important to their welfare, first as chairman of the powerful Culture, Media and Sport Selection Committee, and since the middle of 2015 as Culture Secretary.

But the notion that there was a conspiracy is, I think, nuts.

Because it was never going to work.

If there is one thing I've learned in more than 30 years as a hack, it is that journalists can't keep a secret. They/we can't help ourselves - we gossip. So any deliberate cover up would always have failed.

What is clear to me is that this story was not published because the four newspapers failed to establish that it was a story - and the vendor of the story was asking a lot of money for it, £20,000.

If you doubt the absence of formal conspiracy, here is an interesting piece of circumstantial evidence.

The Sunday People was the first newspaper to be offered the story at the end of 2013.

It approached Tom Watson - the Labour MP, now deputy leader of the Labour party, then a colleague of Mr Whittingdale on the Culture committee - for his advice on whether it should publish.

He told them he did not see there was a public-interest reason to run the story on Mr Whittingdale's affair, since he was a single man, this was his private life, and the People had no evidence that Mr Whittingdale had paid for sex.

So a prominent MP who presumably would feel that embarrassing Mr Whittingdale was in his party's interest cautioned against publication.

Tom Watson (far right) was on the Culture, Media and Sport Select Committee, chaired by John Whittingdale (second left). Credit: PA

And there is the rub. There may be pictures of Mr Whittingdale on the London Underground with the woman, the sex worker.

But those pictures are consistent with Mr Whittingdale's account of events, that he had no idea of her occupation.

The point is that she was not the vendor of the story, and was not talking about the affair. This was no kiss and tell.

The newspapers had snaps and the testimony of a third party. But that hardly constitutes proof that Mr Whittingdale had behaved scandalously. The worst they could probably have said is that she was an unusual social partner for an MP.

That however does not mean that Mr Whittingdale himself is completely in the clear.

He was aware that the papers were in possession of salacious gossip about him - which could be seen, by the wider public, to give him a motive to keep on their good side.

The point is that the perception of a conflict of interest for him as culture secretary is strong and unambiguous - especially since, as I said yesterday, he has huge decisions to make about legal costs for newspapers and whether they undergo a second hacking inquiry.

So it is extraordinary that he didn't tell the Prime Minister or the Cabinet Secretary, Sir Jeremy Heywood, about the press's possession of the unpublished story, when he was offered the job of Culture Secretary just under a year ago.

That would have protected him from any suggestion that he would be or is anything but impartial in the conduct of his office.

Downing Street tells me Mr Whittingdale's omission isn't a sacking offence. But there is a strong sense of irritation that the PM was not kept in the picture (and not just because, as I've witnessed on a few occasions, he too likes a juice nugget of gossip).