Establishment’s tendency to look after its own laid bare in historical Whitehall papers showing signs of scandal that went ignored