The last to go was White House staff secretary Rob Porter, who resigned Wednesday after allegations of domestic abuse