Kennedy ruffled some feathers early in her tenure, but during her time, the conservative Abe and liberal Obama found common ground