Commons Speaker John Bercow insists his impartiality has not been affected after he revealed he had voted Remain in the EU referendum.