Ramping up rhetoric against migrants has won the Tories votes at times but at other points it has alienated swathes of the public, annoyed business leaders and caused the party to tie itself in knots.