BOSTON, England—The changing face of this east coast market town helps explain why many want to leave the European Union that Britain has spent the past 43 years helping to shape.

Resident Andrew Fraser said when Britons go to the polls on June 23 to vote on the country’s membership, he will be voting for Britain’s exit, or “Brexit,” because he believes a sharp rise in immigrants to Boston, where he has lived for most of his life, has radically transformed the town’s character.

...