BIRMINGHAM, Ala.—Democrat Doug Jones of Alabama on Tuesday scored an upset win in a deeply Republican state, capturing the U.S. Senate seat here in a special election that drove a wedge within the Republican party and gave Democrats another burst of momentum ahead of the 2018 midterm races.

The result was a defeat for President Donald Trump, who had endorsed Republican Roy Moore, and for his former White House strategist, Steve Bannon, who had made this the first major test of his strategy of supporting antiestablishment candidates...