SANDY SPRINGS, Ga.—Republicans held on to a hotly contested U.S. House seat in Georgia on Tuesday, beating back an aggressive challenge that showed the Democrats’ inability to turn opposition to Donald Trump’s presidency into electoral gains.

Republican Karen Handel, a former Georgia secretary of state, beat Democrat Jon Ossoff, a onetime congressional aide, in the most expensive House race in history and the most significant test of the two parties’ political strength since Mr. Trump’s election.

In...