SANDY SPRINGS, Ga.—Republicans held on to a hotly contested U.S. House seat in Georgia on Tuesday, beating back an aggressive challenge that showed the Democrats’ inability to turn opposition to Donald Trump’s presidency into electoral gains.

Republican Karen Handel, a former Georgia secretary of state, beat Democrat Jon Ossoff, a onetime congressional aide, in the most expensive House race in history and the most significant test of the two parties’ political strength since Mr. Trump’s election.

In nearly complete results, Ms. Handel had almost 53% of the vote to just over 47% for Mr. Ossoff, the Associated Press reported.

In winning the seat, Republicans overcame a Democratic advantage in campaign spending and demonstrated that Mr. Trump retained political capital in the district. The president, Vice President Mike Pence and other party luminaries visited the Atlanta suburbs to support Ms. Handel’s candidacy.

The result was a big blow to Democrats, who were hungry for a victory to demonstrate that grass-roots, anti-Trump energy gives them a shot at taking control of the House in the 2018 midterm elections. Democrats earlier this year lost two other contested House special elections, in Kansas and Montana.