AUSTIN—A scandal at the heart of the Texas House of Representatives has thrown Republicans in the state into turmoil and led to jitters over whether it could be the straw that breaks the party’s longstanding hold on power here.

Democrats, who have been gaining in popularity as demographics in the state shift, need to win nine seats to gain a majority in the Texas House for the first time since 2001. A pickup of those seats would give them a say in redrawing district lines for the state legislature and U.S. Congress after the...