Story highlights Ryan's recent flap with Trump may put his job on the line

Ryan didn't withdraw his endorsement of the GOP nominee, but said he would no longer defend him

(CNN) Speaker Paul Ryan is facing backlash from House Republicans over his flap with Donald Trump — and his own job may be on the line.

Rep. Jim Bridenstine of Oklahoma — a conservative who voted for Ryan last year for speaker— is threatening to pull his support if the Wisconsin Republican won't fall in line behind the GOP nominee.

"Given the stakes of this election, if Paul Ryan isn't for Trump, then I'm not for Paul Ryan," Bridenstine tweeted Wednesday.

Other conservative Republicans have also flashed their anger toward Ryan over his position that he wouldn't defend or campaign with Trump, raising the specter that Bridenstine could be the first in a crowd of conservatives rebelling against the speaker.

The GOP has been gripped by infighting since a 2005 video surfaced last week showing Trump describing women in vulgar and sexually aggressive terms. Ryan hasn't withdrawn his endorsement of Trump but he did tell House Republicans Monday that he will no longer defend the nominee and will devote the remainder of the campaign season to helping Republicans in House and Senate races.

Read More