Being kept out of the White House for another four years may not even be Republicans’ biggest worry in 2016. Now that Donald Trump is the GOP’s de facto standard-bearer, the party faces the prospect that it could lose more than just the general election (and its identity)—it could also lose control of the Senate.

Republicans currently hold a four-seat advantage, meaning Democrats will need to net four seats (if they also win the White House) or five (if they don’t) to retake the upper chamber. That’s no easy task, but this year’s electoral map has long looked particularly friendly for the party—even before Trump made his grand entrance this past summer. (Thanks to partisan gerrymandering and geographical quirks, Republican control of the House seems safe for now.)

Of the 34 Senate seats up for grabs this fall, Democrats currently hold only 10, and most of those are extremely safe. This means that they’ll be able to play significantly more offense than defense ahead of Election Day. According to the Cook Political Report’s most recent ratings—published in March—only three of the 10 Democratic seats are in any danger of falling into Republican hands: The races in Nevada and Colorado are expected to be competitive, while the race for Barbara Boxer’s soon-to-be-vacant seat in California has the potential to become competitive down the road. Things are much less rosy for Republicans. According to Cook, the GOP is currently at risk of losing seven seats, and another six hold the potential to end up in the competitive column before it’s all said and done.

And that’s where Trump comes in. There’s already been plenty of handwringing among Republicans about the Donald’s drag on down-ballot races, and justifiably so. The rank and file will need to decide whether to run away from Trump and risk alienating his base, or run toward him and risk alienating everyone else. Right now, many are trying to have it both ways. New Hampshire’s Kelly Ayotte—up for re-election in one of the seven Senate races seen as a toss-up—is among those tying themselves in knots to play both sides. As her spokesman put it to the New York Times recently, Ayotte will “support” Trump but not “endorse” him. (Say no more, senator!) Toss in concerns that Trump will put the GOP fundraising apparatus at a significant disadvantage, and that he will depress turnout among Republicans this fall (when turnout models will already favor Democrats), and it’s easy to see the Senate map turning blue.

Consider the potential impact of Trump’s loudly stated views on immigration and immigrants. All three of the Democratic seats at stake in competitive races are in states where Latinos make up a potentially game-changing slice of the electorate. According to Pew, Latinos represent 28 percent of eligible voters in California, where Boxer is retiring; 17 percent in Nevada, where Harry Reid is doing the same; and 15 percent in Colorado, where Michael Bennet is fighting for re-election. It’s a similar story in two competitive races for seats held by Republicans: Latinos make up 18 percent of eligible voters in Florida, where Marco Rubio isn’t running for re-election; and 11 percent in Illinois, where Mark Kirk is. Mitt Romney’s “self-deportation” comments caused headaches for some Republicans four years ago; it stands to reason that Trump’s proposal to build a wall—as beautiful as it may be—will give them migraines.

Even John McCain has privately told supporters that he thinks he’s in for “the race of my life” with Trump as the nominee. “If you listen or watch Hispanic media in the state and in the country, you will see that it is all anti-Trump,” the Arizona senator told donors at a private event last month, a recording of which was obtained by Politico. “The Hispanic community is roused and angry in a way that I’ve never seen in 30 years.” Polls back that up. According to a Gallup survey from last month, 77 percent of Hispanics have an unfavorable view of Trump compared to just 12 percent who felt the opposite.

Trump’s appeal (or lack thereof) isn’t just a problem in states with large nonwhite populations, either. He is also polling poorly in a trio of whiter states where a Republican incumbent is running in a race considered a toss-up by Cook: Pennsylvania, where Pat Toomey is up for re-election; Wisconsin, where Ron Johnson is facing a high-profile challenge from former Sen. Russ Feingold; and New Hampshire, where Ayotte is squaring off with Gov. Maggie Hassan.

Add it all up and it’s easy to see why Republicans are so worried. As for Trump, well, he doesn’t think very highly of Congress anyway.

Also on the Slatest:

Read more of Slate’s coverage of the 2016 campaign.