Even the greatest of the great have places where their powers hold no sway. Superman has Krypton, Napoleon had Russia in the winter, and Michael Jordan has the fitting room.

For Tom Brady, that place is Miami, Florida. Against the other 30 teams in the league, Brady is 85-32 on the road in the regular season. In Miami, he is 7-9, giving him more losses in Miami than anywhere except New England. Hell, Brady has nine losses in 16 Miami games and 19 losses in 132 home games. Brady has thrown 15 interceptions in Miami, which is more than he’s thrown in any season of his career. No matter how destitute the Dolphins are or how unbeatable the Pats seem, a trip to Miami is the most dangerous thing for the team outside of a visit from Bernard Karmell Pollard.

As the Patriots head to Miami for their matchup Sunday, their history suggests that Patriots fans should be concerned about a loss—and maybe their Super Bowl chances too. Let’s run through the Pats’ losses in South Florida:

Last year New England visited the Dolphins while on an eight-game winning streak. The Dolphins won 27-20 as they forced seven three-and-outs and held Brady to his single-game lowest passer rating in four seasons and the Pats without a third-down conversion for the first time since at least 1991, when the league began tracking the statistic.

In Week 17 of the 2015 season, the Patriots visited Miami with a chance to seal the no. 1 seed in the AFC. Brady threw for fewer than half the yards as Ryan Tannehill while taking twice as many sacks, and the Patriots lost the game 20-10 and home-field advantage to the Denver Broncos.

In Week 1 of 2014, Brady threw a whopping 56 passes for a measly 249 yards (4.45 yards per attempt) while losing two fumbles in a 33-20 loss. The month ended with a crushing loss to the Chiefs and Bill Belichick being asked whether he would evaluate the quarterback position.

In Week 15 2013, Brady threw for 364 yards and two touchdowns but threw a game-sealing interception to safety Michael Thomas in a 24-20 Dolphins win.

In Week 13 2009, Brady threw a game-sealing interception with less than a minute left in a 22-21 loss.

Intermission: The one season Brady missed, 2008, the Dolphins won the AFC East in the lone disruption in 16 years of divisional hegemony.

In Week 14 2006, the Nick Saban–led Miami Dolphins held Brady to just 78 pass yards on 25 attempts as they throttled Belichick’s Patriots 21-0. (Miami has held Brady to under 100 passing yards four times. Since he took over as New England’s full-time starter in 2001, that’s happened only two other times: When he tore his ACL against Kansas City in 2008 and in a meaningless Week 17 game against the Bills in 2014.)

In Week 15 2004, the 12-1 defending champion Patriots visited the 2-11 Dolphins. Brady threw four interceptions, including two on New England’s final two drives and was outplayed by A.J. Feeley (!), who threw the game-winning touchdown pass on fourth-and-10.

In Week 5 2002, the defending Super Bowl champion Pats lost 26-13 in Miami as Brady threw two touchdowns and two interceptions. If the Pats had won, they would have won the AFC East. Instead, they finished in a three-way tie at the top of the division and missed the playoffs for the first—and last—time with Brady starting a full season under center.

In Week 4 2001, second-year quarterback Tom Brady, filling in for the injured Drew Bledsoe, led the Patriots to Miami for his second career start and first road game. The Dolphins shellacked the Pats, holding Brady to just 12 completions on 24 attempts for 86 passing yards as the team handed him his first loss as a starter.

“It’s been a challenging environment,” Brady told Westwood One’s Jim Gray on Monday. “I think the climate plays some role, and they always seem to play well when they play us down there. We are not going to get beat by the weather; we’re going to get beat by the Dolphins. That is what we realize. We’re going up against a good opponent that plays us very well at home. We play them very well at home. It is just one of those series where it is kind of like that.”

If the Patriots do fall to the Dolphins on Sunday, it could have serious repercussions for their season. Kansas City leads the AFC at 10-2, and New England is currently tied with the Houston Texans at 9-3 but holds the head-to-head tiebreaker for the second overall seed in the AFC. A loss to the Dolphins opens the door for the Texans (and maybe even the 7-4-1 Steelers, who face the Raiders this week and the Patriots in Week 15) to swipe a top-two seed and a first-round bye. The importance of earning the bye in the Belichick-Brady run cannot be overstated: The Patriots have secured a bye for the last eight consecutive years, which is a major reason they have made the AFC championship game each of the last seven years. (Seven. Consecutive. Years.) If the Pats beat the Dolphins, they have an 79 percent chance of getting one of the top two seeds and a bye, but if they lose that number drops to 45 percent, according to the New York Times playoff calculator.

Even if New England secures the second seed, losing home-field advantage to Kansas City could swing this season’s Super Bowl. New England is currently a game behind Kansas City for home-field advantage in the AFC (though the Patriots hold the head-to-head tiebreaker), and a loss Sunday could put the no. 1 seed out of reach. A win against Miami will give them a 2-in-5 chance for the no. 1 seed, but a loss will drop them to 1-in-10. That’s a huge deal, because home-field advantage has been the Patriots’ secret Super Bowl sauce. Tom Brady is 19-3 at home in the playoffs but 3-4 on the road. In the seven years the Patriots have earned the 1-seed, they’ve made the Super Bowl six times. But when they don’t earn the 1-seed, things can go awry.

In January 2007, the Patriots lost the AFC championship game to the Colts, 38-34, in Indianapolis’s RCA Dome. Both teams finished 12-4 in the regular season, but the Colts hosted the game because they held the head-to-head tiebreaker. If the Pats had not lost their Week 14 matchup with that Nick Saban Dolphins team, they could have been 13-3 and potentially hosted the Colts in the championship game. Nine years later, in January 2016, the Patriots lost the AFC championship game 20-18 to the Broncos at Mile High in Denver. The Patriots had a chance to clinch the no. 1 AFC seed and home-field advantage on a road trip in Week 17, but lost to … the Dolphins. If the Pats had won, they would’ve avoided playing in Denver, where Brady is 4-7 in the regular and postseason combined. Those two losses to the Dolphins may have done just enough to prevent two more New England Super Bowl appearances on top of what is already the best run in the history of pro football.

This week’s game could again factor heavily into the race for the 1-seed. The Chiefs are undefeated at home. Kansas City’s two losses this season both came on the road, including their 43-40 loss to the Patriots in Foxborough. The Patriots victory gives them the head-to-head tiebreaker against the Chiefs, which might prove decisive if the Pats catch the Chiefs in the standings. Kansas City’s final four games are against the Ravens, Chargers, Seahawks, and Raiders, while the Patriots finish out against the Dolphins, Steelers, Bills, and Jets. If they can’t catch the Chiefs, they should be worried: Brady’s Pats are just 1-2 at Arrowhead Stadium lifetime.

Of course, the Pats may crush the Dolphins—because they are still the Patriots and because the Pats have had some memorable wins in their 16 games there in the Brady era. In 2007, Brady had one of his two games with a perfect 158.3 passer rating while completing 21 of 25 passes for 354 yards, six touchdowns, and no picks as the Pats took a 42-7 halftime lead and won 49-28. In Week 1 of 2011, the Pats won 38-24 as Brady threw for a monstrous 517 yards and four touchdowns, including a 99-yard score to former Dolphin Wes Welker. And, crucially, one year after the Patriots blew their 2015 Week 17 trip to Miami that handed the no. 1 seed to Denver and possibly cost New England a Super Bowl appearance, the Patriots had a chance for revenge in Week 17 the following season. They rocked the Dolphins, 35-14, and Brady threw for 276 yards and three touchdowns. The Patriots secured the no. 1 seed in the AFC and beat the Houston Texans in the divisional round, the Steelers in the AFC championship, and then the Falcons in the Super Bowl.

It would be presumptive to say that how the Patriots fare in Miami on Sunday is going to determine whether they make it to the Super Bowl. But history shows that this is more than a normal away game for a team that’s typically been great no matter where it plays.