After an Athletic article dropped earlier this month that listed the 2018 Red Sox as another team who were stealing signs illegally using technology, a lot has been made about how the tech contributed to their success that year, stretching from apathy to cries of vacating the 2018 title. Alex Cora is out as manager, after he and the team reportedly parted ways mutually, with a 1+ year suspension likely coming his way from the role he played with the Astros prior to 2018. The rest of the Red Sox organization awaits a punishment decision.

The question I was most interested in, though, was one I hadn’t seen an answer to - did it matter?

In terms of integrity, of course it mattered. The sign stealing situation isn’t unlike the steroid use throughout baseball in the late 90’s and early 2000’s. Barry Bonds was one of the greatest players baseball the game had ever seen before he even touched a steroid and likely would have continued to have been an elite player without them, but he broke a written rule. It doesn’t matter how many people/teams around you are breaking the same rule(s), it matters that you broke them.

In terms of game scores, runs provided, and competitive advantages though, did it matter? A better way of asking this might be: did the Red Sox see a quantifiable benefit from stealing signs that would support the claim? If so, what would the change in record and perhaps playoff implications look like?

As a reminder, based on what has been reported,

“at least some players visited the video replay room during games to learn the sign sequence opponents were using” and would then signal the pitches from presumably second base later in the game should the opportunity arise. “Opponents were leery enough of sign stealing — and knowledgeable enough about it — to constantly change their sign sequences.”

Luckily for me, this should be easy enough to look into by analyzing the Red Sox performance when a man occupied second base vs. when there was no one on second base. Here’s how the Red Sox, as a team, performed at the plate in 2018:

6302 PA, 0.340 wOBA, 19.9 K%, 9.0 BB%

The 0.340 figure marked the best in baseball, with the team wRC+ trailing only the Yankees, Dodgers, and Athletics. Now we split it. Here’s how Boston performed with no one on second base:

4956 PA, 0.333 wOBA, 20.5 K%, 7.9 BB%

Definitely some slight difference there, which means we should see an uptick with a runner on 2B:

1324 PA, 0.365 wOBA, 17.8 K%, 10.7 BB%

Note that these two PA figures technically do not add up to the 6302 PA I listed before - it’s actually 22 PA shy. I’m not sure how or why they’re missing, but for some reason, there seems to only be 6280 Red Sox PA results in the Statcast pitch-by-pitch data from 2018. Either way, I’m not overly concerned that 22 fewer PA altered either outcome by a noticeable amount.

In terms of wOBA, we see a roughly 10% improvement in results while the Red Sox had a man on second base. With 1324 plate appearances (scientific trials, in a statistical sense), a .365 wOBA has something like a 1% chance occurring naturally in a random sample of 1300+ PA assuming a team skill level of 0.333. Which is to say that, yes, this 10% improvement is statistically significant. The claim that the 2018 Red Sox were sending signs from second base seems to hold up.

Notes: Interestingly enough, Boston’s difference between wOBA when there was a runner on second and wOBA when there wasn’t a runner on second (0.032 increase) was actually 3rd in magnitude in 2018. The Astros, who we know were running a sign stealing operation of their own just one year prior, were first with a 0.046. Second biggest increase in wOBA with a man on second in 2018 belonged to the Minnesota Twins, with a 0.045. The two biggest marks from 2019, both of which exceeded the 0.032 increase that Boston saw in 2018, were the Yankees and the World Series champion Nationals. Take these how you will.

Would it have changed anything?

This is a tough question, obviously, because there’s no way to definitively know. Even if we could scroll through every AB and could identify a sign given to the hitter from second base for each one, we can’t know for sure what the outcome of the AB would be without the sign. Even if we could, we couldn’t know what the outcome of the inning or game that came after that.

What we can do is check how many runs were “created” by the differences in those situational wOBA’s. This is also imperfect, but my be the best method we have for determining impact, so that’s how we’re going to tackle things.

What we want is a weighted runs above average for our 2018 Red Sox while a runner was on second. As a baseline, we’re going to use the 2018 Red Sox when a runner wasn’t on second base, assuming this is the level to which they would perform if not for illegal help from their teammates across the diamond.

Using the 1324 “on-2B” plate appearances, the “on-2B” wOBA of 0.365, the baseline “no-one-on-2B” 0.333 wOBA, and a wOBA scalar of 1.157, we get 36.6 wRAA.

That’s a lot of runs, to be sure. However, we still haven’t answered the question of what, exactly, if anything, would change without these additional runs. In 2018, the runs per win were a tad over 9.7, which means 36.6 wRAA would have created just over 3.75 wins. For the sake of round numbers, let's round it up to 4 extra wins caused by the pitch signalling from second base.

In 2018, the Red Sox won the AL East by 8 games. As I’ve mentioned before, we know absolutely nothing for sure, but given the ~4 additional estimated wins, it looks like the Red Sox still would have won the AL East without the help.

Playoff home field advantage, however, might be a different story. They would have had home field advantage over the 92-win Dodgers in the World Series, but behind the Boston in the league standing were their ALCS opponents, the Houston Astros, who finished with 103 wins - just 5 shy of the Red Sox.

Given our imperfect methodology in determining the impact of the Red Sox regular season sign stealing, we can certainly say that 5 games is within the realm of possibility. The Astros also held the tiebreaker, which means a tie in record would award them home field advantage instead of Boston.

In order to say that home field advantage for the Astros would have changed something though, we need to assume two things. First, we need to assume the Astros would have performed better at home with home field advantage than they did at home without home field advantage, where they won 0 out of 3 games at Minute Maid in 2018. The second assumption we need to make in this hypothetical is that the Astros weren’t stealing signs in 2018 also. They haven’t been mentioned explicitly, but given their history and the high difference in wOBA with a man on 2B, I don’t know if we can assume that, even if it hasn’t been proven.

All-in-all, it feels as though the regular season sign stealing from the 2018 Red Sox wouldn’t have changed much, unless you’re a believer in the butterfly effect, in which case we don’t actually know anything at all. They were, by all measures, an incredible team. They still would have won the AL East and probably would have maintained home field advantage in the ALCS over the Astros if both teams were playing fairly all season. The Red Sox certainly deserve some sort of punishment for playing their part in the sign stealing news that has unravelled in the past few months, but it seems as though they are also deserving of the 2018 World Series banner that will hang at Fenway Park forever as well.