Roofgate 2002

Let's begin with some history, sine it has long been known - since Bank One Ballpark - that Chase Field is more hitter-friendly with the roof open.The first time it came to public attention was early in 2002. Curt Schilling picked up his first loss of the year on April 17 against the Cardinals, Jim Edmonds ripping a fifth-inning grand-slam off him, in a game played at BOB with the roof open. Schilling asked then owner Jerry Colangelo if the roof could be closed, and this was duly done for his next home outing on May 3. Curt won, beating the Expos, but fans were incensed and the team reversed its policy, returning to having the roof open whenever possible and compatible with fan comfort.

Said long-time broadcaster Greg Schulte, "More was made out of it than really what should have been, but, at the same time, Schilling saw that difference." And, indeed, paid the price. According to the Philadelphia Inquirer, "Schilling was the object of public ridicule, including a less than flattering editorial cartoon in the Arizona Republic that portrayed him as a sloppy, overweight, spoiled brat." The paper also reports, when the Phillies came to town later that month, Schilling was teased by manager Larry Bowa and coach John Vukovich during batting practice. "You want us to close this roof?" Bowa asked Schilling, as Vukovich made mock crying sounds in the background.

After that game - where he again won with the roof open - Schilling had his final say: "I prepare to pitch to win. They pay me to win, and I try to give myself the best possible chance. I just thought that that would give me an advantage... You guys have made that into a Watergate scandal, and it really should never have been. He's the owner of the team. The roof is where he wants it to be... . It's filled reams of paper and the airwaves making me out to be something I'm not. I've never been spoiled. I don't make demands. I never have. Anybody here that knows me knows that." Certainly, it's not a request anyone appears to have made since.

What difference does it make?

To figure out how the roof affects play, I looked at almost 54,000 plate appearances by the Diamondbacks at Chase Field since the franchise began. I used only the home team at-bats, to avoid introducing another potential variable, in the shape of the strength of the opposing hitters. Overall, here is what the results showed, through the eighteen seasons from 1998 through to the end of last year. [Note: the two 2014 games in Sydney are excluded, and I've divided games based entirely on the roof state, regardless of whether or not the outfield panels were open, as there is no reliable information available on that] And whaddya know - Curt was right.

Roof open: .274/.346/.455 = .802 OPS, 5.08 runs per game

Roof closed: .265/.334/.427 = .761 OPS, 4.56 runs per game

That's a significant impact, more than half an extra run per game for the D-backs with the roof open. This result is in the same ballpark, if you like, as a study in 2006 by the Tucson Citizen, which found an extra 1.17 runs per game, i.e. on both sides, played beneath the sky. Of course, this is largely an equal-opportunity creature, and the visiting side's hitters will also enjoy a similar boost. However, it appears to enhance slightly home-field advantage. With the roof open, the Diamondbacks have had a winning percentage of .533; with it closed, that drops to .524. Given about 30 open games per year at Chase, that's an extra edge for us, of about a quarter-win over it being shut all the time.

Breaking it down

We can also break the results down to individual events, and see whether they are more or less likely to happen with the roof open or closed. To simplify the numbers, I've put all events on the same scale, where 100 = "happens as often with the roof open as closed." If the number is above 100, the event happens more often with the roof open; if it's below 100, it is less frequent under the sky.

R H 2B 3B HR BB SO SH SF HBP 109.1 102.6 102.0 94.8 120.9 107.5 94.2 100.9 92.7 94.8

As noted, runs per plate-appearance are up about 9%, and there doesn't appear to be any significant overall difference in the number of hits and doubles. What we do see, however, is a drop in triples with the roof open, and a very sharp spike - over 20%! - in the rate of home-runs hit. One presumes the former is the result of balls which would carom around off the outfield walls, now going over them. Why would having the roof open make a difference here? A resulting change in the air currents around the park is likely a factor, but it may also partly be due to higher temperatures. Yale University physics professor Robert Adair, author of The Physics of Baseball, told the Tucson Citizen:

"Roughly speaking, in a 10-degree-warmer temperature, the ball will go 3 or 4 feet farther. That doesn’t sound like much, but if you think of all the balls on the warning track, and all the balls that just go over and land in the first row of the bleachers, you’ll find that three or four feet actually adds to the probability of hitting a home run more like 5 to 7 percent."

One curious and unexpected effect was the walks are more common, and strikeouts down, with the roof open. That's a surprise, because both of those are decided entirely over the 60'6" between the pitcher and hitter, and it seems unlikely air currents would have much impact there. I'm guessing here, but perhaps the different contrast and lighting conditions experienced with the roof open, are helpful to the batter seeing the ball out of the pitcher's hand? I note hit batters are also lower, by about the same amount as strikeouts, which could give credence to that theory.

What changed in 2009

"The last couple of years it's been really strange. There have been nights where the roof has been open and the ball has died in the outfield. There have been nights when the roof has been closed and the ball has really carried, so it's all theory."

-- Greg Schulte

However, if you look at the trend in results, something even more surprising stands out. From when the park opened, through the end of 2008, the effect of the roof was steady: every season, OPS was higher with it open than closed. The amount varied, from 20 to 96 points of OPS, but it was always there. However, since the beginning of 2009, the impact of the roof seems to have declined markedly, to the point where in the last two seasons, OPS has been higher with it closed. Overall, through 2008, the roof open OPS was an average of 54 points higher, but in the past seven seasons, that has dropped by more than two-thirds to just 16 OPS points. The table and graph below summarize these:

OPEN CLOSED DIFF 98-15

.274 .346 .455 .802 .265 .334 .427 .761 .041 98-08 .280 .352 .465 .817 .266 .335 .428 .764 .054 09-15 .264 .335 .439 .774 .264 .332 .426 .757 .016

So, did something happen in 2009? It seems whatever was quietly changed at Chase Field severely dampened the impact of the roof being open or closed - to such an extent that veteran Schulte, who has likely watched more games here than just about anyone, appears to have noticed, going by his comments above. To see if we can figure out what it was, let's break down the numbers for individual events, as we did above, but dividing them separately into the the period before 2009, and also from 2009 through 2015.

R H 2B 3B HR BB SO SH SF HBP SH SF HBP 1998-08 115.3 103.9 99.7 105.5 123.7 109.7 91.3 101.5 104.3 91.6 101.5 104.3 91.6 2009-15 98.6 100.2 106.5 79.3 115.2 103.5 100.1 97.6 71.3 100.0 97.6 71.3 100.0

That's interesting, but perhaps even more confusing. Runs and hits have come back down to being almost even (from 2009-15, runs have been 4.60 per game open, and 4.62 closed), regardless of the state of the roof. While Home-runs have dropped, they are still significantly higher with the roof open. But what has happened is the rate of triples has collapsed. Over the earlier period, the mean number of triples per season at Chase with the roof open was 11, and we saw anywhere as high as 20. But even that average figure has only been reached once since, with last year's Peralta-powered dozen.

This may just be random fluctuations in the noise, but I also note that the doubles rate upticked, suggesting the triples are now becoming two-baggers. I'm open to suggestions as to what could trigger something like this (maybe overall better outfield defense? But the overall number of league triples doesn't seem to have changed much). If you're interested, the Google spreadsheet with all the data is here.

Conclusions