After finishing up some research noting the wide gap between the quality of relief innings depending on the importance of the situation this season, it felt necessary to take a similar look at team performance. If teams were deploying less-good relievers in low leverage situations and good ones in high leverage situations, it could distort our sense of the quality of a bullpen when looking at overall numbers.

We’ll start with a pretty generic view of bullpens this year, with FIP by team:

The Cardinals have the lowest FIP of any bullpen this season, as the group as a whole has pitched very well. The Rays coming in second and first in the American League is somewhat of a surprise given their use of an opener in half their games; they are losing about 60 good relief innings and replacing them with around 180 good-but-not-as-good starting pitching-type innings. The teams fall down in a nice cascade the rest of the way, with the Baltimore Orioles providing a a very heavy base at the bottom of baseball.

But not all innings are created equal, and some of the innings pitched by bullpens are more important than others. If we separate meaningful innings (medium leverage and high leverage) from less important innings (low leverage), we can get a sense of how good a team’s bullpen is when it matters. This also could provide a better sense of which teams might be better prepared for the playoffs, given the consolidation of relief innings in October:

If you wanted to know how the Giants can have a run differential of negative 54 and be a 56-70 team by BaseRuns but still end up at .500, this graph explains it. When the Giants have had important relief innings, they have performed incredibly well. The Red Sox bullpen maybe hasn’t been quite as bad as it has seemed, while the two overall leaders in St. Louis and Tampa Bay have moved closer to the middle of the pack. If you want to know why Pittsburgh has collapsed, their bullpen performance is a huge part of the reason.

Here are the respective performances in low leverage situations:

St. Louis’ depth in the pen has put it at the top of baseball, while the Dodgers’ depth has masked the struggles the team has had in close games a bit. It’s the performance in these situations that makes Baltimore’s pen the worst in the game. As to how the performance in important situations relates to those in less important plate appearances, the answer is not much. Here’s a scatter plot of each team’s FIP:

The two figures aren’t correlated. Some of that is random, and some of it is that different pitchers are assigned to different types of situations; the pitchers put in close games are generally going to be better. Within teams, the number of innings in each situation can vary quite a bit. The graph below shows the percentage of a team’s bullpen innings that happen in low leverage situations:

The Rays are at the very bottom, likely due to their use of an openers, while the other teams at the bottom have had their bullpen pitch in more meaningful situations as a percentage of their overall bullpen use. The very top of the list includes some very bad teams with very bad bullpens. In terms of just innings, the leaderboard looks like this:

We still see mostly bad teams in the lead, with mostly good teams throwing fewer low leverage innings. The Angels and Rays have used an opener a decent amount of the time, which is going to explain how they are ranked highly in sheer number of innings but weren’t near the top when it came to the percentage of low leverage innings.

Would you be interested in a stat the helps capture some of the discrepancies above? A stat that factored in whether a team was pitching in important situations to give a greater sense for just how a strong or weak a bullpen might be? This isn’t a trick, though if the answer is yes, there’s a fairly simple answer. WAR factors in leverage for relievers, so the pitchers pitching in important situations get a little more credit than ones pitching in unimportant situations. We have to assume that pitchers are mainly used according to their talents, but general pitcher performance in low leverage situations compared to medium and high leverage situations seems to indicate this is the case.

WAR also controls for park and league, but if you don’t quite know what to make of the stats above on a team-by-team basis or how to weight different types of appearances, let WAR be your guide. Here are the current WAR leaders for bullpens by team:

The Yankees, due to their prowess in important situations, rate higher than other clubs with better FIPs. The Cardinals take a small hit, and teams like the Pirates move down a bit as well due to difficulty getting important outs (in addition to park and league factors). The NL East looks like it is putting together the worse divisional bullpen in baseball, with all five teams in the bottom seven. There are pretty definitive tiers, with the good teams around four wins or more, the middle class of teams around two wins starting with the Giants and moving down to the Pirates, and the poor bullpens starting with the Blue Jay down to the Marlins. The only contenders in the bottom group are the four NL east clubs vying for a playoff spot and the Chicago Cubs, who are hoping Craig Kimbrel can right the ship.