We’ve all heard it time and time again. “So-and-so pitcher is a #X starter”. And that’s where the debate starts. We all seem to have a different idea of what constitutes of certain spot in a rotation. Scouts have a certain idea of what a guy is. Stats guys have a different idea. General fans have a different idea.

Yesterday Eno Sarris wrote an article at Fangraphs and mentioned that he believed that Brandon Finnegan could become the Cincinnati Reds ace in 2017. That sparked a bunch of conversation in the comments. In particular the idea I’ve floated out there several times: Homer Bailey was a very good #2 pitcher in 2012 and 2013. Many comments back-and-forth revolved around “he was”, “he wasn’t” and some of us explained why we believed our viewpoint. Some of us, well, didn’t really provide much information beyond “he wasn’t”, but they knew he wasn’t.

I HATE the idea that a gut feeling can be evidence, or proof of something. Especially when we can actually use data to answer the question.

The Scouts take on pitching roles

This is simply a guideline, of course. I’ve got the Baseball America Prospect Handbook here and in the opening pages it gets into the roles of players. Let’s see what they define the roles as, as far as pitchers go:

#1 Starter: Clayton Kershaw

#2 Starter: Johnny Cueto

#3 Starter: Unlisted (an elite reliever is listed here)

#4 Starter: Mike Leake

#5 Starter: Tommy Milone

I’ve long said the ideas that scouts use to determine what role a starting pitcher has is simply unrealistic for the game today. Johnny Cueto, for example, was the 5th most valuable starting pitcher in Major League Baseball in 2016. From 2014-2016 he’s the 9th most valuable pitcher in Major League Baseball. Mike Leake, listed as a #4 starter, has been the 49th most valuable pitcher in baseball over the last three seasons (by Fangraphs WAR).

There are 30 teams in baseball. The guy who is the 49th most valuable starting pitcher in baseball over a three year span being a #4 simply isn’t set in reality. The ideas simply don’t match reality. Maybe the ideas matched up in 1950 when there were 16 teams and there were only four starters on each team. That meant you had 64 starting pitcher spots in total. Today we’ve got 30 teams and five man rotations. There are 150 starting pitcher spots at any given point in a season.

With all of that said, let’s use stats, in a scouting way, to determine what roles should be defined as with their actual performance. When scouts fill out a scouting report, they grade pitchers skills on the 20-80 scale (some teams use a 2-8 scale – but it’s essentially the same scale without the half-grades). They then give a pitcher an overall grade, based on the skills that they grade by adding them all up, and dividing by the number of things that they graded. They are also given some leeway to add/subtract a bit from this overall grade based on intangibles they see.

To get started, we need to set up what it is that we want to actually look at. First, we need to limit it to actual starting pitchers. For this, I went to Fangraphs and sorted the 2016 season by innings pitched. Then I looked at anyone with 80+ innings pitched and also had starts. That left us with 140 total pitchers. That’s close enough to the 150 we need to set the “starters” group since some teams, like the Reds, have a handful of guys in-and-out of the rotation after only a few starts. The stats I believe we need to look at, include Innings, walk rate, strikeout rate, strikeout-to-walk ratio and run prevention (adjusted ERA for ballparks played in).

We need to understand the 20-80 scale. It’s actually a scale based on a similar system used in sciences and maths. An 50 grade is considered Major League average. Each jump/decline of 10 points represents a standard deviation in that direction from average. At the extremes of the scale, a 20 or an 80 grade skill is representative of less than 1% of the entire sample. An example for each “80 grade” in the above stats is below, along with the number of pitchers who actually were able to do it:

Innings Pitched: 267 – ZERO Walk %: 1.7% – ONE Strikeout %: 33.2% – ONE Strikeout-to-walk radio: 7.5 – ONE ERA-: 33.6 – ZERO

If we are using the pure scientific method of an 80 grade being three standard deviations above the average, this breaks the scale. 80 is supposed to represent the best player in a category. For innings and ERA, that player simply doesn’t exist.

What I did was hand out the grades, using the scientific method of standard deviations from average in each of those categories. I then used it to come up with an “overall” grade using the add all of the grades together and divide by the number of grades handed out. Here is how the Top 10 pitchers in baseball, by this formula, come out for the 2016 season, and their overall grade:

Name Overall Grade Clayton Kershaw 72 Max Scherzer 65 Madison Bumgarner 64 Justin Verlander 64 Chris Sale 64 Noah Syndergaard 63 Jose Fernandez 63 Rick Porcello 63 Jon Lester 61 Johnny Cueto 61

That doesn’t seem too out of whack from what I would imagine you’d come up with if you had to guess who the ten best starters were from the 2016 season. It’s also very apparent that Clayton Kershaw is simply better than everyone else alive at pitching. If you’d like to, you can see the entire spreadsheet here.

If we are going to use the true scouting scale, and use the overall grades I’ve come up with, then there’s not a single #1 pitcher in baseball. Clayton Kershaw is the closest, but he falls short of that 75-80 grade that would be needed. It also means that there are only eight #2 caliber pitchers in the game – everyone from Rick Porcello and above on the list. That, is obviously, a very ridiculous thing to try and say. No one would agree with that. But that’s how things break down here.

If we break down things into groups of 28 pitchers, which would separate the 140 guys in this sample into five groups, here’s how the #1, #14 and #28 guy would break down:

Top Middle Bottom Reds? #1 Kershaw Tanaka Archer None #2 Tomlin McHugh T. Anderson DeSclafani #3 Hamels Ross J. Guerra Straily #4 Wright F. Hernandez E. Rodriguez Finnegan #5 Peralta Holland Locke None

Now, when we look at this, one thing pops out immediately: Felix Hernandez being listed as a #4. No one believes that to be the case. At the same time, he had a down year in 2016. He was average, or below-average in every category we looked at except ERA-. He rated out quite low in both walk and strikeout-to-walk ratios. It’s why we shouldn’t look at just one season when making these determinations. One season doesn’t make a pitcher. Two seasons is better and three is even better. The larger the sample, the better. But, when you make the sample larger, you also get more noise. Pitchers get hurt – so you’re going to see some swings in grades on guys in a lot of cases. Still, the above breaks it down for the 2016 season.

You can change things a little bit if you’d like. Personally, I think that innings is the most important part of all of this. If your team can rely on you for 190+ innings it’s insanely valuable. That is unless you somehow manage to get that amount of innings with terrible run prevention (guys don’t – if they stink they are pulled). It’s vastly more valuable than a guy with better run prevention skills who can only give you 160 innings because they can’t stay on the mound.

This is just one way to look at ti. And honestly, this article ran much longer than I initially planned for it to. And I left out a big chunk of what I initially planned to write about. I will end the article by giving the “average” stats for each grouping from above for each role.

IP BB% K% K/BB ERA- #1 187.1 5.9% 25.7% 4.3 75 #2 171.1 6.4% 21.1% 3.5 90 #3 157.2 7.4% 21.0% 2.9 97 #4 150.0 8.5% 18.6% 2.3 108 #5 123.2 8.8% 16.9% 1.9 126