For those not checking the rec list on the Fanposts today, Blazer's Edge member blacknoiseNW uses the NBA and Synergy's web sites to investigate Damian Lillard and CJ McCollum's defense compared to other points in the NBA. Interesting stuff. -- Tim

UPDATE: BlacknoiseNW reached out to Synergy about the discussions in the comments and they provided direct, interesting feedback. Their responses are at the bottom of this post.

------

These data are Synergy Stats available at NBA.com. While no defensive stat is perfect - Synergy data rates as very high quality and very different from composite statistical analysis such as +/- stats, or Win Shares, etc.

As far as I understand it, the Synergy data is generated through video analysis - where human analysts dissect every NBA play (college, too) and catalog the data. Access to Synergy data has typically required a fee - but now certain categories of data are available for free at NBA.com.

The data covers both offensive and defensive situations - but considering the generally negative perception of Lillard's defense (and to a lesser extent, McCollum's), I felt compelled to see how our two young guards compare to their peers. My list isn't comprehensive - but does cover the majority of top Point Guards, including Curry, Westbrook, and Paul.

Methods

NBA.com Play Types for Defense include the following:

Isolation When the possession-ending event is created during a "one-on-one" matchup. The defender needs to be set and have all of his defensive options at the initiation of the play. Pick & Roll: Ball Handler A screen is set on the ball handler’s defender out on the perimeter. The offensive player can use the screen or go away from it and as long as the play yields a possession-ending event, it is tagged as a pick and roll. Pick & Roll: Roll Man When a screen is set for the ball handler, and the screen setter then receives the ball for a possession-ending event. This action can include: pick and rolls, pick and pops and the screener slipping the pick. Post-Up When an offensive player receives the ball with their back to the basket and is less than 15' from the rim when the possession-ending event occurs. Spot-Up When the possession-ending event is a catch-and-shoot or catch-and-drive play. Hand-Off The screen setter starts with the ball and hands the ball to a player cutting close by. This enables the player handing the ball off to effectively screen off a defender creating space for the player receiving the ball. Off Screen Identifies players coming off of screens (typically downs screens) going away from the basket toward the perimeter. This includes curl, fades, and coming off straight.

Of this list, approximately 80% of a Point Guard's defensive possessions are from Isolation (9% average), P$R Handler (45%) and Spot Up (25%). For the purposes of this comparison - only these three possession types are evaluated.

Each category shows a variety of data, including percentage of an individual players plays within that category, Points Per Possession (PPP), and Percentile rank amongst all qualifying NBA players as a function of PPP. Other data such as eFG% allowed, foul rates, And-1's, etc. - are all captured in the PPP - so for ranking - I focus solely on the PPP.

RESULTS and DISCUSSION



Isolation Defense

This list includes player performances from multiple teams (e.g., Rondo; Dragic) - but the rankings do show McCollum and Lillard as above average in isolation, relative to the league, and average relative to the top point guards in the league. Overall, amongst PG's, Isolation defense is neither a strength nor a weakness, and considering the number of actual isolation defense plays per game, not overly important. In general, holding a player under 1.0 PPP is a good outcome for the defense, and Lillard/McCollum both do a good job in that regard. That qualifier is especially important in the Pick and Roll results.

Pick and Roll Ball Handler

Unequivocally the most important defensive task for a Point Guard, at nearly double the play type of the next closest category (Spot Up Shots/Drives). McCollum shows very well in this category, Lillard is below average - but not critically so. At #17 on the list, he is still only giving up 0.8 PPP - which is a good outcome for any type of defensive possession. Interestingly, Rubio is the worst Isolation defender and the best P$R defender, while Westbrook ranks poorly in both categories. Speaking of ranks, the list below Lillard should give us all pause before lambasting Lillard's defense. While we knew his P$R defense left something to be desired - it's still better than Rose, Parker, Conley and Westbrook.

Spot Up Defense

Again, Lillard shows up towards the middle of the pack, while McCollum drops a bit. Still - Lillard is holding players catching the ball and shooting or driving to less than 1.0 PPP - better than Chris Paul. This scenario is essentially a PG recovering back to his guy after helping out, and covers both defending an open shot and defending a player off the dribble. Considering this is the second most used play type on defense, Lillard's results are just fine. Not outstanding - but a whole lot better than "terrible".

Overall, this data tells us to cut Lillard and McCollum some slack - especially since Lillard is giving up less than 1.0 PPP for all three types. For comparison, the Blazers gave up 0.81 PPP as a team in Isolation, 0.78 PPP in P$R Ball Handler defense, and 0.94 PPP in Spot Up defense (good for 5th in the league).

Backcourt defense was NOT the weakness we assumed it was - and shouldn't be used as a generic "woe is our defense" disclaimer for next season.

EDIT:

First, I appreciate all the discussion - both positive and negative feedback. The overarching goal for a post like this is to challenge our perspectives on our team. I learned a lot about the Blazers through this exercise - and I also learned a lot about the data that helps inform our perceptions. Regarding Royster's objections - I went looking for similar objections elsewhere to see if his views were substantiated by analysis. I found a corresponding article at Nylon Calculus by Seth Partnow - and decided to see if Synergy had any feedback on the issue. I sent this email to Synergy yesterday:

I recently wrote a fanpost at Blazersedge.com comparing the defensive outcomes for various NBA point guards. The discussion on the post included challenges to Synergy’s data quality – and I’m wondering if you can provide any feedback on data reliability that I can pass along to the audience at Blazers Edge? In general, the complaints about the data fall into this category: Mislabelling and small samples weren’t even the most damaging biases introduced. The application only displayed possession-ending plays, which surely gives a skewed view of a player’s offensive accomplishments; what player doesn’t look like an expert shot creator if all you’re watching is plays where he shot? Carmelo Anthony is not credited for often drawing double teams in isolations, while some notable other players avoid censure for failing to create for themselves before throwing hapless teammates late shot-clock "grenades," forcing rushed shots under duress. And that same frame of reference exacerbated the inherent difficulties when looking at players’ defensive outputs, as the system would only display plays where the player in question was considered the primary defender. In addition to the inconsistency of the assignment as "primary defender", this excluded situations where a player’s defense was so bad, a teammate was forced to attempt to defend for him. Nor does it account for situations where a player’s defensive breakdowns put the whole defense in help-and-recover situations, leading to open shots elsewhere on the floor. My general counter to this position is that any process errors would apply to every player in the dataset, and that it is up to the data users to extrapolate context for the data (rather than simply challenging the data itself).

Synergy did, in fact, respond. They read this fanpost and the Partnow article, and then called me to discuss. Synergy acknowledged that context is the most difficult aspect of tracking. "Intent" is just not something that can be objectively quantified, and is an inherent limitation to any player tracking system, whether it is Synergy's data or SportsVu or what have you. Synergy stated that professional teams face similar challenges in how to evaluate and use tracking data such as Synergy and SportsVU - and that no one has "cracked the code" on developing a perfect tracking system that can capture all the nuances of a play.

However, Synergy did say that there is a specific error in the Partnow's objection - and that is that the primary defender at the start of the play is assigned responsibility for the outcome of the play - even if, for example - the defender is screened off the ball and someone else ends up helping out. Help defenders are NOT assigned responsibility for the play outcome. While the plays are tracked based on possession ending outcomes (by necessity - or limitations of the process), we can be assured that the data generated applies to the defender in question. That isn't to say that help defense does not affect the outcome for a primary defender - and Synergy did say that one way to then assess the impact of individual defense is to evaluate individuals within the context of team outcomes. So when we say that plays where Lillard is guarding the Ball Handler result in 0.8 Points Per Possession - and the Blazers give up 0.78 PPP as a team - we can see that while Lillard is hurting the team with his Ball Handler defense - the impact is less significant than perhaps we thought. Especially considering that as a team, the Blazers' 0.78 PPP ranked them as above average in this category - essentially tied with OKC, Memphis, San Antonio, etc.

Big thank you to Synergy for taking the time to consider the email and respond, even though they knew that this was "fan level" analysis. They were appreciative of seeing how the NBA.com data is being used.