Changing sweaters does change D man

Today at www.tsn.ca hockey analytics writer Travis Yost wrote an article titled "changing sweaters doesn't change the D man" were he attempts to prove exactly that in terms of relative Corsi,(change in team's shot differential when he's on or off the ice). He's trying to refute the claim that relative Corsi changes significantly when a player is moved from one team to another. Evidently he takes a lot of flak for criticizing trades based primarily on relative Corsi. I'm assuming this flak can get pretty nasty, otherwise TSN would allow comments below his piece.

If you want to read the entire article there is a link above. I would like to respond specifically to the analysis portion included below.

If you were to believe the “all the player needs is an environmental change” argument, you would think that driving play for a defender is not a repeatable skill, simply because it would collapse like a house of cards the moment a player changed teams.

So, let’s test that theory. Let’s look at the RelativeCorsi% of every single defender who logged two seasons with one team, then changed teams and logged another pair of seasons. We’ll do this for every two-season sample from 2007-08 through this season. And we’ll also adjust for zone starts to control for players at the ends of the spectrum who either logged lofty offensive zone minutes or tough defensive zone minutes.

First, did the player’s skill repeat across different teams?

If RelativeCorsi% was purely indicative of team effects, we would expect this to be pretty flat. Instead, we see repeatability even across different teams and different seasons. Those players who favourably drive play with one team continue to do so with another team; those players who get shelled on the shot clock with one team continue to do so with another team.

How he came to this final conclusion is flawed. He says that if individual contribution to relative Corsi were nothing and team effect were everything, the line would be flat. The problem is, just because it isn't flat does NOT mean there is a strong correlation. The R^2 for that data is only 0.27. With truly predictive statistics, the data will fit very tightly to the line (by comparison, player point per game production from one season to the next has an R^2 of roughly 0.68 (for players playing over 20 game in each season)).  The data set in the Yost rant has significant variability, enough to prevent any substantive conclusion. His argument seems to be "because it is not 0%, it must be 100%." All this data shows you is a weak positive relationship, nothing more, nothing less. He's making strong definitive conclusions based on weak correlation.

The bottom line is that any given player 5 on 5 is just one of 10 skaters on the ice. His impact, positive or negative, is somewhere around 10%. Some players directly affect the games more than others, but shot differential is heavily influenced by who you are playing with and who you are playing against. Relative Corsi Against might be a useful statistic for certain players in the right context, but would you agree that Josh Manson and Fedor Tyutin are the best defensemen in the NHL? Corsi is loaded with false positives and false negatives, and distinguishing between the two is far more complicated than you might think.

More on that here