NHL Draft: Where Success Equals Failure

At what point in the NHL draft does the probability of success equal the probability of failure? To answer this question we’re going to look at the drafts from 2004 to 2010 and player salaries at age 23, 24, 25. The O22 - U26 is the window between entry level contracts and unrestricted free agency; I call it my draft analytics sweet spot. Success here is defined as a salary over a million dollars post entry level, and failure is 0 NHL GP and less than 30 AHL GP any given season in the same O22 - U26 window. Perhaps the bar for success was set about $300,000 too high, but I’m a fan of the million dollar threshold. Statistics from the 2016/17 season have not yet been included in my database.

The reason for choosing the drafts from 2004 to 2010 because we’re only interested in contracts signed under a salary cap, and cut it off at 2010 so we can see a six year window after the players are drafted. In salary based draft analytics you can’t use data from before salary cap because entry level contracts were a game changer. So if you want to look at that post-entry level pre-UFA (O22-U26) window to measure pick value, you’ll get a limited sample size (in this case 7 drafts). I also prefer to use information from more recent drafts to capture more recent trends in drafting, but like “Kid Icarus”, you need to be weary of flying too close to the sun. The more recent the data, the less information you have to determine success rates; but the older the info, the less relevant it becomes to current trends.

The answer to the question “where does success equal failure” is the 28th pick. After that you are more likely to fail than succeed. Below is an illustration of the probabilities of success and failure for all picks in the draft. You can see that the probability of getting a million dollar player drops very quickly in the first round, and starts to level off after the 2nd round. Statistically there is not much difference between a 4th round pick and a 6th round pick. Fitting a logarithmic trend line in Excel to both the success and failure rate data produces a high correlation, though the success line hits Y=0 at the 162^th pick. All picks after 162 should have a non zero chance of producing a million dollar player. The blue line is the failure line, which I also refer to in baseball terms as strikeouts (or Ks).

Does NHL Shot Differential Matter?

If you have been following the evolution of the hockey analytics community, you’ll notice we have come to a place where many of these people strongly believe that shot differential is the most important measure of the value of hockey players. Putting grades on trades and other transactions only requires an examination of Corsi and its derivatives. They’ll say things like this was a terrible trade because the player has a bad Relative Corsi Against.

I’m a hockey stats nerd who wants to see the hockey analytics revolution succeed, but I have always been skeptical of “shots at goal plus minus” (aka Corsi) as a meaningful statistic. They call it a “possession” statistic, but it doesn’t actually count how long a team is in possession of the puck, and it assumes that all shots are created equal.

Placing ultimate importance on shot differential assumes that it actually matters which team gets the most total shots with no measure of quality of shooting location. It begs the question, how often does the team that shoots the most shots win the hockey game? A smart hockey brain might say something like 65%. If it were 50%, then the final shot differential would have no impact on the outcome of the game. I looked at a random sample of 1100 NHL games, and the team with the most shots won 50.9% of the games. In the playoffs it drops to 40%.

Hold on a minute, getting more shots in the regular season only increases the probability of winning by 1%? In the playoffs the losing team tends to get the most shots? In the 2016 NHL playoffs the 5 games with the largest shot differentials saw the losing team get the most shots (top being Washington losing with 44 shots to Philly’s 11). If shot differential is so important, why doesn’t it have a more positive effect on wins and loses?

There are possible theories. Such as, many teams play differently when they are leading or trailing on the scoreboard. They may hold an extra forward back to box out the middle and limit shots from best percentage scoring areas. In doing so they would hinder their own team’s shot production by attempting to limit the opponent’s opportunities. This would hurt the Corsi ratings for the players who are winning, while teams playing from behind take more risks and shoot more shots (though not always from higher value locations), thus giving the perception of greater value. It’s a theory.

Corsi is a flawed rating system that rewards quantity of shots over quality of shots. It gives players greater value to pepper the net with shots from everywhere instead of making extra passes to get the puck into a higher percentage scoring location. Goals are a function of quality scoring chances more so than quantity of shots. A shot from the red line has the same Corsi value as a breakaway. I never understood why Corsi is supposed to be a better judge of value than regular old fashioned plus minus. Perhaps the sample size is smaller for goals, but goals data is far more important than shots data. I would much rather know who was on the ice for every goal rather than who was on the ice for every shot.

If normal plus minus does not provide us with enough information, then Corsi supplies us with too much. Perhaps the optimal statistic is waiting untapped in the middle of these two extremes. Quality Scoring Chance Plus Minus. That would require a precise definition of quality scoring chance and people to count them, but would be a superior measure of value. In the proper context shot differential and Corsi can tell a story, but should not be the primary statistic for assigning value. Because if the Florida Panthers think they got a quality asset in Jakub Kindl, this Red Wings fan will tell you, don’t hold your breath…