Yesterday on my day off I decided to spend some of my leisure time compiling a database of 7 NHL drafts from 1997 - 2003. I wanted to do a pre-lockout study because the missed season skews proper time series analysis if your goal is to assign a numerical value to how a draft pick turned out. I downloaded the data at www.hockeydb.com which included NHL draft numbers, including the career NHL games and points of each player for a total of 1934 draft picks.
If you want to measure the success of a pick from 97 against one from 03, you can't look at career numbers alone because the older pick has had more seasons to accumulate games and points. I solved this problem by dividing the career numbers by the number of seasons since the player was drafted. This gives you an average such that you can do statistical comparisons of players from different years.
The next step is to set an expected number of games and points that you would anticipate from a player selected on pick X. The pool of available talent drops of considerably from round 1 to round 2, stays steady for round 3, it drops again and there is very little change after the 4th round. Expected games falls 66% from 1st round to 2nd round. The probability of getting a player in round 4 is roughly equal to round 8. The best forecast model to fit the data is a logarithmic scale.
E[GP] = (-9.31*ln(pick number))+ 52.54
That expected games played is a seasonal average. Then you can measure +/- as to how many Expected games under or over any given draft pick was. If a 5th rounder plays 100 games over 5 years after being drafted, that is a better hit than a 1st rounder who plays 150. One exceeded expectation and one was worse than expected. The +/- statistic will measure performance against expected value.
The next statistic of note is plain and simple batting average. Yes or no, did a given player play at least 10 NHL hockey games? That was another reason to cut off the database at 2003, was to leave enough time for the players to get a shot. I am going to guess that 99% of the players drafted in 2003 who have still not played an NHL game never will. I use 10 games as the limit to reach rather than 1, because that means the player was good enough to last a few weeks and signals that the player was NHL calibre (however marginal that might be). Here are batting averages by round for all draft picks. If a guy survives to 10 games, that's a base hit.
Continuing with the baseball analogies, the next statistic I'll call slugging percentage. How many games did a hit produce? It is pretty simple, take a sum of the games played averages and divide it by the number hits. That tells you how far you hit the ball when you made contact. Here is slugging percentage by round.
Again there is very little deviation in slugging after the 1st round. That provides 3 statistics to measure how well a GM does when he's at the plate. Percentage of times he hit the ball, how he did against expectation, and how far he hit the ball when he made contact.