Advanced Hockey Stats Discussion

[Robin Hood]

being an exception to a theory is proving a shortfall in a theory. Often where the most progress is made: it's support for the null-hypothesis that corsi/PDO is not a measure for expected outcomes. Although -> in a grand-population sense, it's a useful measure, it just cannot cross levels.

Disagree. An outlier doesn't imply that the theory is flawed. Also, the Leafs have not proven to be an outlier yet, so we're not even there yet.

Furthermore, if a model has a 50-60% predictive power (heck even 70%), there will be calls that will be "wrong". That doesn't mean the model is flawed. It just means there is room for improvement.

[Robin Hood]

Btw - a hockey analytics panel from last year featuring Brian Burke from the MIT conference. I'm sure some of you have seen it already.

http://www.sloansportsconference.com/?p=4550

[Nick]

in the pursuit of making corsi an applicable measure we're talking about:
-removing face-offs
-controlling for zone-start
- removing the final 2min/5min of 1 goal games
-removing special teams
------------------------------------
at some point haven't we removed too much information? Last i saw the R squared value for Corsi could be made as high as .12 - which is good - but no where near how some people talk about it. and by removing all of the above, and with 3 years worth of data, it can explain 60 something percent of variance. I dunno, on a committed betting basis I'm sure it's great, but from predicting player outcomes on a fantasy hockey basis I see it well behind more easily understood methods (like watching + previous g/a stats).

[Robin Hood]

in the pursuit of making corsi an applicable measure we're talking about:
-removing face-offs
-controlling for zone-start
- removing the final 2min/5min of 1 goal games
-removing special teams
------------------------------------
at some point haven't we removed too much information? Last i saw the R squared value for Corsi could be made as high as .12 - which is good - but no where near how some people talk about it. and by removing all of the above, and with 3 years worth of data, it can explain 60 something percent of variance. I dunno, on a committed betting basis I'm sure it's great, but from predicting player outcomes on a fantasy hockey basis I see it well behind more easily understood methods (like watching + previous g/a stats).

This is what Bruyns and I were talking about earlier with regards to the Leafs and how our Special Teams have been elite.

People are working on ways to include some of the things you are talking about btw. Zone starts and faceoffs in particular are getting a significant amount of attention. And there are actually some theories on how to introduce said data already out there (I was reading up on something around faceoffs yesterday).

As we go along those things will hopefully be added and the accuracy increases from 60 ish percent (which in itself is incredibly high for a chaos-oriented game like hockey).

[MSP4LYFE]

Care to expand?

Almost all I have come across fully admit the stats should be taken with a grain of salt and still place a huge amount of value on watching the games. Haven't seen many arguements from people saying don't watch games all you need to know are these stats, but I have seen a shit load of people although probably the majority are teenagers on HFboards claim the stats have no value, don't understand them and say ridiculous things like "watch the game" as if people taking time out of their day to discuss hockey on the internet don't watch it on TV.

What I meant by an open mind is taking the time to read and educate one's self on what they mean since I am positive there are plenty of people who have no idea or only a vague understanding of different stats, but still choose to chime into the debate like their opinion is valid when they have no clue what they are talking about. You know what they say though... ignorance is bliss

There are many stats geeks, particularly those at PPP who subscribe to the "my way or the highway" approach to advanced stats. That is to say that they treat anyone who does not heavily endorse advanced stats as a retard. Endorsing one way of thinking as a single objective truth is no more close minded than ignoring stats altogether, IMO.

[MSP4LYFE]

Burn!

First paragraph was well put, I have no problem admitting when I am wrong and people that struggle with this can come across as foolish. No problems with Shiv's stance either though as he is passionate in his beliefs and has no current evidence to dissuade him from the opinion he holds.

I don't have a problem with Shiv either, I was just trying to get a rise out of him, lol.

[MSP4LYFE]

Any statistician worth their lunch can set up numbers to reach a desired outcome - back to my original point on the importance of front-end theory needing to be logically founded with proper understanding of it's scope.

To put this in a simpler way, (and please correct me if I am taking you out of context), the conclusion doesn't logically follow from the premises. There is a logical disconnect between the two.

[MSP4LYFE]

I have a big issue with the underlined statement.

Before I start - Kareem just took a shot at me above. Let's stop that. Moving on:

Here is my issue:

What we "know" is from how we watch the game. A good example is a fight. Conventional wisdom says:

1. A fight can turn momentum
2. Fight when you're down, and pump up your team
3. Fight when you're up, and you might help the other team

Now I have NO idea whether or not this is right, I'm just using it as an example here.

To evaluate this conventional wisdom, stats can play a huge role. Using Corsi as an indicator, for example: Does a team's SOG differential/60 minutes increase after a fight? Does the SOG differential/60 increase more when it the game is close?

What we "know" is not CLOSE to being verified. We do not know how much value a fight has. A lot of what you mentioned is subjective. Does a guy like Burrows throwing a hit help you more than a guy that spends most of his shift in the offensive zone?

This is one argument or rather, type of argument, that I take issue with. We have decades worth of personal accounts that insist that fighting has an impact on the players in the game. Though it may be true that fighting has more or less of an impact in certain situations on aggregate, it does not follow that the impact of fighting is overstated. To do so would be to completely neglect the importance of our sensory perceptions, and there are alot of skeptical arguments that would follow from that. This is counter-intuitive, and one example where I believe that stats over-extend their boundaries.

To use a similar example against qualitative analysis, I do not believe it is possible to accurately judge a defensive players value relative to the rest of the league. There are just far too many plays, and far too many players to assess with the naked eye and cross analyze. That isn't to say that we can't assess the strength of a good defenceman vs a bad defenceman, I think there are always extreme cases that are really easy to assess. Instead, I think the problem occurs with assessing that middle tier (the vast majority of players), and cross analyzing them with different players. I also believe this is one of the reasons that so many peoples list of top defenceman differ so greatly. The only way I can think of solving this problem is by creating an objective criteria for defenseman, and applying it equally among all defenceman, much like defensive metrics in baseball. The human eye just isn't very good at accomplishing this task.

[Robin Hood]

I have a big issue with the underlined statement.

Before I start - Kareem just took a shot at me above. Let's stop that. Moving on:

Here is my issue:

What we "know" is from how we watch the game. A good example is a fight. Conventional wisdom says:

1. A fight can turn momentum
2. Fight when you're down, and pump up your team
3. Fight when you're up, and you might help the other team

Now I have NO idea whether or not this is right, I'm just using it as an example here.

To evaluate this conventional wisdom, stats can play a huge role. Using Corsi as an indicator, for example: Does a team's SOG differential/60 minutes increase after a fight? Does the SOG differential/60 increase more when it the game is close?

What we "know" is not CLOSE to being verified. We do not know how much value a fight has. A lot of what you mentioned is subjective. Does a guy like Burrows throwing a hit help you more than a guy that spends most of his shift in the offensive zone?

This is one argument or rather, type of argument, that I take issue with. We have decades worth of personal accounts that insist that fighting has an impact on the players in the game. Though it may be true that fighting has more or less of an impact in certain situations on aggregate, it does not follow that the impact of fighting is overstated. To do so would be to completely neglect the importance of our sensory perceptions, and there are alot of skeptical arguments that would follow from that. This is counter-intuitive, and one example where I believe that stats over-extend their boundaries.

To use a similar example against qualitative analysis, I do not believe it is possible to accurately judge a defensive players value relative to the rest of the league. There are just far too many plays, and far too many players to assess with the naked eye and cross analyze. That isn't to say that we can't assess the strength of a good defenceman vs a bad defenceman, I think there are always extreme cases that are really easy to assess. Instead, I think the problem occurs with assessing that middle tier (the vast majority of players), and cross analyzing them with different players. I also believe this is one of the reasons that so many peoples list of top defenceman differ so greatly. The only way I can think of solving this problem is by creating an objective criteria for defenseman, and applying it equally among all defenceman, much like defensive metrics in baseball. The human eye just isn't very good at accomplishing this task.

I think you misunderstood me. I personally think that fights do make a difference.

My point was that data can be used to illuminate whether our subjective beliefs about the game are actually accurate. I just used fighting as a dummy example.

[MSP4LYFE]

To use a similar example against qualitative analysis, I do not believe it is possible to accurately judge a defensive players value relative to the rest of the league. There are just far too many plays, and far too many players to assess with the naked eye and cross analyze. That isn't to say that we can't assess the strength of a good defenceman vs a bad defenceman, I think there are always extreme cases that are really easy to assess. Instead, I think the problem occurs with assessing that middle tier (the vast majority of players), and cross analyzing them with different players. I also believe this is one of the reasons that so many peoples list of top defenceman differ so greatly. The only way I can think of solving this problem is by creating an objective criteria for defenseman, and applying it equally among all defenceman, much like defensive metrics in baseball. The human eye just isn't very good at accomplishing this task.

I think I need to provide an example to make this argument more clear. Suppose that Dion Phaneuf makes a nice poke-check at the last second to dispossess Ovechkin before he gets off a shot. Intuitively it would seem that Dion Phaneuf made a good play, but what makes a good play good? I think most would agree that terms like good and bad are relative to the rest of the league. Therefore, if we accept this premise, we must likewise agree that in order for Phaneuf's play to be deemed objectively good it would have to be the type of play that the average defenceman or perhaps even most defenceman don't make. But how can we know that for certain? All we know is that Dion Phaneuf in a particulaur circumstance made a particular play. We do not know that another defenceman would or would not have made the same play. Perhaps another defenceman would have been better positioned, and thus would not have had to make a last ditch poke-check at all. Which raises another question, was Dion Phaneuf's play good, or bad because he had to make a last ditch effort to dispossess Ovechkin? Perhaps another, or even the average defenceman would not have had to make that play because they would not have allowed Ovechkin to get into a scoring lane to begin with. This is but one instance that demonstrates the difficulty of analyzing defensive ability, and further cross analyzing with the naked eye.

[MSP4LYFE]

I think you misunderstood me. I personally think that fights do make a difference.

My point was that data can be used to illuminate whether our subjective beliefs about the game are actually accurate. I just used fighting as a dummy example.

I understood your point just fine, I just thought it was a good example to demonstrate the types of analytic arguments I do not like.

[Robin Hood]

To use a similar example against qualitative analysis, I do not believe it is possible to accurately judge a defensive players value relative to the rest of the league. There are just far too many plays, and far too many players to assess with the naked eye and cross analyze. That isn't to say that we can't assess the strength of a good defenceman vs a bad defenceman, I think there are always extreme cases that are really easy to assess. Instead, I think the problem occurs with assessing that middle tier (the vast majority of players), and cross analyzing them with different players. I also believe this is one of the reasons that so many peoples list of top defenceman differ so greatly. The only way I can think of solving this problem is by creating an objective criteria for defenseman, and applying it equally among all defenceman, much like defensive metrics in baseball. The human eye just isn't very good at accomplishing this task.

I think I need to provide an example to make this argument more clear. Suppose that Dion Phaneuf makes a nice poke-check at the last second to dispossess Ovechkin before he gets off a shot. Intuitively it would seem that Dion Phaneuf made a good play, but what makes a good play good? I think most would agree that terms like good and bad are relative to the rest of the league. Therefore, if we accept this premise, we must likewise agree that in order for Phaneuf's play to be deemed objectively good it would have to be the type of play that the average defenceman or perhaps even most defenceman don't make. But how can we know that for certain? All we know is that Dion Phaneuf in a particulaur circumstance made a particular play. We do not know that another defenceman would or would not have made the same play. Perhaps another defenceman would have been better positioned, and thus would not have had to make a last ditch poke-check at all. Which raises another question, was Dion Phaneuf's play good, or bad because he had to make a last ditch effort to dispossess Ovechkin? Perhaps another, or even the average defenceman would not have had to make that play because they would not have allowed Ovechkin to get into a scoring lane to begin with. This is but one instance that demonstrates the difficulty of analyzing defensive ability, and further cross analyzing with the naked eye.

But this is exactly why stats are so important. Everything you just wrote is what is extremely difficult to "quantify" when it comes to stats in sports.

However, what we can do is look at Dion's body of work and say, when Dion is on the ice, his team directs X amount of shots on goal and gives up Y amount of shots against. Then compare that with the rest of the league/the average.

Now, there are still missing factors here. For example, Dion's minutes are significantly different from say Paul Ranger. Based on this, we can confidently say that evaluating Dion vs the whole population is a difficult task using JUST Corsi because the whole picture isn't painted.

But we can illuminate a part of the picture that we couldn't before and with a certain level of accuracy.

[MSP4LYFE]

The Dion example was an argument against traditional analysis, and for analytics. Not vice-versa. The shots example is still far too basic an argument, and one with far too much noise for me to seriously endorse it.

[Robin Hood]

The Dion example was an argument against traditional analysis, and for analytics. Not vice-versa. The shots example is still far too basic an argument, and one with far too much noise for me to seriously endorse it.

You don't think it should be used at all? Or used by itself? Because those are two different things.

[MSP4LYFE]

The Dion example was an argument against traditional analysis, and for analytics. Not vice-versa. The shots example is still far too basic an argument, and one with far too much noise for me to seriously endorse it.

You don't think it should be used at all? Or used by itself? Because those are two different things.

Neither, I just don't think it's a very compelling argument at all. It's a very basic analysis, but my hope is that it leads to more compelling and advanced forms of analytic thought.

[Robin Hood]

The Dion example was an argument against traditional analysis, and for analytics. Not vice-versa. The shots example is still far too basic an argument, and one with far too much noise for me to seriously endorse it.

You don't think it should be used at all? Or used by itself? Because those are two different things.

Neither, I just don't think it's a very compelling argument at all. It's a very basic analysis, but my hope is that it leads to more compelling and advanced forms of analytic thought.

I disagree strongly with this. Strong statistical models do not need to be built on something complex. I agree that there is definitely room for improvement, but that doesn't mean what exists today is too "basic".

[MSP4LYFE]

I disagree strongly with this. Strong statistical models do not need to be built on something complex. I agree that there is definitely room for improvement, but that doesn't mean what exists today is too "basic".

The current analysis is not basic because it is not built on an advanced model, it is basic because it relies entirely on shots. That is the definition of basic. That doesn't mean that it is useless, but I am of the belief that people are trying to draw too many meaningful conclusions from it. If the game of hockey were as simple as taking more shots then I may have a future as a head coach after all.

[Robin Hood]

I disagree strongly with this. Strong statistical models do not need to be built on something complex. I agree that there is definitely room for improvement, but that doesn't mean what exists today is too "basic".

The current analysis is not basic because it is not built on an advanced model, it is basic because it relies entirely on shots. That is the definition of basic. That doesn't mean that it is useless, but I am of the belief that people are trying to draw too many meaningful conclusions from it. If the game of hockey were as simple as taking more shots then I may have a future as a head coach after all.

Btw - if you're interested, read this paper. I've got a few more if you like it. Doesn't relate to what we're talking about exactly but it's a good read.

http://www.sloansportsconference.com/wp ... layers.pdf

[Nick]

advanced stats are nice for adding details and follow-up evidence.

Watching Shea Weber I think he's frigging amazing and in another league from most dmen. In that I believe many dman in the same circumstance as him would fail on my fronts, whether it be the point production, shot generation, defensive play, durability or physical play - there are very few dman IMO who do every in sumation as well as he does, anywhere near as consistently. Now he's rarely watched and has a much smaller fan base than other elite dmen (OEL only other super-small, southern american market elite-ish dman) we all see what Subban, Suter, PieT do - perhaps not an entire game scrutiny like we put Karlsson/Dion/ Letang/Daughty through, - but by mass accumulation of top 15 in points, reputation, some small sample of watching actual game, highlights and advanced stat metrics (qualcomp, PDO, corsi adjusted) my belief that he's the best in NHL is strengthened.

Also reminds me of the intense scrutiny we put all dmen through - to be called the best you first must be #1 or #2 in scoring, be in major fan base, and be exciting - it's about the story not about the actual defensive play. Easily the first trophy (Norris) that I started to ignore (and before Weber was skipped). I wouldn't let Corsi tell me that, confirming or denying.

[MSP4LYFE]

I disagree strongly with this. Strong statistical models do not need to be built on something complex. I agree that there is definitely room for improvement, but that doesn't mean what exists today is too "basic".

The current analysis is not basic because it is not built on an advanced model, it is basic because it relies entirely on shots. That is the definition of basic. That doesn't mean that it is useless, but I am of the belief that people are trying to draw too many meaningful conclusions from it. If the game of hockey were as simple as taking more shots then I may have a future as a head coach after all.

Btw - if you're interested, read this paper. I've got a few more if you like it. Doesn't relate to what we're talking about exactly but it's a good read.

http://www.sloansportsconference.com/wp ... layers.pdf

Awesome, thanks. I will have a look a little bit later tonight.