• Moderators, please send me a PM if you are unable to access mod permissions. Thanks, Habsy.

Are all SV% the same? (Importance of consistency for goalies)

number17

number17

Mod Squad
This started off as a post on the Andersen thread, but I think this topic (and the work I've poured into it) deserves its own thread :)

We all know SV% is a key performance indicator of goalies. And we also know that SV% stays relatively consistent for the same goalie when he goes from good team to bad team, so it is a good 'goalie stat' and not a 'team stat'.

And when we started looking at goalies and compared their SV%, we realize any goalie whose SV% is .915 or above is better than average, we know goalies who finish below .915 are bad, and goalies who see their SV% ~ .900 are terrible (yes that's you Jonathan)

However is SV% the be all and end all for goalies? Are 2, say, .920 goalies the same?

I started looking at CONSISTENCY of goalies, meaning how many good starts vs. bad starts they give their teams. I heard people rave about Andersen's consistency and I decided to pour a little more stats work into it.

So here's what I did. I took a # of goalies, and I count the # of starts (GS only, not relief duty) in each of the 6 buckets:

1. Shutout
2. Game-Stealing start (.950 - .999)
3. Good start (.930 - .950)
4. Average+ start (.915-.930)
5. Poor start (.900-.915)
6. Disastrous start (Sub .900)

And I hope by looking at what kind of starts these goalies give to their teams, we'll get a better understanding of the value of these goalies, and the impact they have on their team.

And which goalies did I choose for this analysis?

Well I picked Andersen (duh) ... not just Andersen's 1516 season, but I looked at all 3 NHL seasons by Andersen to see if there's a progression.

I also picked Holtby, Bishop and Quick from the 1516 season. Why? Because they were the Vezina candidates.

I picked Cory Schneider because I think he's one of the best young goalies in the league.

I also picked Price, but not his 1516 season, it's his 1415 season instead because I think Price is the best G in the game today, and 1415 was his best year (and Vezina year)

I also added Jonathan Bernier from his 1314 season, which is his best season as a Leaf.

The result? Here you go ...

Screen Shot 2016-06-30 at 6.29.08 PM.jpg

From here I think we can draw a few conclusions


- 1, and it should be the biggest disclaimer, and that is Andersen's last season with 37 starts is WAY SHORT of the start all the other goalies had. He missed a # of games due to injuries and sickness, and the biggest risk for the Leafs is whether Andersen can repeat the same kind of solid performance when he is our #1 next year, getting 50-60 starts at least.

- 2. Andersen has improved every single season for his first 3 years in the NHL. His SV% might have gone up and down, but he's giving his team a lot more quality starts, and controlling on the # of bad starts. His "Great Starts", "Good Starts", "Poor Starts" and "Terrible Starts" #'s are ALL trending in the right direction, over the past 3 years. That is IMPRESSIVE.

- 3. In Andersen's final season (yes, yes, I know, 37 GS) his consistency and his quality starts compared very favorably against other Vezina candidates. You can argue in fact he was better than Quick and Holtby and was head to head vs. Bishop

- 4. Andersen's last season was way better than Bernier's 1314 season - in which we thought we finally had a quality starter.

- 5. Price is indeed the best goalie, and his shutout % in that 1415 season was crazy. That said, he was tied with Andersen in 27% "terrible starts", and his "average minus starts' were actually worse than Andersen's. Again, small sample size, I understand.


So what is more important? Which is a 'better goalie'? One who has more absolutely stunning nights and also more terrible starts? Or one who gives you a more consistently solid effort night in and night out? I think in today's NHL, the latter is more important. And that's why I think 2 goalies with the same SV% are not necessarily equal.

This study also shows Dubas & Co have done their homework, and the biggest risk right now is whether Andersen can repeat (or better, IMPROVE) his 1415 performance over a heavier workload next season. He HAS started 52 games before, in which his #s actually looked very respectable too. And as a young goalie he's gotta have improved since the 1415 season now. I think the study gives me plenty of confidence in Andersen as our future #1.
 
I'm not sure this perfectly nails the final target.....but I think there's likely something down this path well worth exploring.

A 'quality start' type measurement to help quantify the consistency of a goalies quality, and the lack of variance in their performances game to game.

Then the question is how much weight to apply to said measurement.

Intrigued to see where some of the minds around here can take this, from your jumping off point.
 
It's an interesting analysis. There could be something here.

To get a better sense of the validity of the measure, it needs to be applied across a wider range of goalies. I think the way you've done it looks pretty good. For instance, bunching shutouts with >.930 games.

There's two reasons to see if this works well with a larger range of goalies and data:

1) Is individual game save percentage a solid measure of performance in an individual game? It could be, I'm not sure. One fluke goal, depending on the number of shots, could be the difference between a 'great' and 'poor' performance. Save percentage has always struck me as a stat that needs a good number of games to have validity. I'm not saying that's so, it's just the way I've always looked at it.

2) If individual game save percentage is a good measure of performance, what are the best ranges to categorize performances? These look pretty good. I would want to fiddle with the data a bit first though.
 
Maybe the missing piece is some analysis of just what a given save percentage in a game does to the odds of winning that game? Gotta weight the importance of a save percentage on a game to game basis before figuring out how important a variance is
 
Aberdeen said:
What I would be interested in is a sort of standard deviation type statistic.
Click to expand...

Well, if you have the population mean (so enough data for what's the mean save percentage for all goalies over let's say 10years), you can figure out the pop. standard deviation from that..and then you can compare an individual goalies save percentage to that data..and you'll see where the individual goalie fits in. Ie what is the probability of having that save percentage, their percentile score, etc
 
Aberdeen said:
Pffft, too much work.
Click to expand...

You don't necessarily have to do that. If the sample is large enough, you can compare a couple or more samples against each other, and it will be representative of the larger pop.

The main thing is to figure out what IV we will have, or factor(s), and what we are measuring.
 
Last edited:
Montana said:
I'm not sure this perfectly nails the final target.....but I think there's likely something down this path well worth exploring.

A 'quality start' type measurement to help quantify the consistency of a goalies quality, and the lack of variance in their performances game to game.

Then the question is how much weight to apply to said measurement.

Intrigued to see where some of the minds around here can take this, from your jumping off point.
Click to expand...

Something as simple as percentage of games with a .915+ SV% would be an interesting stat.
 
Habspatrol said:
Something as simple as percentage of games with a .915+ SV% would be an interesting stat.
Click to expand...
Honestly, even something as simple as 2 goals allowed or less in the game, regardless of shots even, is probably decent enough.

I mean, even if you only give up 3 goals on 40 shots, you maybe kept the team close but odds are you still get the loss. And even if you gave up 2 goals on only 20 shots, most of the time that's barely enough to get the win unless if the other goalie steals the game.

The whole purpose of the "quality start" is basically how often did they play and give your team a legit chance of winning, and IMHO 2 goals is about the break-even point.
 
UWHabs said:
Honestly, even something as simple as 2 goals allowed or less in the game, regardless of shots even, is probably decent enough.

I mean, even if you only give up 3 goals on 40 shots, you maybe kept the team close but odds are you still get the loss. And even if you gave up 2 goals on only 20 shots, most of the time that's barely enough to get the win unless if the other goalie steals the game.

The whole purpose of the "quality start" is basically how often did they play and give your team a legit chance of winning, and IMHO 2 goals is about the break-even point.
Click to expand...

I kinda think that's going in the wrong direction though. That's maybe an improvement on GAA but still not as good as SV%.
 
I understand that but I just don't think I'd see much value in a stat that's based on GAA and ignores SV%.

.
 
There is actually a quality start metric out there that includes both:

A great stat to look at to help determine how effective a goalie is at putting his team in a position to win a game is the quality start metric. The concept of a "Quality Start" is taken from Rob Vollman's Hockey Abstract and functions like the statistic in baseball. In order to earn a quality start, a goalie must achieve a Save Percentage of at least .917 or stop 88.5% of shots while allowing two goals or fewer.

...

Mrazek (75.9%), Crawford (75.6%), Schneider (71.8%), Elliot (70.0%), Reimer (68.2%), Holtby (67.6%), Lundqvist (66.7%), Bishop (66.7%), Andersen (65.0%), Halak (62.5%).
Click to expand...

http://goaliepost.com/default.php?page=ramblings&id=46&season=2014
 
If you guys give can give me factors or independent variables (ie factors that are in your opinion the most relevant in determining the goalies overall save percentage (something quantifiable)..I'll test it.

I don't have the best idea for the factors..but here is how I'd conceptualize and test it...so you get an idea of what factors you can suggest and what I'd do with that information.

Ie What impact does a team scoring first have on overall goalie save percentage.
I'd first put all the teams in order: the team with the most first goals all the way to the team with the least.
Then I'd split them in two groups. So we'd have a group of teams who got the most first goal/their goalies with a certain save percentage in the first group, and the a group who got few first goals/their goalies with certain save percentages.
Then I'd test the means/estimated standard error and see if there's a big difference..based on that factor.

Again, this might not be the best factor to test, and this could even be improved conceptually (maybe), but you get the idea!

And we can do even 2 or 3 factors at the same time.
 
Leafovic said:
If you guys give can give me factors or independent variables (ie factors that are in your opinion the most relevant in determining the goalies overall save percentage (something quantifiable)..I'll test it.

I don't have the best idea for the factors..but here is how I'd conceptualize and test it...so you get an idea of what factors you can suggest and what I'd do with that information.

Ie What impact does a team scoring first have on overall goalie save percentage.
I'd first put all the teams in order: the team with the most first goals all the way to the team with the least.
Then I'd split them in two groups. So we'd have a group of teams who got the most first goal/their goalies with a certain save percentage in the first group, and the a group who got few first goals/their goalies with certain save percentages.
Then I'd test the means/estimated standard error and see if there's a big difference..based on that factor.

Again, this might not be the best factor to test, and this could even be improved conceptually (maybe), but you get the idea!

And we can do even 2 or 3 factors at the same time.
Click to expand...

Wouldn't it be better to use Win/Loss as the dependent variable rather than SV%? What goalie variables are best at predicting the win/loss outcome.

Close games might be better, as w/l is heavily dependent on factors outside the goalie's control. Maybe there's a 'gave the team a chance to win' variable (e.g., within one goal) = quality start.

Just thinking out loud.
 
Leafovic said:
If you guys give can give me factors or independent variables (ie factors that are in your opinion the most relevant in determining the goalies overall save percentage (something quantifiable)..I'll test it.

I don't have the best idea for the factors..but here is how I'd conceptualize and test it...so you get an idea of what factors you can suggest and what I'd do with that information.

Ie What impact does a team scoring first have on overall goalie save percentage.
I'd first put all the teams in order: the team with the most first goals all the way to the team with the least.
Then I'd split them in two groups. So we'd have a group of teams who got the most first goal/their goalies with a certain save percentage in the first group, and the a group who got few first goals/their goalies with certain save percentages.
Then I'd test the means/estimated standard error and see if there's a big difference..based on that factor.

Again, this might not be the best factor to test, and this could even be improved conceptually (maybe), but you get the idea!

And we can do even 2 or 3 factors at the same time.
Click to expand...

Like I said earlier... just a simple percentage of games with a .915 or better would be pretty interesting. Though I'd imagine it wouldn't really change the order a whole lot in comparison to straight SV%.
 
You must log in or register to reply here.
Back
Top