I sometimes include a look at the previous week’s forecasts in my weekly posts, but I don’t think I’ve ever done a formal review of the forecasts at the end of the year before. Given the volatility of PWR, it sometimes seems kind of hard to believe that we can predict tight curves for where teams are likely to end up 10-12 games out, so a review of my success definitely seems in order.
Also, with the conference tournament results in, I’ll take a look at the nagging question of weighted vs unweighted forecasts for the conference tournaments.
A look back the big predictions
On January 15 in A first look at the NCAA hockey Pairwise Rankings I included this chart:
UND went 6-4-4 and emerged #7 in the PWR on March 4 (see Ranking trend charts). That outcome is right on the high end of the thick part of the curve for “Win 8”, which is the closest equivalent to win 6 and tie 4, so the forecast was pretty much spot on.
On the same day, I posted this chart of Minnesota:
The Gophers went 8-4-2 and were #2 in the PWR since mid-January. That’s just on the high side for “Win 10”. The forecast would have led readers to expect a ranking more in the range of 4-5 (between the Win 8 and Win 10 curves) for that performance.
On February 21, I posted this update for Minnesota:
By then it was more clear that Minnesota was sewing up the #2 spot. Indeed, Minnesota went 4-1-1 and finished #2.
Also on February 21, I posted this chart of Boston University:
They went 4-3 and were #16 in the PWR on March 11.
I also noted that Providence had some upside potential but needed to win.
Providence went 4-0-2 and finished #21 on March 11.
On weighted vs. unweighted conference tournament projections
There were some good questions asked about why I post the raw remaining possibilities for conference tournaments instead of weighted probabilities.
I responded that the real reason I do it is because the possibilities are factual, while the probabilities are sort of subjective. But, I also noted that: 1) it doesn’t matter much (conference tournament pairings tend to be of similar strength teams), and 2) KRACH didn’t really reflect the “hot” teams that tend to outperform in the conference tournaments.
After seeing QU lay an egg, CC go on a tear, and Michigan continue its hot streak, I thought it would be fun to run these numbers.
| Team 1 | Team 2 | Team 1 KRACH | Team 2 KRACH | Predicted Winner | Predicted Win Percentage |
Actual Winner | Prediction correct? |
| Niagara | Canisius | 60.0463 | 25.3254 | Niagara | 70% | Canisius | |
| Mercyhurst | Connectictut | 28.7672 | 34.8 | Connectictut | 55% | Mercyhurst | |
| Canisius | Mercyhurst | 25.3254 | 28.7672 | Mercyhurst | 53% | Canisius | |
| Miami | Michigan | 106.036 | 47.7859 | Miami | 69% | Michigan | |
| Ohio State | Notre Dame | 54.5277 | 97.1286 | Notre Dame | 64% | Notre Dame | Yes |
| Michigan | Notre Dame | 47.7859 | 97.1286 | Notre Dame | 67% | Notre Dame | Yes |
| Quinnipiac | Brown | 158.707 | 53.8762 | Quinnipiac | 75% | Brown | |
| Union | Yale | 68.2106 | 87.5804 | Yale | 56% | Union | |
| Brown | Union | 53.8762 | 68.2106 | Union | 56% | Union | Yes |
| Quinnipiac | Yale | 158.707 | 87.5804 | Quinnipiac | 64% | Quinnipiac | Yes |
| Mass.-Lowell | Providence | 109.609 | 74.2102 | Mass.-Lowell | 60% | Mass.-Lowell | Yes |
| Boston University | Boston College | 70.2589 | 103.267 | Boston College | 60% | Boston University | |
| Mass.-Lowell | Boston University | 109.609 | 70.2589 | Mass.-Lowell | 61% | Mass.-Lowell | Yes |
| St. Cloud State | Wisconsin | 90.3013 | 78.0123 | St. Cloud State | 54% | Wisconsin | |
| Colorado College | Minnesota | 58.256 | 162.307 | Minnesota | 74% | Colorado College | |
| Wisconsin | Colorado College | 78.0123 | 58.256 | Wisconsin | 57% | Wisconsin | Yes |
| North Dakota | Colorado College | 100 | 58.256 | North Dakota | 63% | Colorado College | |
| Minnesota State | Wisconsin | 106.123 | 78.0123 | Minnesota State | 58% | Wisconsin |
KRACH predicted 7 of 18 games correctly. Given the small sample, I’m happy to call that a coin flip. Somewhat amusingly, it missed on the four largest.
Conclusions
The forecasts seem grounded in reality and really do seem to provide pretty useful information.
Despite my note above about KRACH and tournaments, I’ll still probably post my annual “KRACH predicts the NCAAs”, and that will be it for this season. I’ve got lots of great ideas for next year, so hopefully I’ll find the time to get some of them implemented, and I’ll see you then!