The U SPORTS Top 10 rankings are undergoing significant change this season.
All team sports, including football, men’s hockey, women’s rugby, and men’s and women’s soccer had previously determined the weekly Top 10 based on votes from coaches or media panels. Now, those rankings will be fully based on a statistical approach in the case of soccer and rugby, and determined by combining that approach with an equally-weighted media panel in football and in men’s hockey.
Ken Saint-Eloy, U SPORTS Senior Manager of Marketing and Communications, said this change was driven by the will of the organization to implement a more visible, more consistent and ultimately a more credible Top 10 for all sports across the board.
Given the situation and in the interest of making things more visible and structured following the rebranding of the organization, we mapped out the rules and process for every single sport and looked at options of objective, stats-based solutions for these Top 10 rankings.
“There wasn’t a clear and objective answer when we were challenged by our audience about why a team moved up or down, or didn’t make the Top 10 after a series of wins. We were not able to provide simple answers on who were the voters, and/or what criteria they were voting on,” Saint-Eloy said. “Even in the same sport, for example, women’s soccer was not following the same process as men’s soccer. Given the situation and in the interest of making things more visible and structured following the rebranding of the organization, we mapped out the rules and process for every single sport and looked at options of objective, stats-based solutions for these Top 10 rankings. Because you want your Top 10 to create conversations about the teams, the games played the past weekend or the coming matchups, not about how consistent or reliable the Top 10s are, especially if they play a role in the seeding of the national championships.”
The statistical approach U SPORTS will be using is based on the Elo Rating System, originally created as a chess rating system by Arpad Elo (and first adopted by the United States Chess Federation back in 1960), but since modified for many sports. Particular modifications of that system for the different U SPORTS rankings here have been made by Mario Kovacevic - a former player for the University of Toronto Varsity Blues men's soccer team and assistant coach with the York Lions - who’s now producing and running various rankings systems for U SPORTS and other sports organizations through his RankR company. Each system is modified and tailored for the specific sport in question, and it brings in historical data from past seasons to help compare teams that don’t directly play each other, such as those in different conferences. Kovacevic said he first started looking at this idea around the 2011-12 season while he was still a student-athlete.
I just started playing around with different rankings systems. From there, I put together a really simple rating system, and I sent it to some of the coaches and they were really impressed.
“I was playing soccer for U of T and also doing a master’s of engineering, and between being a grad student and playing for a U SPORTS soccer team, I was interested in the rankings,” Kovacevic said. “It was a lot to do with votes and you couldn’t really drill down to the bottom and figure out ‘Okay, what does my team have to do to get better than the team ahead of us?’ And furthermore, if you were outside of that Top 10, you really didn’t have an idea if your team was 40th in the country or 11th in the country. So between those two main things, I just started playing around with different rankings systems. From there, I put together a really simple rating system, and I sent it to some of the coaches and they were really impressed.”
The system then sat for a few years, but with U SPORTS looking to go to a more statistically-based rankings system, it came up again. Kovacevic said there had been some frustrations in men’s soccer over the voting system, both based on the results it produced and the amount of time it took to vote and tabulate votes from coaches. He said Guelph Gryphons coach Keith Mason – the U SPORTS men’s soccer Top 10 representative – and York Lions coach Carmine Isacco played key roles in the process, which eventually led to it being more widely adopted.
“Coach Keith and Coach Carm remembered that I had this thing lying around, and Coach Carm brought forward this idea to the men’s soccer committee and they really liked it.”
But getting from there to the adoption of this across many sports was a long and involved process. Kovacevic said that led to him working with different sports’ Top 10 reps and committees, refining the formula for each sport, and showing them how this would look on a week by week basis.
“Over the last year or so I showed the proof of concept, what would have happened last season if this ranking system was applied week by week.”
I think it is for sure a step in the right direction, especially when we have so little inter-conference play in some of our sports, so it’s really hard to make those kind of direct comparisons. Any objective data that we can use I think just makes the rankings that we’re putting out that much more legitimate, because they’re as objective as they possibly can be.
And there are many who think this will be a positive change. Ben Matchett, the University of Calgary’s assistant athletic director for operations and communications, said this is a logical step for U SPORTS.
“These things evolve, and we’ve even seen it with the NCAA moving on from the RPI for the men’s basketball tournament, creating their own metrics,” Matchett said. “The analysis tools that are available now are significantly better and probably in a lot of ways more objective than some of the previous ways we’ve been doing things.”
Matchett said this should particularly help with cross-conference comparisons.
“I think it is for sure a step in the right direction, especially when we have so little inter-conference play in some of our sports, so it’s really hard to make those kind of direct comparisons. Any objective data that we can use I think just makes the rankings that we’re putting out that much more legitimate, because they’re as objective as they possibly can be.”
Matchett said these changes will help boost the credibility of the Top 10, and help schools use it to start conversation about the rankings.
“The point of the Top 10s for most of our sports is to generate conversation and generate some publicity,” he said. “Obviously we want it to be as objective and straightforward as possible, and anything we can do to further that really helps the cause - as a sports information director, as a school, to be able to say ‘Here’s legitimately where this ranking system places us, and it’s not based on the whim of a coach, for us in Atlantic Canada for example, who saw us play one game three years ago.’”
He said rankings play an important role in drawing attention to different teams.
I think it’s positive because the intention is to make the process better and more realistic regarding the true strengths of everything we have in the country.
Isabelle Leclaire - head coach of the Montreal Carabins’ women’s hockey team
“Rankings can always be turned into a story for us. Maybe we have a team that’s rebuilding that cracks the top 10 for the first time in 10 years or whatever. Or we have a team that reaches the highest point they’ve been in X number of years, or that is No.1 for the 15th time in the last 16 years. Every time that something like that happens, there’s going to be an opportunity for a story…And the opportunity to say that you are the No.3 team in the country playing the No. 7 team in the country is pretty significant in terms of generating interest, which ultimately is what we’re all here trying to do.”
Women’s hockey is one of the sports that isn’t fully adopting the statistical system yet, and that’s because its Top 10 rankings are used for the purposes of national championship seeding. That means any change to that procedure can’t be made without altering the official playing regulations, which require consultation with the sport’s technical subcommittee. But these rankings will still be calculated for women’s hockey too and sent to the voters each week, giving them some statistical information they can consult if they wish. And the plan for women’s hockey, along with cross country, men’s and women’s basketball and women's volleyball is to eventually move towards this system, thereby standardizing all U SPORTS offerings, with swimming, track and field, wrestling and men's volleyball having been calculated statistically in the past.
Isabelle Leclaire, the head coach of the Montreal Carabins’ women’s hockey team, said she thinks the new system will work well for her sport.
“I think it’s positive because the intention is to make the process better and more realistic regarding the true strengths of everything we have in the country,” said Leclaire, also the U SPORTS Top 10 rep. “I think that’s a positive position for us in women’s hockey.”
Leclaire said a huge issue with the current system is the difficulty of evaluating out-of-conference teams.
“The challenge for us is we don’t play a lot of games inter-conference, and we don’t see enough games, and nobody has the information of every team, or can see it and have a good evaluation of who the strongest team in every division is. For us to have that objective information, that will be a great help.”
Leclaire said it will also be useful to have a ranking of teams beyond the Top 10.
“For us, the change that I think will be appreciated is instead of just having the information on the top 10 teams, the teams that are just outside of the Top 10, not knowing why they’re not there or where they stand in the country, now we’ll have the information on every team in the country and where they figure.”
Leclaire added that this stats-based approach will eventually prove valuable across U SPORTS
“Every sport has a different reality, but overall, I think this system is something that can be useful for every sport.”
Andrew has been covering university sports in Canada since 2005 at outlets such as The Queen's Journal, The CIS Blog, and Yahoo Canada, where he also served as the editor of the Canadian football blog 55-Yard Line. He has a Bachelor of Arts (Honours) degree from Queen's University with a major in history, and currently works as a staff writer and editor for Awful Announcing and The Comeback.
Follow Andrew on Twitter @AndrewBucholtz