Dubas explains value of hockey analytics at Sloan

BOSTON -- Kyle Dubas was putting the finishing touches on his list of prospects for the 2015 NHL Draft when he noticed a troubling trend.

The ones who played in the IIHF World Junior Championship were, relatively speaking, ranked higher than similar prospects who did not. Dubas found this to be a form of bias, something he was desperately trying to weed out of his rankings.

"Does playing in the world juniors at 17 make you a better prospect than someone that doesn't?" Dubas said during his presentation of "How Analytics has Limited the Impact of Cognitive Bias on Personnel Decisions" at the 2015 MIT Sloan Sports Analytics Conference. "I don't think so. I think we know that from years of going through and measuring at different points."

Dubas, whose hire as an assistant general manager by the Toronto Maple Leafs was the highlight of the NHL's "Summer of Analytics" in 2014, spoke for more than 20 minutes Saturday about his experiences as general manager of the Sault Ste. Marie Greyhounds.

His ability to incorporate the use of analytics with the Soo made them one of the most successful teams in the Ontario Hockey League and helped convince the Maple Leafs to hire a 28-year-old with no professional playing experience to help overhaul the collective thought process for an organization once considered a leader in being anti-analytics.

The major points Dubas focused on in his presentation were a window into the future of hockey analytics. Collecting the data and knowing how to interpret it is one thing, but being able to communicate to others in your organization and have them buy in is quite another and extremely valuable. He also spoke on how different types of bias can negatively affect teams, and how the use of analytics can help prevent that.

Hockey fans (and team representatives, for that matter) have squabbled for years about the validity and use of certain advanced statistics, and whether there is a place for analytics in the sport. That larger battle is over, even if some choose to continue to engage in minor skirmishes.

One of the biggest themes of this two-day conference across several different sports is simple: OK, we have the data and we know it's good, but how do we get everyone else in our organization on board?

Dubas mentioned the transformation of his team during his second year with Sault Ste. Marie. The Greyhounds were controlling about 47 percent of the shot attempts in the first 30 games and improved to 57 percent for the rest of the season.

The difference? Dubas hired a new coach, Sheldon Keefe, who took over and was open to using the data Dubas' small team was able to deliver.

"Basically, there is a big difference between raw statistics and data gathering and using that data to incorporate into your process as a team," Dubas said.

Dubas also was part of the "Changing on the Fly: The State of Advanced Analytics in the NHL" panel, along with Edmonton Oilers analyst Tyler Dellow, former Oilers coach Dallas Eakins, former NHL executive Frank Provenzano of ESPN, and James Mirtle of The Globe and Mail.

Eakins, Dellow and Dubas spoke of their experiences trying to communicate better about the data that is available.

"Everybody in hockey is interested in more information, but not everyone speaks the same language," Dellow said, pointing out that he has been able to use video to bridge the gap in the Oilers organization.

The other focus of Dubas' presentation was filtering out bias and how thinking analytically can help. He listed five types of bias (conformation, recency, information, sample size, and simplicity) he has experienced during his time with the Soo and the Maple Leafs.

Beyond the draft rankings, Dubas brought up two critical times on the NHL calendar when bias becomes a big problem. The first is right now, the days leading up to the NHL Trade Deadline.

"We have the trade deadline coming up on Monday," Dubas said. "You're in a discussion with another team and they say, 'We want Player X from you,' and we say, 'OK, what are you going to give us?' And they say, 'We'll give you prospect A, B or C, and a fourth-round pick.'

"The initial reaction is to open the floodgates unleash our scouts and they rush to, pick an AHL city, Binghamton, they go to see the Binghamton Senators to watch prospects A, B and C, and they say, 'Prospect B was great, prospect C was terrible, and prospect A was just OK.'

"So immediately our discussion shifts to prospect B. We're eliminating hundreds of games that we've scouted of these players and many data points that we have and we're putting it on one game on Feb. 23, 2015 when we've watched this player for four or five years."

The other time bias can lead a team to make the wrong decision is during training camp. A large amount of players are together, and a small amount of time is used to try to decide which ones should be on the roster.

Dubas warned of letting any results during camp play too large a role in the decision.

"We have years of information and scouting reports on players," he said. "But if a player comes into camp and has a good training camp we put him on the team and take off a player who has proven for years to be better and put him in the minors, or you release him and someone claims him on waivers."

The theme of Dubas' presentation was the importance of being willing to learn. He incorporated analytics in his first year with the Soo, but they missed the playoffs. So he learned from the failure, and that helped him turn Sault Ste. Marie into the one of the top teams in the OHL.

It is a desire to learn more and test what has long been considered truth in hockey, even if it might not be, that spurs people who want to be part of the analytics movement in the sport.

"I don't understand why it has to be for and against [analytics]," Eakins said. "You should be for everything."

---