Andrew Thomas and Sam Ventura introduced War-on-Ice.com, the best and most comprehensive advanced hockey stats site, on September 5, 2014, and it rose quickly to prominence within the analytics community. Over the weekend, I spoke to Thomas on the phone about how the site was developed, how it works, and what’s in its future.

Extraskater.com, the previous standard bearer in the advanced stats world, disappeared on August 19, 2014 when its founder, Darryl Metcalf was hired by the Toronto Maple Leafs. Two weeks prior, Tyler Dellow had been hired by the Edmonton Oilers and his site, mc79hockey.com, disappeared as well. However, the nugget of an idea that would grow into war-on-ice was actually born before the so-called Summer of Analytics.

Thomas and Ventura had had little contact with the hockey blogosphere. They had tried unsuccessfully to produce a metric for players that accounted for the effects of playing with good or bad teammates. At a seminar on sports research in April 2014, Thomas got a good reaction out of what would become Hextally diagrams. But despite compiling what he thought was good, interesting research, it was gaining little traction.

When Extraskater went down, however, Thomas and Ventura saw their opening. “We’re doing this, right?” With that, they set out to build a new site.

“I’m an academic first and foremost,” Thomas said. “So we wanted to make sure others could develop what we built. We wanted to make the data shareable.”

He and Ventura also designed their site so that if either (or both) of them were hired by an NHL team, the site and data wouldn’t disappear.

“We asked the question right off the bat: how to structure it so that everything would stay live. If one of us happened to go away, someone else could hop up and join in. And everything is pretty portable.”

It was a short turnaround between the decision to build the site and its debut on September 5, 2014. Thomas had the technology readily available to “scrape” the NHL’s play-by-play data and turn it into readable, usable files for their servers. He also had screenshots from Extraskater before it disappeared to ensure he could account for any features users might want.

Although Thomas believes the NHL provides mostly accurate data (not to mention, it’s the only publicly available source of necessary data), he acknowledges its drawbacks.

Though it’s uncertain exactly how it’s done, it would seem difficult for one human being sitting three stories above the ice to see a shot attempt and instantly, precisely chart it on his iPad, while also trying to determine whether it was a wrist-shot or a snap-shot, while also noting the exact time it took place.

“They’ve never invited me to observe the tracking process,” Thomas joked. “But the Florida Panthers said the room is locked for the integrity of the [off-ice officials who do it].”

(Thomas mentioned that shot location bias is particularly strong at Madison Square Garden in New York, where shots are charted much closer to the net than they really are, and at Amalie Arena in Tampa, where they’re charted much farther from the net. So good luck trying to figure out who’s shooting from where in this year’s Eastern Conference Finals).

“However,” he reiterrated, “if you’re willing to accept a small margin of error, it’s just fine.”

Acknowledging that margin for error led to a key innovation in evaluating goaltenders. Whereas this 16-zone diagram:

…had been the shot location standard, fewer zones would mean minimizing the margin for error. Hence, high-danger, medium-danger, and low-danger zones, and the ability for fans to see how goalies’ save percentages on shots from each of those areas.

Thomas said he, Ventura, and the War-on-Ice team calculate their site’s definitions (such as their scoring chance metric) meticulously to remove bias, but that they still have to trust the data the NHL provides.

For the first few months of his site’s existence, Thomas assumed that since he didn’t charge anyone to use his site, the NHL wouldn’t come after him for using their data. But he was still uneasy about the prospect, until NHL COO John Collins said on the record the league had no problem with secondary sites using their data, and had no intention of coming after them.

Thomas is confident War-on-Ice’s dataset is as comprehensive as there is.

“There are definitely things we don’t have, things we try to fix. But we’ve got the most comprehensive and de-biased dataset in the public sphere,” he said.

As for War-on-Ice’s competition, such as hockeystats.ca, stats.hockeyanalysis.com, and naturalstattrick.com, Thomas believes his site is out-performing the others since Extraskater has been dead.

“In terms of total traffic, we’re doing quite well,” he said.

But when I asked how he assesses whether War-on-Ice is succeeding, he offered a different definition.

“To me, success has nothing to do with visits. If people not only respect us but want more, that’s the key. It’s like if I cook dinner and people ask for seconds, I know it was good. If people we know and trust are using us, we’re successful.”

Thomas added, “It’s a weird position to be in. We’re competitors to produce the best product, but we’re not combatants. We all gain from each other’s success. We network and communicate and become better known.”

As much information as there is on War-on-Ice, there’s more Thomas, Ventura, and their team could include. They have the capability, Thomas said, to break data into the most granular levels.

“We could add columns to the site with splits for how a certain player performs on third-period power plays on the road,” he said. “And anyone who’s asked for that data has gotten it. But eventually we have to decide what’s enough. Every time you add a split, you’re increasing the work the site has to do, and microsplits are misleading anyway.”

However, War-on-Ice has consistently incorporated new features since its debut, and Thomas, Ventura, and their team continue pushing the envelope. They’ve arranged to host Stephen Burtch’s DeltaCorsi metric and Emmanuel Perry’s Bombay Scores, and with the invaluable assistance of Alexandra Mandrycky are compiling a contract and cap hit database. Ryan Stimson’s Passing Project is soon to come.

But it’s the “single number” metric, capable of firmly evaluating every player’s direct contribution to goals and wins (a la baseball’s Wins Above Replacement stat) that gave the site its name and represents the greatest challenge for not only the War-on-Ice team, but for the hockey analytics community in general.

Thomas, though, said he’s “not married to that idea” anymore, and that a “single number would be most useful to compare what aspects of the game are most important.”

Regardless of any metric or advancement to come, War-on-Ice has established itself as the go-to advanced stats site and has surpassed Extraskater’s capabilities. It’s done so through Thomas’ and Ventura’s hard work, but also thanks to their cooperative, generous approach to the site.

“People we were talking to wanted certain features, so those were our first steps,” Thomas said. “Most times people make requests and it’s simple and takes 10 minutes. The biggest request has been for WOWYs (With Or Without You metrics) but I’m not as big a fan of them because of their statistical properties.”

Thomas said the site has glitches occasionally, including when the playoffs started, because the NHL’s play-by-play data was coded differently. “But no one has ever said that our site sucks, thankfully,” he added.

The site is a labor of love, ultimately. Of course, Thomas and Ventura have become better known through their efforts, which helps in their academic pursuits. But Thomas also acknowledged War-on-Ice can carve out a legacy in the history of the analytics movement.

“I’d like to think we play the role Extraskater did and that Behind the Net did before that,” Thomas said. I’d like to think we help people do their own research. I wouldn’t mind being the last of our generation, before the NHL comes up with a great product themselves that blows everyone out of the water.”

That day might be on the horizon, with the NHL and SAP having partnered to provide advanced stats on the NHL’s website. But it’s not here yet.

Does anyone even use the http://t.co/kIQ9F3necC advanced stats section? — Draglikepull (@draglikepull) May 21, 2015

My mentions right now are just "No" over and over — Draglikepull (@draglikepull) May 21, 2015

And until that day arrives, we’re in good hands with War-on-Ice.