Hours before the Australian Open started in Melbourne on Monday, BuzzFeed News and the BBC published results from a joint investigation showing that tennis authorities hadn’t punished male pros repeatedly flagged for suspicions that they were fixing matches — deliberately losing, or arranging for their opponent to lose, to maximize their or others’ betting profits. Tennis authorities quickly gathered in Melbourne for a news conference responding to the charges, saying they had “thoroughly investigated” any evidence brought to them.

The process by which tennis investigates alleged match-fixing is so secretive that it’s impossible to judge the accuracy of authorities’ response. But the BuzzFeed-BBC report, and its aftermath, does provide a case study of how difficult it is to evaluate what could look like suspicious betting activity. It’s possible to use data analysis, as BuzzFeed did, to raise questions about certain matches and players; it’s much harder, and may be impossible, to use that data to accuse specific players of throwing matches without the additional investigative powers tennis authorities wield — and according to the BuzzFeed-BBC report, often aren’t using.

As part of the investigation, John Templon, an investigative data reporter for BuzzFeed News, spent more than a year analyzing 26,000 professional men’s matches and found 15 players who lost matches with unusual betting patterns “startlingly often.” (Match-fixing is also believed to occur in professional women’s tennis, but the BuzzFeed-BBC investigation focused only on men’s tennis, so we are in this article, too.) BuzzFeed and BBC didn’t name these players, citing a lack of evidence of wrongdoing and possible alternative explanations for underperformance, including injury. But BuzzFeed did release an anonymized version of the data it used on GitHub, including a file containing betting odds and the year for 129,271 matches.

Quickly, people wrote on Twitter and on GitHub that the data could be de-anonymized, thereby identifying the 15 players Templon mentioned. Ian Dorward, a London-based tennis bettor who used to set and adjust tennis betting lines for a bookmaker, emailed me the list of what he believed to be the 15 names. After Chris Bol, a data analyst based in Utrecht, the Netherlands, published the same names, Dorward went public with his findings, which criticized BuzzFeed for making the data relatively easy to crack.

As much as I appreciate Buzzfeed releasing their data, they must have known that it would be possible to identify the players from it… — Ian DW (@sportdw) January 20, 2016

It took me 6 hours to identify the 15 names on their list, which begs the question of why exactly they anonymised it, but still released it — Ian DW (@sportdw) January 20, 2016

BuzzFeed hasn’t confirmed the list of names — we’ll call them the BuzzFeed 15 — though the methods Bol and Dorward used appear straightforward and arrive at the same names. Asked for comment, BuzzFeed investigations and projects editor Mark Schoofs sent a statement by email. “The betting data we used in our analysis is publicly available — that’s how we got it,” Schoofs said. “In our journalism, we try to show as much of our work as possible, which is why we made the algorithm public.”

Dorward looked more closely at eight matches that BuzzFeed’s analysis flagged and concluded that for each one, there was “no evidence of anything suspicious.”

“It’s very, very dangerous to make blasé assumptions about a match being dubious because of prematch movements,” Dan Weston, a tennis analyst and trader who writes for the website of the sports book Pinnacle, said in a telephone interview. (Using only data on betting and results to demonstrate fixing has proven problematic in other sports.)

“By itself, the analysis of betting data does not prove match-fixing,” Schoofs said in his statement. “That’s why we did not name the players and are declining to comment, and also why our investigation went much wider than the algorithm and was based on a cache of leaked documents, interviews across three continents, and much more.”

So how could a player lose matches with big odds movements “startlingly often” without fixing matches?

Well, lots of ways:

A player could tank a match — deliberately lose it — without fixing. Sometimes players stand to make more money by losing early in one tournament so they can get to another. Other times, players might collect a bonus or appearance fee for showing up to a tournament they’d rather not play and then lose early so they can rest and focus on a bigger upcoming tournament.

No. 1 Novak Djokovic was accused of doing just this — showing up to collect a bonus but losing deliberately at a tournament in Paris — in 2007 by the media and tennis-forum posters at the time, and by an Italian newspaper this week. At the time, Djokovic said he wasn’t well. Djokovic said Wednesday that he didn’t throw the match: “It’s not supported by any kind of proof, any evidence, any facts. … It’s not true.”

Deliberately losing a match is punishable by a fine, but is a much less serious offense than fixing a match for gambling purposes.

Bettors could have inside information on a match outcome without the player’s involvement. For instance, if the player isn’t at full strength, his coach, trainer, spouse, family members or friends might know it before betting markets do and use that knowledge or pass it on to other bettors. It might not even involve an insider at all. In the early rounds at small tournaments, a fan who happens to overhear a conversation or witness an injury at practice could trade on that knowledge before anyone else in the betting markets knows.

Betting markets could simply get the opening odds wrong. BuzzFeed’s analysis identified matches for which at least one of seven major bookmakers’ odds moved by so much from when the market opened to when it closed — generally, the day or so between when a matchup is set and when the match starts — that one player’s implied chance of winning decreased by more than 10 percentage points. That typically happens when many bets are placed against the player, suggesting the initial odds were too bullish on his chances. Bookies then adjust the line to increase and balance betting volume and to reduce their exposure.

Heavy betting against the player could mean some bettors know he’s going to lose. But it could also mean that many bettors spot favorable odds for reasons that the bookmaker isn’t taking into account. Often bookmakers use an algorithm to set initial odds. Depending on how sophisticated it is, that algorithm could fail to take into account injuries, or a bad matchup, or lack of play on the court surface. The more obscure the athletes involved, the more likely sports books are to whiff with their opening bid. (For reference, professional bettors in the U.S. say they focus their energies on a single, relatively unnoticed part of the sports landscape — say, backwater college basketball conferences or Major League Soccer — and use their expertise on this little swath of the sports cosmos to beat the relatively uninformed book. These bettors will often also make arrangements to trade their picks for another bettor’s picks in a different, equally obscure sport, which is how syndicates are formed and lines are moved.) Many of the flagged matches involve little-known players in third-tier tournaments, making the lines vulnerable to a well-informed bettor.

BuzzFeed’s analysis included only the 39 players who lost 11 or more matches in which the odds moved heavily against them, and the 15 players it flagged were ones who lost far more of those matches than would be expected. BuzzFeed also corrected for what’s known as the multiple-testing problem, which can produce spurious results that look statistically significant, by using a Bonferroni correction — and it still found four players with significant results. So that should help mitigate concerns about any one match being a false positive. But some players are particularly tough for bookmakers to handicap, whether because they’re coming off an injury, or because they don’t play that often, or because they’re ranked higher than their true talent after a run of good luck that bettors, but not the bookmakers’ algorithms, account for. These kinds of reasons could help explain the presence of several of the players Dorward identified as being on BuzzFeed’s list. They’re also why alternative sourcing is so crucial; BuzzFeed provides supplementary evidence where it can, but as we’ll cover below, seemingly straightforward things like video of the matches in question can be hard to come by.

The details of how BuzzFeed chose to do its analysis could affect which players are flagged as losing suspicious matches suspiciously often. BuzzFeed’s analysis is impressive in many ways. It’s vetted by two professors of statistics, covers 26,000 matches, excludes books with opening odds that are major outliers, accounts for multiple testing and chooses the same bookmakers that Dorward says he would have used. But any analysis involves making choices, and the more robust findings are ones that hold even when different reasonable choices are made.

To check that, we enlisted the help of Jeff Sackmann, a tennis data analyst who wrote his own code, at our request, to collect and analyze tennis betting data. He checked more than twice as many matches — nearly 63,000 — from late 2008 through the start of this year. These included matches from the ATP World Tour and Grand Slam tournaments, which are included in BuzzFeed’s analysis, but also from Challengers, the sport’s minor league, where prize money and public attention are lower and the risk of match-fixing is believed to be higher.

Following BuzzFeed’s methodology, Sackmann found similar results for his expanded data set, including the same four players topping the BuzzFeed 15 list by losing the most matches relative to expectations. However, he also found that some players excluded from the analysis because they had too few flagged losses otherwise would have appeared because they lost every match with big odds movements.

Sackmann also found that the results had less statistical significance — just one player, not four, lost a significantly larger number of matches than expected, after applying the Bonferroni correction. That’s in large part because Sackmann made one different choice: He used the median of all bookmakers’ opening odds for the true probability of a player winning the match, as opposed to the probability suggested by the opening odds from the bookmaker that had the biggest odds movement. That bookmaker usually was more bullish than its competitors about the player’s chances, so using its odds makes the player’s loss seem more surprising than it really was to the market as a whole. Also, Sackmann tested all players with at least 10 matches in which the odds moved heavily against them — not just players with 11 or more losses in matches like that.

He also checked how the analysis would differ with a different set of bookmakers. When Sackmann used the same methodology that reproduced BuzzFeed’s list above, but with this set of bookmakers, he got very different results. Most of the names he identified as losing these matches surprisingly often were not the same as the ones he identified using BuzzFeed’s list of bookmakers.

In its article, BuzzFeed writes that at least six of the 15 players it identified “have been flagged to tennis authorities by outside sources.” But the overlap could just mean that BuzzFeed and the outside sources were studying similar data with similar methods. Many of these outside sources named by BuzzFeed were using betting data as their basis for suspecting players of fixing; some, in fact, were part of the betting industry — a firm, a watchdog, a sports security association that collects alerts of suspicious betting from bookmakers. And some of the decisions BuzzFeed made in its analysis — such as where to set the cutoff in odds movement for a match to be worthy of more investigation — were based on suggestions from sports-betting investigators.

None of this means that the BuzzFeed 15 haven’t fixed matches — just that, as BuzzFeed and the BBC themselves have made abundantly clear, the data analysis by itself isn’t conclusive.

“It’s incredibly difficult to actually prove fixing,” Dorward said in a telephone interview.

So what would be more conclusive?

Other betting data. Tennis betting experts say the market has moved toward so-called in-play betting — bets placed during a match, as odds shift in response to what’s happening on the court. So, for instance, when a player wins a set, or a game, or even just an important point, bookmakers or betting exchanges quickly change the odds to reflect the increase in his probability of winning the match. That creates opportunity for bettors who know the fix is on to bet against the player who is ahead with even more favorable odds than the prematch line. BuzzFeed published a document from a 2008 investigation into match-fixing that identified several matches with that kind of suspicious betting pattern — including the sport’s most well-known example of suspected match-fixing and two other matches whose participants can be identified from the scores and opponent listed. None of the players involved are among the BuzzFeed 15. In-play betting data is available for purchase from some past matches, though it is difficult to use because it is not coded with information on the score at the time of bets. We also don’t have data on betting volumes and on maximum bets, which would show whether large amounts of money were at stake in flagged matches.

Video evidence. Former player Daniel Koellerer — who was banned for life from pro tennis for fixing but denies the accusations — told the BBC that it would be easy for a pro to go unnoticed while fixing matches. But not every fixer covers his tracks well. Being able to review video of suspected matches would at least let authorities (or casual but interested onlookers) scrutinize a player’s effort throughout a match. However, not all matches are televised, and video is hard to get after the fact even for those that are. Tennis authorities ask YouTube to pull unauthorized matches and make video of archived matches available on the subscription site TennisTV for just seven days. Some older matches are available through the ATP Media Digital Archive, but this includes just one of the matches flagged for any of the BuzzFeed 15.

Other corroborating evidence. This could include texts between or about players, bank records and other information.

BuzzFeed and other journalists don’t have ready access to this kind of data. But tennis authorities do. The Tennis Integrity Unit — backed by the men’s and women’s pro tours, the four Grand Slams and the International Tennis Federation — can compel players to turn over phone and bank records, and it has access to detailed betting data. “Co-operative agreements with the betting industry, regulators and other parties (including ESSA, Betfair, UK Gambling Commission) can provide immediate real-time access to gambling market intelligence,” TIU spokesman Mark Harrison said in an email.

Is the TIU using all of this information, along with tips about players suspected of fixing, and pursuing it as far as it can? The TIU says yes. It also maintains extreme secrecy around its operations, going so far as to not reveal details of its inquiries even in the rare cases when it announces a punishment. “TIU estimates that most, if not all, of the 18 successful corruption charges laid since 2010 would not have been achieved without the ability to work in confidence,” Harrison said.

Maybe the TIU really has done all it can to root out corruption, chasing every player whose name comes across its desk. Maybe some turned out to be red herrings, like some of the BuzzFeed 15 might turn out to be. Maybe others really are fixing — giving in to the temptation to earn far more than they can by playing to win — but have gotten wise to tennis’s investigative approach and avoid using their phones or bank accounts. Or maybe fixing is very rare, and suspicious betting usually has innocent explanations.

However, experience from other sports tells us there is also good reason to suspect that when sports regulate themselves, oversight can be lax. That’s really what’s at the core of BuzzFeed and the BBC’s reporting, more than the data analysis: a group of six former tennis insiders on one side saying tennis authorities haven’t followed up on what the former insiders think is compelling evidence of match-fixing, and on the other side those same authorities saying they have followed up, but confidentiality rules bar them from saying much more.

Andrew Flowers contributed analysis to this article.