After some oblique references via tweet , Microsoft has directly responded to our request for comment on our recent Xbox data analysis project . That response has led us to issue the following corrections and clarifications to our piece.

Microsoft has given us reason to believe the usage data provided by Xbox API consisted of incomplete estimates of total Xbox Live usage, and does not reflect a complete account of recent usage sessions by the sampled Gamertags. While the data provided seemed reliable in our spot tests, Microsoft tells us the API was "intended to display to each Xbox gamer an approximation of the time spent in a game so that they have the option to compare it with other gamers on the service."

That fault in the underlying data has led us to vastly underestimate total usage times for the apps and games in our usage sample. Graphs and charts that refer to average minutes played or percentage of users who played a game during that 4.5 month period seem to be an order of magnitude lower than the actual per-app usage rates and times. The "My Games and Apps" section of the Xbox One was used by 71 percent of Xbox One players during our sample, according to Microsoft, not the approximately 6.3 percent shown by our data.

"We specifically know, based on our complete view of Xbox Live usage data, players are highly engaged with backwards compatible game titles," Microsoft said by way of additional example. "It’s why we continue to support this well-loved feature and the games that use it."

It's still unclear to us whether or how the "approximations of usage" provided by the API impact the relative usage rankings in our sample. If all apps and games were undercounted by the API's "approximations" at the same rates (or even roughly the same rates), then these relative rankings would still be largely valid over a random sample. That would apply to figures and charts describing various apps or categories (such as Netflix or backward compatibility) as percentages of total sampled playtime.

(Multiplying out our raw reported backward compatibility usage by an "underreporting" factor of about 11—to match up with the relative gap shown in the "My Games and Apps" example above—would also bring Microsoft's reported numbers and our sampled numbers on that score much more in line...)

Microsoft's response to us indicates the API usage data allowed users to "compare [their usage] with other gamers on the service," suggesting to us that this kind of relative comparison from sampling may still be reliable and worthwhile. But Microsoft has not provided us with sufficient information to answer this question of relative comparison reliability to our satisfaction, simply saying it considers the current analysis "grossly inaccurate and misleading due to an incomplete set of data and drawing conclusions about actual usage from data that approximates usage."

Microsoft also cited "the sample size" and "the data of many users who opt out of sharing this type of data with other users" in questioning the reliability of our usage data. While we addressed these potential skews in our original piece, we don't believe they are sufficient to explain the size of the usage data errors described above, absent other underlying problems with the API usage data.

What we're not correcting

Microsoft has given us no significant reason to doubt the basic reliability of the separate "ownership" sample in our report, which measures what games show up on the public Achievement lists on Xbox.com (even for games where the player has zero achievements). However, the company points out that "the data source used in that section returns data "about users’ profiles that were active when a game was played; it doesn’t provide any information about physical or digital ownership of the game."

We also warned about this in our original piece, noting that "owners" was being used as a shorthand for all sampled users who had the game show up on their Xbox Live account. We also noted in our Limitations section that "ownership" data couldn't account for the circulation of used discs or single copies of a game accessed by multiple Gamertags, and therefore couldn't be used to extrapolate reliable sales data. That said, going forward we will refer to the data generated by this source as a "player" report, rather than an "ownership" report.

Microsoft has been very gracious in working with Ars to try to correct and clarify sources of error in our data. We've tried to be as upfront as possible about our data collection and interpretation methods, and we regret any misperceptions that were spread from errors in our underlying data or our analysis.

"We acknowledge the work it took to put your story together," a Microsoft spokesperson told Ars. "It’s quite a task to collate the stuff you can gather from the outside without having direct access to proprietary tools and data, and create a 10,000-plus word story from it. We salute that effort... We appreciate the work and effort by Ars Technica to share more information about the Xbox community and we are continually looking for ways to do so that also protect the interests of gamers and our partners."