The Top Escape Rooms Project consists of two phases: a nomination phase and a ranking phase.

Phase 1: Nominations

In the nomination phase, pre-approved nominators (consisting of enthusiasts that have played 200 or more escape rooms) are asked to privately nominate up to 20 rooms that they feel should be considered in the conversation for best in the world. Any room with two or more independent nominations is deemed to have sufficient support to move on to Phase 2, with the idea that for a room to be worthy of the distinction of being one of the best in the world, then it would surely be nominated by at least two people.

Owners, designers, consultants, investors, employees, and business affiliates with significant ongoing financial relationships are not allowed to nominate any of the corresponding rooms for which they may have a conflict of interest.

Additionally, two new rules were added in 2019 for a room to be eligible for nomination. First, the room must have been open for at least 30 days between November 1, 2018 and October 31, 2019. Second, the room must be playable for a team consisting entirely of English-speaking players. Aside from these two rules, nominators were free to include any room they had played, using any criteria they felt was relevant to a room being considered one of the best in the world, including the very definition of what counts as an escape room.

In 2019, a total of 105 enthusiasts contributed to Phase 1, resulting in 507 rooms nominated and 229 rooms with multiple nominations that moved on to Phase 2. Companies that had at least two rooms nominated and at least one room move on to Phase 2 were considered for the company awards, and 79 companies met this criteria. For the complete list of rooms that were nominated in Phase 1, see the Phase 1 Room Results.

Phase 2: Rankings

In the ranking phase, pre-approved voters (consisting of enthusiasts that have played 50 or more escape rooms) are asked to provide a complete, ordered stack rank of the rooms that survived Phase 1. They are only allowed to rank rooms that they have played and for which they do not have a potential conflict of interest, as defined in the previous section.

Stack ranks are used for this process rather than numerical ratings because it forces participants to think hard about every entry in their stack, and this approach does not suffer from different people having different calibrations for numerical ratings. Stack ranks also have the nice property that you can always add new rooms at the very top as rooms worldwide get better and better without having to recalibrate your numbering system or add "6-star" ratings.

In 2019, a total of 370 enthusiasts providing rankings in Phase 2, including all of those who contributed to Phase 1, and the output of these individual stack ranks provided the input to the final global rankings. The algorithm for generating a global ranking from a set of partial rankings is subject of substantial academic research and there is no universally agreed-upon method for generating such a result. TERPECA uses a fairly well-established method that was inspired by an academic paper about ranking college football teams that is itself based on the Perron-Frobenius theorem. In a nutshell, each stack rank is used to create all possible pairwise comparisons between rooms in that stack rank, and then those are used to create a matrix of scores between rooms using all the direct comparisons from any of the participants. To make the analogy to the college football analysis, we consider the score of a match between two rooms to be a combination of any direct comparisons that can be made between two rooms and (new for 2019) any transitive, secondary comparisons that can be made between a pair of rooms. A small correction is added using the Wilson score binomial confidence interval to ensure that more confidence is given when more comparisons between the same two rooms are available. Finally, the eigenvector corresponding to the greatest eigenvalue is calculated, and the entries in this eigenvector is used as the score for each room.

The results given by this analysis were studied in detail to verify that the final rankings stood up to careful scrutiny when comparing two rooms to each other based on all the stack ranks that included both, but this data is excluded from the final report because it can be used in some cases to infer individual preferences between rooms. Suffice it to say that we are quite confident that the outcome here faithfully represents the data provided by those participating in the project.

In 2018, companies were ranked using the same process as rooms, however, participants expressed difficulty in ranking companies, and the results indicated that company results were highly enough correlated with room results that we could generate the company rankings in 2019 from the room results instead. So, company results for 2019 are now generated based on the findings from the 2018 results applied to the 2019 data, and roughly works based on the observation that two or more games in a higher ranking tier is weighed more heavily than more games in a lower tier.

Ultimately this analysis provided an aggregated stack rank of all the rooms and all the companies in Phase 2, but we have chosen only to recognize a subset of these rooms and companies as official award winners. The primary reason for this is that two nominations to reach Phase 2 was a pretty low bar designed to make sure that the very best possible rooms and companies would make the list if any two participants had played them, but it did not guarantee that those that did get nominated actually were significantly better than, say, those that got only one nomination in Phase 1. Regardless, we still present complete final rankings of rooms and companies from the Phase 2 analysis, which you can find in the Phase 2 Room Results and Phase 2 Company Results.

General Notes and Caveats

It should be noted that while there are nominators and voters from many countries, and in particular many more than in 2018, there are still some regions that have had far more players than others. As a result, rooms from some of the less well-played countries (such as Australia, Israel, and Switzerland) are limited as to how high on the list they can reach, given that there is the limit to the extrapolation we can do when there is not a lot of comparison data about a given room. We hope and expect that as rooms from new countries show up on the list, more enthusiasts will play them, and in future years, those regions can find their "proper" place in the rankings.

Due to the new rule that required rooms to be playable in English, there were a handful of rooms that appeared on the list in 2018 (including some winners!) that are not included in 2019 only due to the new rule. There are also of course many world-class rooms that just aren't available in English. We plan to review the impact of this decision this year and have not decided yet whether it will be in place for 2020. That said, if the rule is kept for 2020, we plan to require that companies that do support games to be played in English to clearly state this on the website, to ensure that our decisions about which games should be included is more transparent and verifiable. We believe that if a room is to be considered for an international award, it should at the very least clearly indicate which languages it supports or doesn't support on the website.

Finally, the methodology chosen does have a bias that the more rooms someone has played, the bigger an impact they'll have in the rankings, as they'll be able to provide significantly more comparisons between rooms. We accept this as a feature, not a bug, as we believe that it is certainly a defensible position that those with more experience be given greater weight in this regard.