For a long time, the escape room ranking was calculated according to a certain formula (we will remember it here as well). But since the beginning of 2022, we decided to update it and make it more lively and dynamic, so that it would reflect the current state of affairs as much as possible, and the rating would be up-to-date. Let's take a look at how it worked, what problems arose and how we solved them.
Previously, our formula looked like this:
There are 4 degrees of trust, which mainly depend on the experience of the person - if he has passed a lot of escape rooms (and accordingly wrote on them objective meaningful reviews), then the level of trust in such a User will be higher. That is, the beginner “trust” will be 0, the one who has passed at least 5 escape rooms and can more or less compare - 1, and so on. Each LoT(level of trust) has its own specific weight in the final score: with trust 0, 1, 2 and 3 respectively, 10%, 20%, 30% and 40% go into the score. Let's take an example:
The escape room called “Find the slippers” was rated by 15 beginners (LoT 0), 7 slightly more experienced (LoT 1) and 3 most experienced (LoT 3). The former had an average score of 9.3, the latter had an average score of 8.9, and the latter gave only 8.8. We should end up with 9.3 * 0.1 + 8.9 * 0.2 + 8.8 * 0.4 = .... But, if you noticed, we have no reviews with trust 2, and its weight must go to someone, otherwise the figure will be underestimated. In this case, its weight is equally distributed on the rest, that is, we get this result: 9.3 * 0.2 + 8.9 * 0.3 + 8.8 * 0.5 = 8.93, which for obvious reasons differs from just the average score of all reviews.
There are a couple more nuances: we automatically underestimate if the escape room has less than 30 reviews and / or if all the reviews he has only with zero level of trust (here you have to wait until the contingent of connoisseurs will be more diverse).
Overall, the ranking itself wasn't bad, with some cool escape rooms at the top. Why did we decide to change it? For a long time a lot of moments have accumulated, which we decided to correct.
And what now?
1) The rating was inflexible
Once a less firm rating was formed, it was difficult to change it. This demotivates some escape room owners.
2) The rating was not always up to date
We wanted, firstly, to exclude situations where the escape room was heavily revised and updated, but it was being dragged down by early ratings. Secondly, to exclude situations when the escape rooom gains a minimum threshold of reviews and then does not seek to get new ones. That is, an escape room with 30 reviews and all tens would be higher than an escape room with 300 reviews, where there are scores of all kinds.
3) The influence of masters and pros was excessive
Yes, it could be that the evaluation of one Master player put an end to the career of the escape room :) This is not quite right, especially in a situation where a master gives a rating early on and the escape room is then significantly refined (see point 2).
4) Rating did not take into account the popularity of the escape room
The number of reviews (namely the desire of people to share their emotions), in our opinion, reflects the popularity of the game. And the line between popularity and rating is very thin =)
How does the new formula work?
We won't describe it in too much detail. It looks rather complicated. But we will describe the main factors that influence the formation of the rating.
1.Evaluations for the last 2 years and the level of players
2.Total number of reviews
3.Number of reviews for three months
4.Percentage of confirmed games (when a person books a game on the site and we know that the review can be trusted).
Let's break down each of the points.
Ratings for the last 2 years and player levels
Here we solved the problem of inflexibility of rating and excessive influence of masters. And to some extent the problem of relevance. Previously, the rating was calculated on all reviews, now only on those that have been written for the last 2 years. Previously there were 4 levels of trust with a very large spread of weights (0.1, 0.2, 0.3 and 0.4), now they are 2, and almost no spread. Beginners, amateurs and experienced have entered the first trust level with a weight of 0.33, while pros and masters have entered the second and have a weight of 0.34.
Total number of reviews
We believe that the number of reviews actually significantly reflects the “ranking” of the escape room. On some escape rooms people leave reviews more willingly and more often, and this indicates its popularity, and therefore the rating as a whole. In addition, a greater number of reviews gives a more objective sample and evaluation.
Number of reviews in 3 months
The point is that if the number of reviews is good, a small bonus is added to the rating, but if the number of reviews is too low (less than 5 in 3 months), it can be significantly reduced.
Percentage of confirmed games
It's simple. If a game has been played, then we have no reason not to trust the review =). And if most of the reviews have the icon “confirmed” (blue check mark), then such an escape room will get a bonus to the rating. This should, in our opinion, contribute to the formation of a more objective final assessment.
