metzomagic.com Feature

Adventure Games: The Statistics of Prejudice

By Chris Tolworthy (August, 2003)
"Mainstream game reviewers are biased against adventure games." That is a common refrain on this web site. To anyone who doesn't love adventure games, the complaint must look like simple prejudice. Just whinging. Whining. We adventure gamers are paranoid. Well maybe we are. But we are right to be paranoid. They really are out to get us. Here is the evidence:

A few weeks ago I purchased a copy of "The Ultimate Guide to PC Games." It contains ten years of game reviews from the pages of "PC Gamer" (arguably the number one games magazine in the UK). Two and a half thousand games are reviewed, in alphabetical order, and each game is then given a score out of 100. Each game is identified by its genre (RPG, Sports, Platform, Adventure, etc.) so this seemed like a perfect opportunity to test our thesis: are mainstream games reviewers biased against adventure games?

Methodology
Rather than analyse every single game (which would take days) I first chose a sample: all the games beginning with numbers, symbols, or the letter "A." In other words, every game up to page 31 in the book, a total of 176 out of 2507 games, or seven percent - a reasonable sample size. Later I will look at a larger sample in less detail.

Of the 176 games in the sample, 160 were not labelled adventure. The total of all their scores came to 10730. So the average score was 67.1 This included a wide range of games, with a wide range of scores, from "Action Man" (scoring just 4 out of 100), to "3D Lemmings" (scoring 94).

Sixteen of these were labelled "Adventure." Judging by the reviews, at least three of them were not real adventure games by any measure accepted on this web site. Others appeared to be cross-genre. But in order to be objective, I have accepted the reviewers' definition. However, I have included the "adventure" names and scores, so anyone with more experience of these games can move them into the "non-adventure" category and re-calculate the results. They are:

  1. Ace Ventura (34)
  2. Aladdin in Nasira's Revenge (38)
  3. Alice (81)
  4. Alien Earth (54)
  5. Aliens: a Comic Book Adventure (53)
  6. Amerzone (29)
  7. Arcatera (43)
  8. Are You Afraid of the Dark (77)
  9. Arthur's Knights II (42)
  10. Arx Fatalis (82)
  11. Asterix and Obelix Take On Caesar (33)
  12. Atlantis (72)
  13. Atlantis II (35)
  14. Atlantis III (52)
  15. Atmosfear (26)
  16. Azrael's Tear (64)

Total of all scores: 815

160 non-adventures is a reasonable sample size, but only 16 adventures is maybe too few. Perhaps there is a curse on adventures beginning with the letter "A." Let's look at our inventory to find a way around this <grin>. OK, we will look at a hundred or so. Because of time constraints I will not list them individually. Again I have accepted the reviewer's definition of adventure, except in about six cases (such as the high scoring Grand Theft Auto, or the low scoring G.I.Combat) where the game is very obviously wrongly labelled as "adventure" when it is not.

Taking all the adventure games up to the letter L gives us a total of 106, and includes many high-scoring games such as the Discworld series, Grim Fandango, Full Throttle, Escape from Monkey Island, etc. These bring the average up, but there is still a large difference between the scores given to adventure games and the scores given to other genres. The large sample size means we can say with confidence that this is not a coincidence or a statistical anomaly. This signifies a real difference.

Results
In the first sample, non-adventure games had an average score of 67.1. Those labelled as Adventure had an average score of 50.9. This 67.1 is 1.32 times 50.9. So put another way, the average non-adventure game in this sample scores about 32% higher than the average adventure game.

The second sample includes more games, and most of the big names. To make it easier to check these results, I have listed the games by letter, giving the number of games and their total score. This takes us up to page 143 in the book, and includes just under half of all major adventure games published between 1993 and 2003:

  1. 16 games, total score 815
  2. 17, 1054
  3. 12,  753
  4. 15,  805
  5.  2,  103
  6.  8,  521
  7.  7,  412
  8.  7,  338
  9.  5,  357
  10.  6,  240
  11.  1,   88
  12. 10,  586

Total: 106 games, total score 6072

Average score 57.3

To summarise, 160 non-adventure games averaged a score of 67.1. 106 adventure games averaged a score of 57.3. This 67.1 is 1.171 times 57.3. So put another way, the average non-adventure game scores about 17% higher than the average adventure game.

Statisticians would expect these things to follow a "normal distribution." Anything up to a five percent difference would generally be ignored as insignificant. Given the still modest numbers, we might even say that anything up to ten percent difference could be overlooked. But a seventeen percent difference? Statistically, this is very significant. It is highly unlikely to happen by chance. We can safely conclude that adventure games are systematically given much lower scores than non-adventure games.

Reasons
Let us look at the reasons given by the reviewers for some of the lower scores. These are most enlightening. Ace Ventura is "deliberately frustrating" (D'oh!). Aliens has "ugly, jerky, pixellated sprites." Amerzone is "a trawl for hotspots... a shoddy Myst clone." Arcatura is "too obscure for adventurers [how would they know?]" Arthur's Knight's "disjointed puzzles make this an exercise in frustration." Atlantis II has "illogical, arse-numbingly daft puzzles." And Atmosfear is "a mind-numbing puzzle-fest."

Perhaps the reviewers' priorities are summed up in their review of Atlantis III: they blame their score of 52 on "a barmy, random story" but think the female lead is "really pretty... So, 50 percent for the girl and two percent for the pictures. Can't say fairer than that."

Indeed. They can't say fairer - or they don't want to.

Summary
So what have we learned? Nothing we didn't already know. Reviewers like flashy graphics, action scenes, girls, speed and shooting. They find puzzles to be frustrating. This reflects the "12-14 year old male" stereotype that has been commented on many times before.

So it is no surprise that non-adventure games are consistently scored much higher than adventure games, regardless of game quality.

So yes, we adventure gamers are paranoid. And we should be. They really are out to get us.

Update ... September, 2003
I had some spare time this week, and finished reading "The Ultimate Guide to Computer Games." Here are the remaining figures. The situation is worse than I thought.

In the above article I found that the 106 non-adventures (up to the end of letter 'A') averaged 67. Further analysis showed that this average was remarkably consistent across the remainder of the book. However a comparative sample of adventure games up to the letter 'L' averaged 57. But that group included a number of big name games, particularly from Lucasarts, the reviewers' favourite developer.

When we include all adventure games, right up to the letter Z, we find that 220 adventure games were reviewed by 'PC Gamer' magazine between the years 1993 and 2003. (I excluded about ten games that, though labelled 'adventure' were purely platform or shoot-em-up, and I included one - Syberia - that is generally regarded as Adventure but was mislabelled as something else. If I had not adjusted for these, the average score would have been even lower, because the reviewers seem to call anything 'adventure' that they cannot understand or enjoy). The cumulative total score for these 220 games was 11743. The average score given to an adventure game was therefore 53.4%. 

A fairly typical (and revealing) example is the original game of Myst. Now I have not played Myst, but it appears to have been widely copied - numerous games are referred to as "Myst clones." Plainly, Myst had a great influence on people. People liked it. People bought it. People made games like it. It was apparently one of the great games of all time. And what score did the reviewers think it warranted? 59% The reason? The puzzles are "uberhard" which makes it all "dull." 

Remember those numbers. 53% for adventure games, 67% for all others. 67 is about 1.25 times 53. Think what this means. If you had a very poor adventure game, and a reviewer gave it a score of 4, an equally bad non-adventure game would be given a score of 5. If you had a mediocre adventure game that scored 40, an equally mediocre non-adventure game would be given a score of 50. If your excellent adventure game scored 77, the equally excellent (but no better) non-adventure game would be given a score of 96. What more can I say?

Copyright © Chris Tolworthy 2003. All rights reserved.