Tuesday, September 1, 2009

MMO Review Format

As previously stated, I play, or at least try, many MMOs. Most don’t hold my interest for long, usually less than 2 months. So far the records are held by City of Heroes at 27 months and still going (not counting breaks for other MMOs), Guild Wars that was off and on for over a year, Dungeons and Dragons online at 8 months, and Age of Conan at 6 months. Everything else scores below those numbers.

One of the problems I’ve had is quantifying my feeling and experiences in the games into a system that is understandable to people besides me. Especially one that won’t get excessively long winded, as I often become extremely verbose when talking about a subject I am emotionally invested in (I once turned a 2 hour martial arts class into a 4 hour dissertation on the history of fighting arts because my audience was too polite to tell me to shut up).

The first thing was; what categories did I want to rate games on? I thought about the traditional, but then went a different direction. Every game website and magazine rates things like graphics and sound, usually a 1 to 10 rating or some variation of them. But those are meaningless and suffer from ratings creep. Of course Metal Gear Solid 2 looks better than Metal Gear Solid 1. So, if I gave MGS1 a 7, that means MGS2 needs at least an 8. Since MGS3 looked better than MGS2 it needs a 9, and that would leave me with only a 10 for MGS4. Is MGS4 a perfect game visually? Is it right to compare the appearance of MGS4 to MGS1 considering the different hardware capabilities and the visual precision of TVs between now and then?

So I went with the following categories:

Creativity: Is this something new and interesting, or is it a complete rehashing of previously seen elements? If it is a rehashing, are the elements used in a new and distinct way, or could the game best be described as some other game with new curtains?

Character Models: Do the character models fit the genre or theme of the game? Are they appropriately detailed when compared to the rest of the game? Do they animate smoothly and effectively? This also includes sounds, are the character sounds appropriate? Are the sound effects synced up properly with the character’s actions. This also has to do with how much customization I have on my character’s appearance.

Environment: Does the environment’s appearance add or detract from the game? Does it match thematically with the game? Are the environmental sound effects appropriate? Is the in game music appropriate and does it add to the feel of the game?

Clarity & Playability: Are the in game features, systems, and functions clearly understood and easy to use? Is the User Interface clear and easy to use? Are the menus clearly labeled? Are the help files actually helpful? If I need to refer to the manual, is it clear and concise? Also, can I just sit down and accomplish something in a short period of time, or does this require an investment of time more in line with a second job? Finally, does the fame smoothly transition you around the game world or is it herky jerky? Do you have to do a lot of back tracking and time sinking? Are you suddenly given quests well outside your level range or ability to complete.

Accessibility & Community: This has 2 parts to it. One is, does the game run well on my hardware or hardware you can purchase at major retailers, or do you need to make adjustments to your hardware? Do you need to update drivers or other pieces of software manually, something most users don’t know how or won’t do? Also, are the forums, game community, or people on the in-game chat system helpful, knowledgeable, and polite? Or are they elitist, rude, and dismissive of new players? (You may question that value of that, but that is the number 1 reason why my wife and I don’t play WoW, the community and social interactions of players). Finally, how easy is it to form a team in the game and is Player v Player accessible and balanced? This is a multiplayer game after all.

Replayability & Longevity:
Is this something that you do once and you’ve seen it all? Is there an incentive to make alternate characters? Are there multiple ways to solve quests or play the game? Are there enough quests that you couldn’t hit them all on your first character? Also, is there enough variety in the enemies that fights feel different, or is everyone just a palate swap?

Next was, what system would I use? Numbers without context are meaningless, which is why statistics tend to confuse people. Without knowing what the context of the number you are presented with, how do you really know what that means? If I tell you 9 out of 10 Dentists endorse my product, does that tell you anything? What if I only used retired dentists? What if I only interviewed Periodontists? Or Forensic Dentists? What if my sample set was abnormally small, like 20 dentists? What if I threw all dentists who voted Democrat in the last election out of my sample set? Without knowing about the context under which I derived my numbers, you have no clue if my numbers mean anything (always remember that when someone cites a statistic).

I settled on a 5 tier description system:
Worthless: It’s bad, really bad. “I can’t believe the publisher is putting it out there in this state, bad”. Any game that scores one of these is an instant “Do Not Buy”. It shows a complete lack of caring on the part of the developers and publishers and a disregard for the customers.

Junky: It’s not horrible, but it could be a lot better. It still comes in below being serviceable or adequate. It might get better with time or work, but it shows quality control was lacking.

Serviceable: It does its job, it isn’t terribly impressive, but it isn’t cringe worthy bad. It exists. It will get you from point A, but you park it at the rear of the parking lot so no one sees you arrive in it.

Workhorse: It more than does its job, it does it well. Still room for improvement, but it’s worth including on a resume. I few systems of this quality can pull an otherwise subpar game out of the mire into something enjoyable.

Exemplary: It is at the top, and there is little room for improvement on what it does. This is the way all other games should be doing it.

Finally, and this was the easiest part, what would be my ultimate conclusion. I’m basically going to say, “I recommend you play this” or “I recommend you avoid this”. If this is a game that I think the subscription price is worth, I’ll say so. If not, I’ll tell you to avoid.

I’ll start using this system with my next MMO review, which will be for Fallen Earth.

No comments: