Friday Slide: Bad Reviews, Good Policy

Loyal readers may have noticed that STP has been handing out more 1s and 2s over the last month or so than we’ve been known to in the past. What’s the deal? Are iPhone games actually getting worse, or have I just been in a terrible mood recently? Maybe it’s the post-Holidays slump.

None of the above, I assure you. There are still plenty of great games on the App Store, and I’m in fine fettle. The main phenomenon at work is something social scientists like to call selection bias. To a casual observer, it may appear that there’s been a glut of rotten games coming out recently… but that’s without any knowledge of my methodology for selecting which games to review. There’s no way in hell a small outfit like ours can review every game, given that 10-20 new titles come out every single day. I’m not sure all the major review sites working together could handle that kind of volume for long. So, we have to be choosy.

As far as I’m concerned, this selection is far and away the most important thing STP does–more important than individual reviews, news coverage, reporting, or anything else. It cuts to the very core of why this site exists in the first place: to tell our readers about the best games on the iDevices AND to save them money. A lot of sites are all over the first part of that equation, but they neglect the second. They tend to avoid reviewing many bad games.

I can’t really blame them for that. No gamer likes to play bad games for any length of time, let alone go over them exhaustively during a review. But over the years, I’ve come to appreciate the value of systematically dissecting busted-ass products. If you force yourself through enough of them, you begin to recognize patterns of errors that developers tend to repeat from game to game, depending on the platform and the genre you’re looking at. Over time, this harrowing experience makes you a better reviewer. You cultivate a sort of predictive sense for design flaws, allowing you to cut to the chase and focus your energies on the elements of the game that really matter; on the flip side of the coin, you are better able to appreciate games that step around or bridge common pitfalls (I’m still calibrating for iPhone games, incidentally).

Ultimately, you exit the gamer mindset altogether–where you are playing games primarily for your own enjoyment–and the process becomes almost clinical. That idea may horrify some of our readers, but that’s what I strive to do when I write a review. I’m a big believer in artistic merit in games, and I certainly appreciate the gaming experience from a holistic standpoint… but I also know all too well that game development is a highly regimented, technical process. Every game ever made boils down to design (controls, objective), assets (art, sound), and the code that glues them together. I think that it IS possible to take everything apart and weight it objectively.

I digress. The main point of my argument has to do with the utility of games criticism. My basic assumption is that it is more useful to you, our readers, to be warned about a bad game you might otherwise buy, than to be alerted to the presence of an excellent game you might otherwise pass over. We are loss-averse creatures; in the App Store context, this means that wasting money on a bad game pisses us off more than purchasing an equally good game pleases us. Plus, there is generally less reliable information available for bad games, and customers burned by unreliable information (*ahem*userreviews*ahem*) will be more likely to ignore it the next time around–if they continue to purchase games at all.

Think about it. The App Store is filled to the brim with games that look really cool, but how many iPhone games are actually worth buying? Maybe ten percent, at the absolute outside? If you pick a random game out of the bin without reading any reviews, you will most likely have a poor experience. But nobody picks at random, of course–games with important brands, catchy names, high star rankings and so forth are going to get more business.

That’s why I think that the iShoot review may have been my most important review yet. Here is the #1 paid game on the entire App Store, averaging a ludicrously high 4.5 stars out of 833 reviews. That’s a huge volume of positive data, and as far as I can tell, it’s completely undeserved. iShoot isn’t a bad game, per se, but it’s no quality leader either. Someone who doesn’t play many iPhone games might buy it on the strength of that information, assume that this is the best the iPhone can do, and stop bothering with iPhone games altogether.

Given that the quality bar for iPhone games is constantly on the rise, and that iPhone developers increasingly seem to be figuring out how to game the App Store’s enormously janky automated systems (or least market their less-than-stellar games more effectively), I feel duty-bound to slap down as many bad games as I can.

Of course, we’ll continue to shout out the great stuff when we find it, too. I’m no martyr!