I had always suspected as much, and am glad to see you took the effort to generate some cold hard statistics on it. "8" nearly always translates to "I'm a lazy reviewer who didn't really hate the game, but don't want to piss off the publishers by giving it the 6 or 7 it deserves."

If only gamerankings.com had a service that ranked the gamerankers on what the average score each review site gives a game, so you know how much their opinion is worth. Assuming that a "5" means, "half the games published are better than it, and half are worse", the average score given should be 5. However, who knows what the number really mean. I guess higher is just better.

Stephen King had an interesting commentary on the back page of last week's Entertainment Weekly about the same phenomenon in movie reviews, calling it "4-Star Syndrome".

It makes sense as a grading metaphor:  If I recall correctly (it's been a long time heh), my grade-school's system was something like 74-82 is a C (average), 83-91 is a B (above average) and 92-100 is an A (exceptional).

In any case, you're post is still funny and grumpy, so don't mind me.

Right on, Mr. G.

If your interested, I decided to answer your questions:

Are the vast majority of games released worthy of an 8?

No, I believe the vast majority would be worthy of a 7, since 7 is the perceived average and average generally means "majority."

Should reviews be holding games to a higher standard?

I believe that depends on the motives of the publication in question.  Personally, I prefer to write and read reviews that are very critical.  I suppose my answer to this would be, yes.

Is reviewing games such a subjective matter that ranking them is fruitless?

Perhaps they are fruitless, but I don't think they have to be, if properly implemented.  There are ways to make ratings useful and informative, in my opinion.  However, when static systems are mixed with extreme subjectivity, they are always subject to inconsistencies...

Also, a natural consequence of forming opinions on quality is that "winners and losers" are formed.  A hierarchy of best to worst tends to exist in almost every aspect of society...  Game reviews seem to follow into that trend.

I suppose whether or not ratings are fruitful depends wholly on what you hope to gain from them.  Even if properly implemented.

Do crappy games just not get reviewed?

If this is true, it makes me sad.  It's always amusing to read a humorous review for a horrible game.  Plus, I would assume it gives developers a "Oh, I guess I shouldn't do that" type lesson.

Should games be graded on a curve?

I think that games will always be graded on a curve, whether we want them to be or not.  Games will always be compared to the heavy hitters of the genre, just as developers try to emulate the successes of those heavy hitters.  Take Half-Life, for example.  It introduced quite a few new aspects in the FPS realm, which now all subsequent games are being measured from.  Half-Life set the curve...  I think, perhaps, this is one of the flaws in the "system."  Emulation is the sincerest form of flattery, perhaps, but it's also the battlefield in which great ideas go to die...

Sorry, I may have just typed a bit too much into this comment, but you really got me thinking!

I think 7-8 is a score that leaves the decission to us, so they're not doing their job there.
Maybe we should only attend to our opinion of the games by playing the demos and learning something else about the plot from the reviewers.

I don´t take seriously those crappy scores.

Here's a little exercise in cynicism.

I've noticed many times that a game will get an awful numeric score, but a pretty positive written review. Then, there are lots of games with the opposite. Decent scores, but a written review that craps all over the game. I've noticed in both instances you'll usually see that a big publisher put the game out. Most of the time when I've seen a fully negative review (both written and numeric) the game comes from some tiny publisher who doesn't buy much ad space.

That's the real problem with professional game reviews, IMO: Most magazines make the bulk of their income from selling ad space, not subscriptions. The problem isn't how games are scored (100 scale, stars, whatever) but the fact that publications won't bite the hand that feeds them and tell gamers what they really need to know. Hence the "70-something" hump.

I don't pay much attention to review scores anyway. I buy games based on word of mouth or failing that the box art/blurb, recommended specs and/or price ;)

Yeah. Lately the recommended specs make all the work for me!. I wish I were in the '90s. Sniff :0(

I feel really close to your opinion Hullabaloo.

I've spent way too much time thinking about this topic over the years, so excuse the blathering.

Grade inflation occurs most frequently in 100 point systems, from Gamespot's 1-10 with decimals to PC Gamer's 100% system (to use two prominent examples), for a number of reasons.

Percentages are associated with grades, so basically anything under 60% is varying levels of suck. So the real rating area is pretty much 60-100, matching A-D grades. A couple of years ago, PC Gamer tried to amend this by using more of the 100% system, but I don't think anyone feels that a 60% is really "average." (To its credit, Gamer is sticking with using more of the system.)

Gamespot's rating system makes grade inflation even worse, because by taking a bunch of arbitrary numbers representing "gameplay," "sound," "graphics," etc., and putting them in an arbitrary formula that weighs each in some arbitrary manner, you end up with the world's best game getting a lower rating because it has lousy sound (bringing the upper range down), or the world's worst game getting credit for having decent graphics or sound (bringing it up). On the plus side, this system is almost guaranteed never to offend; if you're always between 6-9, everyone can be happy. Readers won't send angy e-mails, publishers won't send angry e-mails, and developers will still e-mail you scoops. Yay for not taking any strong stands whatsoever.

I used to track all of the ratings for all of the three PC gaming magazines. I did this over a 3-year period, or so, and it went something like this:

PC Gamer 75%
CGW 3.6/5
CGM 3.1/5

That doesn't mean anything, really, but it was interesting.

As for the cynicism, it's easy to disprove the "big publishers/advertisers" accusation, but there's really a very simple reason the text doesn't matching the rating: most editors and writers suck.

It's up to a writer to, when submitting a review, make sure these things match. Alter the text, figure out the proper focus, make sure it's mostly positive for a positive review, or use language that denotes when things are REALLY bad or REALLY good.

But it's really the editors job to make sure that the text matches the rating, and adjust either (hopefully with the input of the writer) when they don't match.

Oh, and there are plenty of other reasons for inflated ratings: People reviewing too many games and/or spending too little time with them (aka "Black & White" syndrome), the groupthink that sets in once the first online review hits and message board chatter deems it "correct," and general cowardice on the part of critics to truly challenge their readers.

I think the latter is important. Too many are more concerned with relfecting what the majority of their readers feel rather than challenge them, and hopefully make them understand why Game X is better than Y, or why Z is overrated.

But with direct feedback online, people are afraid of pissing off readers; at the same time, however, others will gladly toss the hate around just to be daring and controversial.

In either case, if you can make your argument, fine. But then you fall back to the fact that most of the people writing reviews aren't very good at what they do. Most reviews spend more time summarizing features than actually delving into whether or not they're good.

And then there's the review formula. We have, in order: Opening that puts the game in its historical perspective; Talk about: Plot, Mechanics, Graphics/sound, Multiplayer; Mention bad things; then close.

Bah. I'm cranky too.

the groupthink that sets in once the first online review hits and message board chatter deems it "correct," and general cowardice on the part of critics to truly challenge their readers.

No kidding.  Show me a bad DOOM 3 review.

Indeed. Where's the Cahiers du Cinema of gaming? Why isn't there one?

Or is there, and I just haven't seen it?

Be wary of so-called "exclusive" reviews, like that mag that gave 95% to doom3 and just happen to be the first with their hands on a copy.
Using preview press-beta's to make a full 'review' seems to be a year-old trend too, that never seems to stop.

When will hype-based game sales/reviews stop?  Probably when gamers stop buying (into) that crap.

Be wary of so-called "exclusive" reviews already been caught on that I'm afraid with Driv3r. Personally I like the idea of holding games to a higher standard. where 5 would be the average. It would mean we get a more even spread of scores.

The absolute best game review/other gaming stuff  site ive ever been to is a site called 'Game Revolution'. They dont follow any numerical grading system but instead they have an 'A+ to F-' system which is pretty effective. Plus, they state all the pros and cons of the game along with other comments all integrated into a witty, highly entertaining review!!

P.S-No! I have not been paid by the Game Revolution team [if by payment you mean Chinese finger-traps, then I'm the new Bill Gates]

P.P.S-NO! I am indeed NOT on crack. [It's a little thing I'd like to call a 'Hero-juana' [made from fresh bananas]

Isn't that just another way of saying one to, what...? ... 18? Just because it's letters and not numbers doesn't mean it's better...

"Show me a bad DOOM 3 review."

Mine hasn't been published yet.

But DOOM 3 isn't a "bad" game; Sniper: Paths of Vengeance is a bad game. DOOM 3 is a fairly mediocre game with super duper technology. Because of the latter, it's worth checking out just to see how games will look in the future.

People running around saying it's bad are either guilty of anti-hype (i.e. they need to hate a popular game), or haven't played enough truly awful games to have proper perspective.

"Indeed. Where's the Cahiers du Cinema of gaming? Why isn't there one?"

Where are the games worthy of such a treatment?

Games are mostly lowbrow pop-culture, with only a handful aspiring for something greater. Treating them like high-art is navel gazing of the worst sort.

I gotta post some replies to these comments...  sorry.

People running around saying it's bad are either guilty of anti-hype (i.e. they need to hate a popular game), or haven't played enough truly awful games to have proper perspective.

That's a valid opinion with some people.  However, not all.  The result of a review saying it was a bad game depends on what the game is being judged on.  If it's being judged by improper execution, or falling short of its potential, then perhaps it's a bad game.  I think it's another mediocre game, in a sea a mediocre games from id, and it's about time they are confronted with it.  How long can a company keep putting out mediocre games before they are finally considered bad? If you aren't raising the bar, something isn't being done right.  That's just my opinion.

As for: "Games are mostly lowbrow pop-culture, with only a handful aspiring for something greater. Treating them like high-art is navel gazing of the worst sort."

Wow. That kinda sounds like how movies started out...

If you aren't raising the bar, something isn't being done right.

ID is raising the bar, the technical bar.  This industry has a bad habit of confusing the Academy Award for Best Picture with Best Special Effects.

ID is raising the bar, the technical bar.  This industry has a bad habit of confusing the Academy Award for Best Picture with Best Special Effects.

Good call.  I should have been more specific and mentioned I was talking about gameplay.

I've always viewed id as a technology company, rather than a game design company...  Same goes for Epic.

"The result of a review saying it was a bad game depends on what the game is being judged on.  If it's being judged by improper execution, or falling short of its potential, then perhaps it's a bad game."

That's assuming the review is occurring in a vacuum. That's a fine way to review a game, I suppose, but if you've seen truly bad games-those with poor design, poor direction, poor controls, poor gameplay, poor AI, etc.-there's just no comparison. It seems like you're holding id to a higher standard, which isn't fair. A mediocre game from a major company-regardless of its hype-is still a mediocre game.

"I think it's another mediocre game, in a sea a mediocre games from id, and it's about time they are confronted with it.  How long can a company keep putting out mediocre games before they are finally considered bad? "

Fair enough. They would only stop being mediocre when their particular form of mediocrity has been so surpassed by others that it's no longer mediocre but outright bad. I think that requires more of a fundemental shift in quality of all games, not just those from one company.

"If you aren't raising the bar, something isn't being done right.  That's just my opinion. "

There's something to be said for solidly executing existing concepts. Painkiller is a good example. It doesn't raise the bar, but it's a superb game.

"ID is raising the bar, the technical bar.  This industry has a bad habit of confusing the Academy Award for Best Picture with Best Special Effects."

That's true. Despite people always claiming "gameplay>graphics," what they really mean is "gameplay=graphics." It's hard not to marvel at DOOM 3 even when it's boring, though I find it interesting that, based on anecdotal evidence showing it to a few non-gamers, only those with technical no-how find it impressive. Non-gamers just say, "it's dark and sort of gloomy," whereas their eyes light up over World of WarCraft.

But that owes more to the fact that Blizzard has superior art direction. The monsters of DOOM 3 looks like things I scribbled on my notebook when I was 12. "Ooh, flaming skulls, bitchen! Upside down crosses! That's so edgy!"

By the way, anyone who wants to see real art direction and design in a 3D shooter owes it to check out the final level of Painkiller. It is, by far, one of the most creative and jaw-dropping levels ever put into one of these games. Seriously.

"Ooh, flaming skulls, bitchen! Upside down crosses! That's so edgy!"

That's funny!

Steve,

You make some excellent points!  I'm not going to quote you again, since this comments page is getting pretty bloated, and we know what we've said. :)

I agree with what you're saying but I think we might be looking at this from different angles. You are correct, there is something to be said about "solidly executing existing concepts."  If we were talking about reviewing cars, you'd be completely right on about that.  I can picture phrases such as "Still rock solid, as usual!"  However, this isn't merely about function and I think there's a point where you have to say that the "same old, same old" is just that... exactly the same and pretty old.

It comes back to your previous comment, about games not needing a critiquing medium similar to Cahiers du Cinema. Perhaps I'm wrong, but the angle you are taking towards critiquing games seems to be one of function, rather than creativity or entertainment.

The only point that I was trying to bring to the table is that there may be some reviewers that are trying to review Doom 3 based solely on creativity and gameplay, which may explain why they would give it a bad review. This would be a different way of looking at the situation, perhaps, but I don't believe that makes it invalid. Also, reviews such as these would possibly be more useful to the industry than to a consumer. That's one thing that I think the game industry is lacking: reviews aimed at industry professionals - for the "art," so to speak.

After all that being said, I think Doom 3 was mediocre, so I agree with you on that.  I'm really playing devils advocate here, more than anything...  To me, however, mediocre spells boring and boring spells failure. Doom 3 was pretty mediocre and as such, was quite boring, leading to my perception of it being a bit of a failure as a form of entertainment. However, that's very subjective, so I have to give it props for what it did do correctly and leave it at "mediocre."

Lastly: Steve, you have awesome viewpoints and opinions.  Do you have a blog?  If so, I'd love to read it.

I certainly won't contest Steve's point that most writers and editors suck, but I do think there is something essentially broken on a very fundamental level with the American gaming press which goes a little deeper than that. I admit my "big publisher/big ad bucks" theory is facile, not to mention just one charming facet of my own gaming grumpiness.

But there is a very real element that big publishers bring to the table, which the little guys don't: big PR firms. I've personally seen PR reps get bad reviews softened up and/or delayed at major media outlets. Their leverage is usually future exclusives, interviews, access to a developer and the continued good will of the publisher they represent. It's slimy, but it happens more than many people realize.

I can't help but wonder how that figures in Ron's graph. I would feel absolutely GREAT about this biz if it could only be chalked up to bad writers, editors and kooky, inconsistent scoring methods.

sigh Time for some gin.

Alan, I wouldn't say games don't need a critiquing medium; I'm more of the opinion they haven't earned one yet. I don't know the origins of some of the film magazines, but I suspect they didn't pop up until film had really established itself and there was a developed "language" of film.

Maybe the publication created that language, I don't know. But I suspect it's more likely that film was taken more seriouosly after DW Griffith and Fritz Lang came around, then it became an academic subject, and then it moved on to having deeper analysis. Gaming is barely at stage 2. Give it time.

(As for a blog, I don't have one, but you're more than welcome to subscribe to Computer Games Magazine and read my whining every month..)

As for the previous comment from Hullabaloo, I've not worked for any console publications, so I can't speak with any authority on influence peddling there. The dollars and stakes are higher, at least today, so it's possible.

But I've been doing PC game editorial for about 15 years and have never experienced any pressure like you describe, even back when PC games were bigger.

Anyway, editors certainly shouldn't cave in to that kind of pressure, because they need us as much or more than we need them. A PR company that says, "We're not going to do business with you" is effectively shooting itself in the foot, because its worth is determined by... the amount of coverage it can drum up.

I have a simple response to anyone threatening canceling ads and the like: "Can I quote you on that?" It would be a PR disaster for a company if it got out that they were trying to influence editorial. Over the years, we've had a number of companies cancel ads after reviews, but they always come back. And in most cases, they were at the end of an ad campaign anyway.

They can't really cancel advertising because retail uses "support" like that to help them figure out sell-in totals. If you're supporting your game, they want more copies. If they weren't required to advertise, why would they bother? They can get coverage every day online just by releasing a screenshot or writing out daily diary crap where they discuss lunch. Most websites are so desperate for content they don't discriminate at all between what's "news" and what's useless fluff. They'll run anything and everything.

Subscription ho!

I agree with you, games probably haven't reached this point yet... Which is a bummer, since I'm an idealist in some ways and really wish that they would just reach that dang point, already.

Plus, I'm also a realist, and I notice that trends within the industry are starting to move in directions that don't seem to follow what we're talking about. In other words, they are skipping stage two and jumping straight to stage 4: Corporate Sludge. Not that I'm an anarchy flag waving corporation hater, but I think we can all agree that some of the most impressive movies came from independent directors/producers.  Right now, there isn't much of a space for independent work...

Wow, I'm really getting off topic!

Basically, I love games.  I love them as a product and I love them as an art. Video games and the video game industry are my greatest passions. It saddens/angers me when the industry is moving in directions that I perceive to be wrong. Perhaps I'm just cynical.

One thing I would love to see, however, is a critiquing medium for games. There are always many paths to take, in order to reach a destination. Perhaps the movie industry evolved because of their own growth, or maybe a magazine helped them, or maybe something else was involved. Regardless, I think a solid critiquing medium may help propell the game industry in the correct direction, so that it improves as an art form, not merely as a product or piece of technology. I'll be one of those navel gazers, as you put it, and say that I would love for games to become "high-art." :)

PS: You can expect a subscription from me. I was just curious if you had a blog, outside of the magazine. Being that your an editor, and very well spoken, I'm sure you have interesting ideas outside the realm of video games. ;) I just like absorbing as many opinions and ideas as I can.

Good point Steve. I've worked in both the PC and console realm, and don't recall anything like what I described on the PC side of things. Maybe it's some kind of nerdy code of honor.

I think the purveyors of console machines exert greater influence over the review process. I wouldn't call what I've seen out and out influence peddling. It's more subtle than that. Call it spin control. Such and such magazine or site would announce they were ready to release the hounds on a given game. PR Guy would ask them to hold off until the next build, or just a few more weeks, sometimes throwing in some exclusive content. What started as the relatively benign practice of "embargoing" content has basically turned into an open negotiation between the game publication and the game publisher. The game publisher isn't buying a good review, but they may well be tempering a bad one.

In the end, a review can't really be suppressed, but some publications will play ball for a time if they can secure exclusive content. Websites are particularly eager to drive up their unique page views. I think that flabby, not-good-but-not-bad middle of the road review is the end result.

One bright spot here is that everything still comes back to the game itself. Those ornery players will sniff out review disparities and over-hyped games every time. A reviewer once told me "We'll give you all the rope you want to hang yourself." The dark spot is that the practices I describe can only fuel consumer apathy, which hurts everybody.

Incidentally, Once Upon A Time I made a little speech very similar to your last post when I was talking to a PR rep once, touching on all the same issues about how pointless it is to fight with reviewers, and how ultimately self-defeating that is.

They looked at me like I was from the moon.

Actually, movies were "corporate sludge" long before they were independent and interesting, not the other way around - it was called the studio system. It was in this sludge that the reviewers of Cahiers du Cinema found their gems, looking at them in a way that hadn't occurred to anybody in Hollywood.
I think that the really interesting criticism will come (if it ever does) from people completely outside and unconnected to the industry, and will be surprising to everybody. Nobody thought Howard Hawks was a genius until somebody looked at his work from a new perspective.

Anthony,

That's very interesting. I can only speculate as to how things worked/work in the movie industry, since I am no afficiando, not in the least.

Thanks for the enlightenment! It just makes me wish we had such a critiquing system, even more.

Ya, but the problem is more or less that a videogame takes a LOT more time than making a movie. I'm not saying a movie doesn't take time, but with a movie you have the pleasure of having already made the actors, and the way they move.
So a movie is easily made alone, while you need a studio for a game, and even then it takes +2 years.
So yeah. And hell, there are enough independant games out there, but are they even getting noticed? (Looking at how many indy adventuregames there are, you'd think that someone up high starts thinking "What? they still want those?"

Mess,

Just an FYI :)
Though I'm only just starting my video game journalism "career" (I don't plan on making money) I am doing my best to try to recognize the independent games. I've already done 2 stories for 2 different online publications, covering independent adventure games. Especially since one is one is just a small publication, this number is not nearly the kind of saturation needed to generate interest from the public in significant numbers. I'm trying, though.

I completely agree with your observation, you'd think that someone up top would notice after a while. After all this interest, you'd think the publishers would at least consider more "semi-adventure" alternatives. However, it always comes down to marketing/sales, in every industry, and that equates to "tried and true" in some books. So, I understand the logic and motivation behind their thinking. I don't necessarily agree with it. :)

I don't know how much I can agree with the movies bit, though. Large scale movies take a huge team of people, just like large scale games. Small scale movies take much less people but the same goes for small scale games. For instance, Apprentice II (a great & free adventure game) was made by 2 people. Counter-Strike was made by only a few people, in the beginning.  Before that, Action Quake 2 was made by only a few people... All are beautiful examples of independent games.

I do agree, though, that from my experience, making games is more difficult than making some types of movies. I don't know about movies with heavy special effects, because I've never tried to make a movie like that. However, I don't think that the difficulty factor in making games is necessarily a breaking point.  Rather, I see it merely as one of the many hurdles that we have to overcome. While it's a rather large hurdle, keeping most independent developers out of the game, it's still something that can be done and is being done. I think more people just need to become more aware of it. Our community still needs to gel, just like the movie industry did. As Steve said earlier, we have to "give it time." I also think we need to take it one step further and proactively make the changes necessary to evolve the art form. Things don't happen on their own.

PS:
Ron, I'm sorry that I've more or less turned this comments area into a forum...  These discussions are just too interesting to pass up. ;)

Film shoots might only last 4-8 weeks, but the planning for a big blockbuster move can easily take 3-4 years. They start scouting locations, building sets, designing costumes, a lot of directors plan every shot, and they work on technology well before they shoot a single frame.

The film industry just looks more efficient than the game industry, because they've been doing it a lot longer and it's more structured.

As somebody who's worked in the film industry for a few years and just started working in the games industry, I have comments!
A big budget shoot is likely to last closer to twenty weeks, and you're forgetting about post production (my area) which can go on for nine months or more.
I agree that the film industry is a lot more efficient than the games industry, but I think this has something to do with the different nature of games, and not just the maturity of the industry (although thats a factor).

During the making of a film, there's always a place where the film is being made, and everything that goes on supports that place, that moment. It''s usually, but not always, where the director is. While the camera is rolling, the film is being made in front of it, and nowhere else. Everything is focused there.
Obviously, other things are happening (design, cgi, music) but they don't actually get to be part of the film until they have their moment with the director, a moment during which they are the place where the film is being made.

Games (and feel free to correct me - I'm new here) don't seem to have this focus. The game is being made in lots of places, in lots of ways, by lots of different people simultaneously. They're all important, they all interact. There's a hell of a lot less linearity than in filmmaking. It all looks much more chaotic. It all is much more chaotic.

And there's another factor. Films tend to be made with freelance labor - that's the basic industry model. This means that people learn their trade on a bunch of different movies with a bunch of different people, and then bring that experience to a bunch of different other movies, with a bunch of different other people. There are shared standards, well established vocabularies.
Games companies (and again, waiting to be corrected) seem to want to do almost everything in house - there not a lot of mobility. I see a lot of wheel re-inventing. I guess this has to do with paranoia, to some degree (I never had to sign an NDA before this, and I've worked for Miramax!).
But I think it hinders the devlopment of independents. Independent movies are generally crewed by out of work, but experienced, professionals - hoping to garner some good will, work at a slightly higher level of responsibilty than they're used to and gain some experience, or just keep themselves off the streets. Most indie movies that  are any good couldn't have been made if the inexperienced director didn't have access to an experienced crew. These kind of Ronin don't seem to exist for young, ambitious game developers. Do they?

And because of my experience I tend to automatically think in terms of Games::Movies (hence my Cahiers du Cinema comment), but I'm not actually sure it's a helpful metaphor. Maybe up to a point. I'm sorry this comment is so long. I'm here to learn, not to teach!

Well, yeah, I wasn't talking about the Big Scale movies or games (Doom 3 and Spiderman 2, for example) but more along the lines of Indy movies and games. Like Clerks, or Apprentice, wich I am really going to enjoy.

It just seems that after the script is written, the movie can be ready in about 2 months, and games.. hardly.
But yknow, just my 2 cents.

I don't want to argue with your basic point, but that "two months" is just wrong! It usually takes longer to make a low budget movie - when you don't have money, you have to make it up with time. I'd be prepared to wager that it took longer to make Clerks than Spiderman 2.

But this is beside the point. You are still right. Not only does a game take longer, but it takes more people for longer. A movie employs lots of people, but usually only for a few weeks or months each. Very few people are on board for the whole thing, unlike on a game.

Hi,
I work at Treyarch, which is the studio that just finished the Spiderman 2 video game (NOT the PC version). Before that I worked at a very small dev called Cyberlore. My perspective on reviewers is from the victims's side.

Some observations about game criticism:
Game reviews are very harsh on anything that isn't aimed squarely at hard core gamers. This is especially true in the related areas of difficulty and game length. Games that do not personally challenge the reviewer are descrbed as too easy.  Games that are not exremely long are often described as too short. These two often lay off each other. Games are virtually the only form of entertainment where you consider it good if people experience it until their heartily sick of it.


Innnovation in general is praised in the abstract, but not when it is acctually attempted.

Many game reviewers buy into "hype" and are easily dazzled by high end tech. This is especially true because they generally play the game on extremely high end machines.

Game reviewers are often extremely ignorant of the actual game making process, especially compared to movie criticism. For instance, in Spiderman 2 we built a large.streaming model of NYC. By it's very nature it CANNOT physically be as goodlooking as a crafted linear level. There are simply technical limitations that do not allow it. This is not a obscure fact but should be well known to anyone who understands console systems and their memory limitations etc. In many reviews the fact that this was a inherent tradeoff is clearly not expressed.

Game reviews often use a very odd system that apportions percentages of the game to various facets. For instance 30% to sound. These systems usually rate "gameplay" at some very low number (which of course screws little devs which can't hit all the other areas hard) and also doens't take into account that in different games different things matter more. In Thief sound mattered a lot more then in MTG Online.

I've enjoyed reading your posts about the state of the industry et all, I just wanted to post something a little more directly  on topic.

Tom Henderson
Treyarch

Sounds like the Henderson we know and love :)

A review I'd written not so long ago was shown to the game's lead designer and he responded to me through e-mail. It was the scariest thing ever. There are so many aspects to a game's design that its creator knows by heart, but might easily escape the attention of a reviewer. That said, I do think reviews in general have their expectations set way too low. I think reviews should be a lot harsher, not on technical feats or even innovation, but harsh on creativity (or lack thereof).

Everyone keeps talking about innovation, but the age of constant innovation in games is coming to an end. We can no longer expect every game to feature entirely new technical inventions. Creativity is completely underrated in reviews, as far as I see it. I almost typed up a movie analogy, but I'm sick of them, so I'm not going there. But it's obvious that we're going to reach a peak of technical excellence pretty soon, so we'd better improve our analytical skills when it comes to the heart and soul of a game, the experience of the player. (Oh man, I've started doing Ron's thing.)

I feel encouraged by the comments posted by Tom. I have the exact same beef with most game reviews. I just hope game journalism as it is today (or the fan's discussion forums) won't turn developers cynical (or has it already?). I think people do appreciate the kind of things that go into good game design, but that kind of appreciation is still developing and expanding.

When reviewing a game you have to be able to put blinders on when it comes to the superficialities. It's really hard to go deep into what makes the game work and what doesn't, but when you have that down on paper, you can put those blinders off again and look at the presentation and technical feats in a different light. But it's very, very hard. Reviewing is often thought of as something that's very easy, but it isn't (at least, doing it properly).

^^^
On the technical cieling, no we aren't close to reaching it.  But it does feel that way how the technology is being used.
One of the latest fads: rag doll physics.  Nice ... and how does that exactly enhance the player's experience?  Doing puzzles with physics, now that's more something of putting that fancy technology to use.  And how come the last game I really experienced terrain deformation was in sacrifice summoning a huge vulcano.

Very true about creativity being underrated in reviews.  But try the soap-opera anology instead of the movie one.  
Replace "in this episode nancy finds her newly wed husband in bed with ... her sister" with "in this sequel, special operative [lame animal nickname] sneaks up behind a terrorist ... again".

And as for creativity, one glance at the sales charts tell me a lot of people buy sports games.  I have a special aversion to sport games being the fact that it totally oblitterates any creativity, the only difference in sequels being "look I can see hair on his arm" and "it looks even more like on tv".  (with a few exceptions like sega soccer slam or mario tennis that at least try to be different)

Extra note:

The fact that a lot of "those" games are being made is because people actually buy that crap!  What this beloved industry needs a crash, when that fabled 'common gamer' that buys all those games suddenly realises that bladehunt3: deathspank: the third coming: ... again is actually the same as the previous five games he bought.

I think I have spewed enough acid for tonight.

I really fundamentally disagree with people who say that the "common man" doesn't want creativity but wants the tried and true. Man, even Ernest Adams, Jason dela Rocca, et al seemed to think that way at an IGDA chapter meeting last year. At least, I made a point about marketing in an open Q&A session and everyone thought I was nuts for suggesting people in general do in fact appreciate creative or risk-taking games.

The thing is this: creative games often don't get the marketing they deserve. So these games are up against thousands of marketing teams who are trying to convince the masses that Star Wars: Yet Another Game and Medal of Honor 16: We're Running Out Of History are in fact the greatest things they'll ever purchase. Marketing is the art of making people think they want to buy something. When people are being bombarded with marketing messages for familiar brands, do you think they're going to go with their instincts and buy something they've never heard of before, but sounds new and interesting? Most people won't.

Only when proper marketing force is put behind the creative and risk-taking games - he same kind of marketing the derivative games get - only then can we really know for sure if the public really hates quality (which I don't think they do).

Okay, this post might make somewhat of a charicature of the issue, since some really great games do get a full-on marketing push, and some really bad games don't, but hopefully there's a point somewhere.

"I think reviews should be a lot harsher, not on technical feats or even innovation, but harsh on creativity (or lack thereof)."

How is putting creativity over other elements any better then doing the same for graphics?

Isn't execution always more important than creativity? You're not reviewing what a game might be like, or what people will do with this idea if it's better evolved. You're reviewing how well it's executed.

Since we're at Ron's site, was Monkey Island 2 a 3-star game solely because most of its "creative" elements were established by the first game? How is that fair?

I think we're just working with very different definitions of creativity. You're using it in the same way as the word innovation is used, except applied to the (non-technical) premise of the game. That's a pretty narrow use of the word creativity. Isn't it all about the power to create, ranging from solving a problem in a unique way to coming up with an imaginative setting?

Also, how does creativity not apply to the graphics? Or programming? Or all of the game's execution?

You know what, I'm an idiot.

I was thinking of innovation, not creativity. Duh.

No, you are not a statistician. I was unfortunate enough to have to do statistics at an Advanced level in school (It's so cool, I can say it's an advanced level, when all I got was a British, A-Level, something most students my age do in Britain anyway) and basically, your posing questions on single reviews for games from a single website (I assume) which means that the average 7 seems to be once persons opinion (even if there are numerous reviewers, they wouldn't all be reviewing the same game so it's one persons opinion on each game) and yet your questions seem to be about game reveiwers in general.

What you need to do is do the same thing on other websites (a good sample would be 20), and then average the averages to find a common average (am I overuseing that word... single?).

And then repose your questions if they still stand.

Another (better) method would be to average out each games score from  20 websites and then put those averages into graph form as above to find a real average.

Of course, that would include getting avereages for 6000, meaning that in theory, you'd have to look at a total of 120000 reviews, which would just be stupid and a huge waste of time.

So instead, a sample of the 6000 games would be more adequate (again, 20 would do, and your sampling method would be totally up to you, but I suggest every 300, just to be on the safe side). Even still, it would be 400 reviews to be averaged into 20 averages, but it would be a better statistic.

Of course you are reading a comment by a guy who failed his statistics final, but I got the overall Advanced level Maths mark (plus, I failed in stuff  like distributions and crap)

Steve: heh. Damn. I was looking forward to a lengthy and pointless argument ;)

http://www.gamerankings.com has review scores of pretty much all sites, maybe you can ask for a chunk of their database for 'scientific research'.

Or just grab their results and pawn them off as your own "research" and pose you're questions to that. Statisticians do it all the time, how else can 150,000 people die of alcohol abuse every year in my town? My town has a pupulation of 98,000.

Of course we do drink like horses and breed like rabbits, but that's besides the point.

(Posted by Marek
when it comes to the heart and soul of a game, the experience of the player. (Oh man, I've started doing Ron's thing.)

This is one of the things that I'm starting to put extra-value on, the 'feel' of a game which is a combination of many things, graphics, writing, character, gameplay content and how it feeds into creating this atmosphere.

For example Frontier's Dog's Life for PS2 is a game that I love the feel of, it is set in a mischevious small town American setting, with relaxing music, almost no violence, and some decent little challenges. It was just a pleasure to roam around the beginning levels in a non-linear way.

So I think that feel is a definite area of interest, what sort of feeling do you want the overall experience to produce, what sort of themes interest the player?