Game Reviews: How Can the Practice be Improved?

[Variable Gear]

I've been wondering about the modern age of reviews for a long time. We're seeing quite a few cases of reviews as buying advice during a time period where the number of demos available has skyrocketed. In addition, buying the game is not the only way to play it. Many people use GameFly in order to experience some of the shorter games of the year that "don't deserve to be bought." There are also brick and mortar shops that allow their customers to rent games. So, obviously, the consumer's decision isn't as simple as "do I buy the game or not." Instead they ask, "is this game worth experiencing?"

Reviewers often answer with lines such as "I'm sure you've had this game pre-ordered for months, and let me just say that the wait was well worth it" and "Only fans need apply." Who are they speaking to? If you've had the game pre-ordered for months, why are you reading the review? Same goes to those who are fans of a specific genre or series. Reviewers also have a hard time giving a game a legitimate critique, because they often rate a game before its release date. This means that they run the risk of ruining the experience for everyone who bought the game if they reveal anything that could be construed as a spoiler. It's difficult to give an analysis of a game if you are unable to speak in specifics about the areas where the game succeeds and fails.

There's also the disconnect between the score and the actual text of the review. I'm sure we have all seen reviews in which the game gets docked a ton of points for a few minor issues or reviews in which the reviewer is able to ignore a wide array of issues and still give the game a high score. In situations like this, there seems to be no similarity between the reviewer's critique and the reviewer's rating.

The environment the game is reviewed in can also be brought into question. For example, being flown out to a hotel for a three-day period in order to review a game with a PR person constantly in your ear can't be the optimal environment to experience a game in. Being able to sit down on your own, for days/weeks at a time, and plug away at the game seems to be the best solution, as that is how the consumer will be playing the game.

Just as well, the fact that reviewers are often given a free copy of the game to review doesn't help my confidence level. This is especially true in cases of "reviews as buying advice." It bothers me when reviewers imagine how wise it would be to spend $60 on a given game. "This isn't worth buying, but it would make an excellent rental," one reviewer typed. Really, did you imagine that?

Another potential stumbling block is a game's multiplayer mode. Reviewers often play games on a debug system, on a network similar to, but not equivalent to the one used by consumers. They compete and co-operate with other journalists, who have their own debug systems. Additionally, many modern games, console or PC, have beta tests. In the same way that reviewers comprise a small group of the gaming population, so do those that get into the beta. However, once a game gets into the wild, it can still suffer a variety of issues that are impossible to test in a closed, beta, pre-release environment. You won't really know how minimal wait times are, or how many exploits there are for less honorable players to take advantage of, before the game is released. A minor tweak to the current system would be to not review the multiplayer portion of a game, or do follow-ups throughout the game's online lifespan as new patches, maps, and expansions are released.

I'm curious what our community thinks about the current state of reviews. Do they need to be scrapped entirely, or can they be improved? Are scores important, or should they be shelved? Is the environment a game is reviewed in as integral as I made it sound? Did this make any sense?

For your further education, I present you these links, which I have read or am in the process of reading.

Additional Reading:
Unrealized reviews symposium: The One That Got Away (http://shawnelliott.blogspot.com/2008/11/unrealized-reviews-symposium.html)
Commencing Countdown (http://shawnelliott.blogspot.com/2008/12/commencing-countdown.html)
Reviewing, Criticizing and Games Media (http://flashofsteel.com/index.php/2008/12/01/reviewing-criticizing-and-games-media/#more-1112)
Shawn Elliott's questions (http://etelmik.blogspot.com/2008/12/shawn-elliotts-questions.html)
Reviews symposium: Partly, badly realized (http://insultswordfighting.blogspot.com/2008/12/reviews-symposium-partly-badly-realized.html)
Inside the Reviewers Studio (http://www.joystickdivision.com/2008/12/inside_the_reviewers_studio.php)
Reviews symposium; Because I like the look of my name published next to articles (http://graffitigamer.com/?p=231)
Can't Wait Till Tomorrow (http://www.escapistmagazine.com/articles/view/issues/issue_106/1286-Cant-Wait-Till-Tomorrow)

[Generation ABXY]

Just as well, the fact that reviewers are often given a free copy of the game to review doesn't help my confidence level. This is especially true in cases of "reviews as buying advice." It bothers me when reviewers imagine how wise it would be to spend $60 on a given game. "This isn't worth buying, but it would make an excellent rental," one reviewer typed. Really, did you imagine that?

Interestingly enough, this is one point that actually struck me one day. I was playing through Genji: Days of the Blade for the PS3, a game I bought along with the system (that is to say, with my own money), when, despite having read some pretty awful things about it, I found I was actually enjoying it quite a bit (though I should add that the camera really is awful). That’s when I realized that this could really go either way. On the one hand, having invested nothing in it and therefore having nothing at stake, the reviewer may not have any reason to look for a game’s high points, to see where the real value is. On the other hand, they are getting a free product and, while they still have to work for it, it is money they didn’t necessarily have to spend, and so they may feel inclined to heap praise upon it for that reason.

Either way, I can’t say I really consider it a threat. Does it ever skew the scores? I'm sure it has, but I have to say that I think being a consumer is a shared experience, and these people likely did have to pay for games at one point in their life – I mean, they weren’t born reviewers – and know exactly what it feels like to waste money on a crappy product. I have to believe that helps to balance it out.

[Orca]

I don't see how the review copy being free affects your decisions on a review one way or the other. If the company didn't supply it, then whoever you're writing for would.

In either case, it comes down to deciding if, having played it, you'd be satisfied if you had spent the $40-60 that the game would cost. It's not really that difficult.

A post-release review of multiplayer components would be more accurate, but how much value would they be? Video game sales tend to be front-loaded, so for most gamers it would be a waste and reviewers have enough on their plate as it is.

I could see a site that does nothing but review the multiplayer component doing well, providing they could get up in-depth reviews quickly enough after release.

[Variable Gear]

I don't see how the review copy being free affects your decisions on a review one way or the other. If the company didn't supply it, then whoever you're writing for would.

In either case, it comes down to deciding if, having played it, you'd be satisfied if you had spent the $40-60 that the game would cost. It's not really that difficult.

A post-release review of multiplayer components would be more accurate, but how much value would they be? Video game sales tend to be front-loaded, so for most gamers it would be a waste and reviewers have enough on their plate as it is.

I could see a site that does nothing but review the multiplayer component doing well, providing they could get up in-depth reviews quickly enough after release.
Generation ABXY and I conversed about the provided review copy issue, and we irrevocably proved that it does in fact influence review scores. How much or little is up for debate, however. I don't want the reviewer imagining anything, including what it would be like to pay money to buy the game.

Also, this "quickly enough" stuff needs to go away. If you are buying the game, you'll likely know before the reviews go live. Reviews, in my opinion, need to change into more mature critiques of games instead of a simple Buy/Rent/Pass statement based on the game's technical mastery. I mean, there are still those who depend on reviews or Metacritic to make their decisions for them, but that population is reduced whenever it rains. Also, post-release content from a major site? Don't make me laugh. That shit almost never happens.
(There are sometimes "postmortems," but that's as deep as the game journalism industry gets.)

Your multiplayer site review theory is possible. If a co-op site can exist, so can a multiplayer site.

And I'll say it again, please check out the links in the OP if you haven't already. There are some interesting perspectives contained within.

[KingGorilla]

The audience needs to grow up. And then maybe there will be more demand for writers like Shawn Elliott or Jeff Green. But with the market as an indicator. The majority will stick with Maxim Magazine's editor running Gamespot. IGN selling reviews and scores. Joystiq putting up blogs and articles with no merit or research just for page views.

Maybe the answer is in Sore Thumbs or Giant Bomb.

A big thing is that either retail games need to get to press a month before release, or the press needs to stop clamoring to yell FIRST! Press needs to stop serving as free marketing for the game companies and start acting as consumer advocated. A fan industry has no business in giving advice, in being critical, in telling their opinion at all.

[Wilkz07]

great post. i lean more towards average score (gamerankings.com) and player reviewers rather than reviews by those who do it for a living (i.e. biasd ign). I've played some games that get really poor scores but have a lot of fun playing - Cabela's 'any kind of' Hunting. They are pretty basic but still a lot of fun to play and won't ever a good score.

good i got the gamerpass from blockbuster. now i can try out all the lower scoring games to see which ones are hidden gems even though they aren't halo realted or have mp.

[Karak]

When I reviewed we got the games free. For me it felt that if I trully stuck with a game for a long time and felt the want to play it even when I was done with the review build...well that was a special game.
So free can influence a review in both ways. For me I felt it was a honor to review so I gave games what they deserved when it came to time and chance.

As for how it could improve.
I think video reviews are an excellent addition. Graphical glitches, install problems and so forth could be show on these. They have no yet really been used for that, but for PC gaming that would be incredible.

I also think that funfactor should be DEMANDED in every review. Of course its not the same for every reviewer but if a person says, this game is awesome looking, sounding and playing, but I didn't have fun I could still see buying it. Because...maybe I would like it. But I have always liked when people said their base emotional enjoyment of something. It doesn't always need numbers.

[Orca]

Also, this "quickly enough" stuff needs to go away. If you are buying the game, you'll likely know before the reviews go live.

The majority of sales for a video game are front-loaded, and yes - people who make up their minds based on reviews still tend to be in that 'first week' crowd, so that's when a review needs to be up. Once you're past that first week it's a case of diminishing returns - the website that's putting up that review needs it to generate hits after all. It's a business on both sides and timely reviews help both sides...unless the game sucks, of course.

Reviews need to be, at the least, day and date of a release. I'd agree that the best solution would be for publishers to get games - retail versions - into reviewer's hands earlier.

I still can't buy into games being free holding any influence over reviewers and their scores. That's the least of the moneyhatting excuses that are out there.

[Variable Gear]

The majority of sales for a video game are front-loaded, and yes - people who make up their minds based on reviews still tend to be in that 'first week' crowd, so that's when a review needs to be up. Once you're past that first week it's a case of diminishing returns - the website that's putting up that review needs it to generate hits after all. It's a business on both sides and timely reviews help both sides...unless the game sucks, of course.
There are sites that exist to generate ad revenue, and then there are sites that exist to allow easy access to well-written reviews. Well, that's an oversimplification, because there is the rare occurrence of overlap from time to time. Still, I'd argue that the 'first week' crowd has already made up its mind. As well as the 'second week' crowd. Perhaps the third week of potential buyers is already decided as well. Reviews exist in a nebulous space that is half justified and half unjustifiable.
Reviews need to be, at the least, day and date of a release. I'd agree that the best solution would be for publishers to get games - retail versions - into reviewer's hands earlier.
The best solution is for journalists to take as much time as they need to complete a game. It's unwise to have your review go live when the game comes out, because then you have to pussyfoot around spoilers and the like. It's best, in my opinion, to be less timely and have a little more to say.

[Jeffool]

I've been wondering about the modern age of reviews for a long time.I get what you're saying about game reviews being product purchase recommendation, rather than game quality critique, but that's just a different on the choice of words, really (and I agree with you, but still. The chasm isn't as wide as I believe you think.) Saying 'buy it', 'rent it', and 'avoid it', is just like saying 'must play', 'eh, it's okay', and 'avoid it.' So, those small things aren't really the point, moreso the larger review in itself. I, like you I'm guessing, would love to see more 'critique' over review. But such things would have to wait until long after the game came out, and you and I would have already made up our mind to buy the game or not. You're arguing for critique, and I agree, but it doesn't have to come at the expense of reviews.Reviewers often answer with lines such as "I'm sure you've had this game pre-ordered for months, and let me just say that the wait was well worth it" and "Only fans need apply." Who are they speaking to? If you've had the game pre-ordered for months, why are you reading the review? Same goes to those who are fans of a specific genre or series.People who've pre-ordered a game months earlier will no doubt absorb all media about that game. They're reading reviews avidly. What they're reading for is in case everyone's reviews start out "Cancel your pre-order, they bombed." Said gamers do cancel pre-orders, and rent instead, because they hope against hope that the game is still fun.Reviewers also have a hard time giving a game a legitimate critique, because they often rate a game before its release date. This means that they run the risk of ruining the experience for everyone who bought the game if they reveal anything that could be construed as a spoiler. It's difficult to give an analysis of a game if you are unable to speak in specifics about the areas where the game succeeds and fails.I would say let'em spoil, but Roger Ebert manages to review films all the time without spoiling them. Say what you want about the guy, but he's a hell of a film reviewer.There's also the disconnect between the score and the actual text of the review.That's because a score as anything more than a general guideline is just silly. 1-5, thumbs up/down, A-F, those are fine. You get into one hundred point scales and you're just being an asshole.The environment the game is reviewed in can also be brought into question. For example, being flown out to a hotel for a three-day period in order to review a game with a PR person constantly in your ear can't be the optimal environment to experience a game in.Then it's completely the responsibility to reviewer to ask the guy to stop talking. And if you free trip gets a better review, then your faith in reviewers as people is horribly shitty, and I doubt you'd trust anything they'd do. I mean, you think (again) Roger Ebert waits in line on opening day? Hell no, he's flown to press screenings. So I have no problem with reviewers being given free copies. To me, it's silly to have a problem with that. Again, this just seems like your distrust is the people completely. If you don't trust them to review a free product, you'll never trust them to review any product.

And I can't speak to how reviewers do multiplayer, but games altogether have so many possibilities due to their interactive nature that most bugs will go completely unseen by a reviewer. Look at all of the bugs in Fallout 3. I'm certainly not going to be pissed at a reviewer for not going into detail about them more than 'there are some bugs.'

And to wait until the game is widely available? That's just bad business; way to make your reviews pointless to most gamers. Like I said earlier, advocating serious criticism isn't the same as eradicating reviews.I'm curious what our community thinks about the current state of reviews. Do they need to be scrapped entirely, or can they be improved?I think they're (largely) fine as is. They serve their purpose, even if that's not what you want from them.Are scores important, or should they be shelved?For reviews, I understand them. I like the 5 point scale myself: 1-trash, 2-not my thing, 3-okay, 4-good, 5-a rare paragon of gaming goodness.

Is the environment a game is reviewed in as integral as I made it sound?Nah.Did this make any sense?Yeah, some. But I don't think wanting a serious discussion on game quality and effectiveness is something that has to replace the current system of game reviews. I don't think most gamers want to read about about how nearly all open-world game's side-missions hamfistedly break the flow of the game's main plot if you decide to do them. Or that they completely disrupt any narrative flow that the game forces you into for any quality content, rather than let you carve your own story in the world given. Most gamers don't care, so mainstream reviews won't go in that direction.

They just want to hear if side missions are few or plentiful, and if those are repetitive or not.

Hell, I'd say that mainstream gamers are only just now starting to realize that a game doesn't have to break down barriers visually to be worth buying. The last thing most care about is narrative flow or the moral premise of a game, and most games just aren't made with such things in mind.

[Variable Gear]

I get what you're saying about game reviews being product purchase recommendation, rather than game quality critique, but that's just a different on the choice of words, really (and I agree with you, but still. The chasm isn't as wide as I believe you think.)
It's more than a different choice of words. A review is a simplified rating of the game's strengths and weaknesses, while a critique delves deeper into the design of the game and the philosophy behind that design. Most modern reviews have a bit of both, but there's a higher percentage of workmanlike rating than involved critiquing. That chasm is pretty wide.
Saying 'buy it', 'rent it', and 'avoid it', is just like saying 'must play', 'eh, it's okay', and 'avoid it.' So, those small things aren't really the point, moreso the larger review in itself. I, like you I'm guessing, would love to see more 'critique' over review. But such things would have to wait until long after the game came out, and you and I would have already made up our mind to buy the game or not. You're arguing for critique, and I agree, but it doesn't have to come at the expense of reviews.
Like I've said, I think it's wise to wait a long while before putting out a review. For one, this gives the reviewer access to the entire game, not just the bits that aren't in danger of being spoiled. Anyone who's anyone has already played the game to completion, allowing the reviewer to discuss in detail just how well everything came together. Secondly, I'm not arguing against reviews. I'm just saying that the policy of writing down some words and a number and pretending that there is any connection between the two is ridiculous. I'd like to see as little of that as possible, which I why I don't read many reviews anymore.
People who've pre-ordered a game months earlier will no doubt absorb all media about that game. They're reading reviews avidly. What they're reading for is in case everyone's reviews start out "Cancel your pre-order, they bombed." Said gamers do cancel pre-orders, and rent instead, because they hope against hope that the game is still fun.
That's great. I'm so happy for them. I still think it's a really fucking stupid thing to jam into a review.
That's because a score as anything more than a general guideline is just silly. 1-5, thumbs up/down, A-F, those are fine. You get into one hundred point scales and you're just being an asshole.
Some sites still have 10 point scales, which I'd argue don't work very well. Then there's the issue of IGN and its 100-point scale. Half-points on a five-point scale is almost as bad. I dislike any kind of dissonance that results from the presence of the score and the text itself. Therefore, I dislike scores. Pretty simple, right?
Then it's completely the responsibility to reviewer to ask the guy to stop talking. And if you free trip gets a better review, then your faith in reviewers as people is horribly shitty, and I doubt you'd trust anything they'd do. I mean, you think (again) Roger Ebert waits in line on opening day? Hell no, he's flown to press screenings. So I have no problem with reviewers being given free copies. To me, it's silly to have a problem with that. Again, this just seems like your distrust is the people completely. If you don't trust them to review a free product, you'll never trust them to review any product.
Do you not understand the word "irrevocably?" Also, what's this about a "free trip" resulting in a better review? I made no such comment. You made up something about it, though. One, there are no free trips. Two, I am very skeptical of any situation where a game reviewer is given special treatment, be it being flown out to review the game or whatever. I am not sure why I should be chastised for perhaps thinking that this could have influence on weaker minds.

In addition, the issue of review copies is similar, but smaller in scale. The reviewer was given a free copy of the game, and that has an effect. I didn't argue that it would have a huge impact, however, I did argue that it would have an impact. Getting something for free (even if it's earned, as it is in the case of reviewers) paints their perception of the product in a way that the general public will have a difficult time relating to.
And I can't speak to how reviewers do multiplayer, but games altogether have so many possibilities due to their interactive nature that most bugs will go completely unseen by a reviewer. Look at all of the bugs in Fallout 3. I'm certainly not going to be pissed at a reviewer for not going into detail about them more than 'there are some bugs.'
That's cool. I have no idea what you are talking about. Multiplayer is an actual issue worth discussing. Bugs are not. Saying "there are some bugs" is enough, unless there are some show-stoppers that deserve coverage. Again, reviewers feel the need to review the game before it's out there in the hands of the masses, and this means that their multiplayer critique should be read with great trepidation. They should produce follow-ups on the multiplayer mode as time progresses to track if the matchmaking system works properly, if the game doesn't have any outstanding exploits, and the impact of future maps and modes.
And to wait until the game is widely available? That's just bad business; way to make your reviews pointless to most gamers. Like I said earlier, advocating serious criticism isn't the same as eradicating reviews. I think they're (largely) fine as is. They serve their purpose, even if that's not what you want from them.
Most gamers are idiots. I'm not saying that the major sites should hold off on their reviews. Instead, I'm saying that it's also possible to reach an audience by holding off on your review and actually completing the game, then following with a critique. I'm saying that there is an audience for the thing, not that this other thing must be killed or whatever. There will always be stupid people that need traditional reviews, and people need to continue making their decisions as easy as possible. Because of this fact, I'm very happy that the buy/fry/try methodology has become so common.
Nah.Yeah, some. But I don't think wanting a serious discussion on game quality and effectiveness is something that has to replace the current system of game reviews. I don't think most gamers want to read about about how nearly all open-world game's side-missions hamfistedly break the flow of the game's main plot if you decide to do them. Or that they completely disrupt any narrative flow that the game forces you into for any quality content, rather than let you carve your own story in the world given. Most gamers don't care, so mainstream reviews won't go in that direction.
That's okay. Like I said, there are mainstream sites for that kind of stuff. It's okay to have a mainstream review and an alternative critique that actually discusses the design of the game. Possibly with the game's creators. That would be cool. Maybe I'll start a blog and do just that...
Hell, I'd say that mainstream gamers are only just now starting to realize that a game doesn't have to break down barriers visually to be worth buying. The last thing most care about is narrative flow or the moral premise of a game, and most games just aren't made with such things in mind.
Again, this is fine. I have no problem with the uninformed mainstream crowd asking for the simplest of reviews. There's an established market for that bullshit. If you want that, you can get it. In great numbers. If you want critique, too bad, because it's tough to find. And there is obviously no audience that would enjoy such writing.

[Generation ABXY]

On another note about review copies (looks like you were right, Variable, it got stuck in my craw), the product isn’t ever entirely free, as the reviewer does, in fact, have to review it. You may not think that is all that much, but, when you consider it, they have to put a lot of time and thought into writing up that evaluation. So it is rather like washing dishes to get a free meal at a restaurant. If that doesn’t seem tantamount to payment, ask yourself this: do you write up a review for everything you experience, whether it is the meals you eat, clothes you wear, books you read, etc.?

Generation ABXY and I conversed about the provided review copy issue, and we irrevocably proved that it does in fact influence review scores. How much or little is up for debate, however. I don't want the reviewer imagining anything, including what it would be like to pay money to buy the game.

I really can’t remember as it was so late, but did we decide that definitively? I’m not arguing the point, but I would say it can influence it, though I can’t say for sure it always does.

Even your own site is going to be skewed, than, if you truly consider this an absolute. You may not have ads and you may not get free games, but you’ll have to pay for this setup somehow, and – unless you are independently wealthy (in which case, can I get a grant?) – that means working a job. If you’re working a real job, you’re not playing games full time, and you’ll likely hit them in your downtime, a sort of leisurely activity to help you relax. If that’s the case, the pleasure could constitute an exchange in services, just as a masseuse or hooker get payment. Since you put out money for games and we turn in a review for them, you might think that gives independent reviews some sort of objective viewpoint, but it really doesn’t. In those cases, they got to use the games as an escape, whereas we are immersed in them all the time. I’m not saying that makes them worse than professional reviews, but merely on par.

[Variable Gear]

On another note about review copies (looks like you were right, Variable, it got stuck in my craw), the product isn’t ever entirely free, as the reviewer does, in fact, have to review it.
I've heard stories, specifically from Jim Sterling, of reviewers refusing to play their review copies of games. I'm not pretending that this is standard practice, but it does grate with your assertion that the reviewer "does, in fact, have to review it."
I really can’t remember as it was so late, but did we decide that definitively? I’m not arguing the point, but I would say it can influence it, though I can’t say for sure it always does.
Yeah, we decided definitively that it influenced scores.
Even your own site is going to be skewed, than, if you truly consider this an absolute. You may not have ads and you may not get free games, but you’ll have to pay for this setup somehow, and – unless you are independently wealthy (in which case, can I get a grant?) – that means working a job. If you’re working a real job, you’re not playing games full time, and you’ll likely hit them in your downtime, a sort of leisurely activity to help you relax. Since you put out money for games and we turn in a review for them, you might think that gives independent reviews some sort of objective viewpoint, but it really doesn’t. In those cases, they got to use the games as an escape, whereas we are immersed in them all the time. I’m not saying that makes them worse than professional reviews, but merely on par.
The difference is that I'm not trying to get my reviews out right on launch day or any such ridiculousness, The benefit of my approach is the ability to spend weeks with a game before sitting down to review it. So, yes, I will be able to spend as much time as the professionals do being immersed in digital entertainment, if not more. Additionally, I would argue that this practice vaults my review system over the professional review system. We aren't on par.

[Orca]

The best solution is for journalists to take as much time as they need to complete a game. It's unwise to have your review go live when the game comes out, because then you have to pussyfoot around spoilers and the like. It's best, in my opinion, to be less timely and have a little more to say.

Why in the world would you talk about the story specifics at all? You can review movies or books without revealing plot points, and games are no different. I read Empire Strikes Back reviews, but they don't tip me off as to the whole Vader-Luke thing... I read a review of a Harry Potter book without finding out about Dumbledore...

Yeah, we decided definitively that it influenced scores.

Post that definitive proof then, because even the person you say you "definitively" decided it with doesn't agree.

[Variable Gear]

Why in the world would you talk about the story specifics at all? You can review movies or books without revealing plot points, and games are no different. I read Empire Strikes Back reviews, but they don't tip me off as to the whole Vader-Luke thing... I read a review of a Harry Potter book without finding out about Dumbledore...
For your information, I'm not only talking about story specifics. I'm talking about levels and boss battles that people wouldn't want spoiled. Reviewing games is different than reviewing books and movies, where the story takes precedence. Instead, in games, it is the experience that takes precedence. Reviewers can't talk about this kind of stuff, because they don't want to spoil everything, but a critique that went live later could actually tackle just how successful the developer was at creating the experience they intended.
Post that definitive proof then, because even the person you say you "definitively" decided it with doesn't agree.
It's from a chat log. I'm sure you can imagine what that would look like.

[KingGorilla]

Why in the world would you talk about the story specifics at all? You can review movies or books without revealing plot points, and games are no different. I read Empire Strikes Back reviews, but they don't tip me off as to the whole Vader-Luke thing... I read a review of a Harry Potter book without finding out about Dumbledore...


You see, this is what I mean by the audience has to grow up. Roger Ebert can talk about the plot line and story to a film. But geeks have this jihad on "spoilers." Let me clue you in, anything that can be "spoiled" by discussing any particular in the plot, was not that good of a story to begin with.

[Urizen]

On this topic, and perhaps only this topic, KingGorilla is my soulmate.

[Generation ABXY]

I've heard stories, specifically from Jim Sterling, of reviewers refusing to play their review copies of games. I'm not pretending that this is standard practice, but it does grate with your assertion that the reviewer "does, in fact, have to review it."

My mistake; let me rephrase that. Any reviewer who wants to keep his job or has an ounce of integrity does, in fact, have to review it. If I didn’t turn in a review for a game I was sent, I wouldn’t be issued another, and I have morals enough to not fudge a review. :p

Yeah, we decided definitively that it influenced scores.

Okay, I'll trust you on that. I fully admit I have piss for memory. :)

The difference is that I'm not trying to get my reviews out right on launch day or any such ridiculousness, The benefit of my approach is the ability to spend weeks with a game before sitting down to review it. So, yes, I will be able to spend as much time as the professionals do being immersed in digital entertainment, if not more. Additionally, I would argue that this practice vaults my review system over the professional review system. We aren't on par.

If it is any consolation, I rarely receive games early (though Atlus, I know, has done it from time to time), and I do often spend weeks playing a game. Hell, I had an RPG out for more than several months once, although there were some other factors involved there. And my argument wasn't about the time you spend with it, merely that, for you, you relax with it, whereas we live and breath this stuff and can tend to see it as work; hell, with the exception of sandbox titles, I never relax with video games.

I take great pride in my work, though, and I suppose we’ll have to disagree about the specific merits of our respective techniques.

[Variable Gear]

And my argument wasn't about the time you spend with it, merely that, for you, you relax with it, whereas we live and breath this stuff and can tend to see it as work; hell, with the exception of sandbox titles, I never relax with video games.
Well, in this case I'd argue that we are looking at things in a similar way. I live and breathe games. I followed 360 news religiously before I ever owned one. I would watch every new video of the dashboard in order to find out what was added. I also follow PS3 news, even though I've never owned a PS3. I can't wait to hear what the new Rock Band tracks are going to be, what's going to be up on the Virtual Console as well as WiiWare, what's going to be on XBLA on Wednesday, and what's just been released on the PSN. I follow many sources online, and I read as much as I possibly can. I really enjoy Gamasutra's feature articles and pretty much anything that goes live on the Sore Thumbs blog. I find it funny that you tried to divide us using this thought process. We're more similar than you think!

I find it easier to relax in a game than you do...although some parts are really tense. :o
I take great pride in my work, though, and I suppose we’ll have to disagree about the specific merits of our respective techniques.
I'm not saying that specific reviewers aren't able to have success with the current system, however, I am saying that in general the current system provides a greater opportunity for weak ratings instead of inspired critiques. I'm sure that you do a great job, and I have no hatred or disdain for people in your position. It's not an easy road.

[KingGorilla]

On this topic, and perhaps only this topic, KingGorilla is my soulmate.

I feel the love, it is warm and sticky like...EWWWW!

Look, people can have their kinks. But don't ham string an entire aspect of journalism and writing because of some weird OCD. I am so sick of "Rock solid audio, stunning visuals, and a rich plot, make this gaming goodness the bread and butter of our meat and potatoes."

[Orca]

I've heard stories, specifically from Jim Sterling, of reviewers refusing to play their review copies of games. I'm not pretending that this is standard practice, but it does grate with your assertion that the reviewer "does, in fact, have to review it."

Jesus Christ, Jim Sterling is one of your sources of information? Read one of his reviews some time. He's not a reviewer, he's a critic - and a poor one at that.

[Variable Gear]

Also, KingGorilla, journalism is dead and buried. I'm talking about game writers.
Jesus Christ, Jim Sterling is one of your sources of information? Read one of his reviews some time. He's not a reviewer, he's a critic - and a poor one at that.
I like Jim Sterling. I think his reviews are top-notch as well. Long live the podcastle!

[Dark Prince]

Reviews are an iffy thing and something I don't think we'll ever find a happy medium with. Personally I think they should be just critical analysis of games considering the amount of access we have to demos these days. Reviews that I've seen end up just giving us facts and nothing really about the game itself. We have news to deliver us facts about the game. With the way the internet has boomed, we can find just about anything we need too about a game.

We don't really need scores to determine how good or bad a game is since the public can make up their own minds about that. I think their should be game critics that analyze the game experience within itself and the game on basis overall.

[Urizen]

Also, this "quickly enough" stuff needs to go away. If you are buying the game, you'll likely know before the reviews go live. Reviews, in my opinion, need to change into more mature critiques of games instead of a simple Buy/Rent/Pass statement based on the game's technical mastery. I mean, there are still those who depend on reviews or Metacritic to make their decisions for them, but that population is reduced whenever it rains. Also, post-release content from a major site? Don't make me laugh. That shit almost never happens.
(There are sometimes "postmortems," but that's as deep as the game journalism industry gets.)

Your multiplayer site review theory is possible. If a co-op site can exist, so can a multiplayer site.

And I'll say it again, please check out the links in the OP if you haven't already. There are some interesting perspectives contained within.

I'm fairly certain I liked Braid a lot more than you did, but perhaps you'll enjoy this (http://www.nintendojo.com/editorials/view_item.php?1222725730).

Let me know what you think.

[J Arcane]

How to improve reviews?

1) Kill the payola. Kill the fucking cross marketing, the "I'll buy X # of ads for X number of word review", the release date dickering based on review score, fuck, kill free review copies, fuck 'em, most of the less gigantor sites can afford to buy their own bloody games, I'm sure the big fuckers can manage. Basically eliminate the conflict of interest involved when the reviewer is getting something from the reviewee.

2) Get out of bed with the marketing and PR industries. REal fucking newspapers and magazines don't run virtually unedited press releases on their front bloody pages, wso why the hell should we? Treat the shit that comes your way from these sleazebags as exactly what it is: junk mail. Toss it in the fucking bin, play the goddamn game, and then tell us what you think about it. What the marketing department of a publisher has to say about a game means jack goddamn squat. Stop bending over backwards and racing to the presses with ever minor detail, every press release reconfirming shit we already know, every "leaked" screenshot, every word that comes out of the mouth of someone vaguely related to the devteam. All you're doing is feeding the fucking hype machine, and that does jackshit for anyone except sell copies, which is not your fucking job.

3) Get reviewers to grow some fucking testicles. This is where I probably disagree with half the wanna be journoanalysis types like GameSetWatch and such. Yes, some level of journalistic integrity is important, that's where #1 came in, but after that, the responsibility is fucking done. Reviewing is a subjective fucking judgement, period. Appreciation of the medium is an important criteria, and it's important to include some comparitive analysis, but fuck all these nonsense fanboys crying for "balance" and "fairness" in reviews. Say what the fuck you liked, say what the fuck you didn't like, and why, and if a bunch of dipshits bomb your comments page because they don't like it, they can stick it up their arses.

I'm sick to death of all the caveats and weasel words and general beating around the bush, of reviewers bending over backwards to try and avoid just flat out saying a game has some fucking problems. Or even just avoiding really, really obvious comparisons, like how I had to read 10 fucking "professional" reviews, before I finally found some GamerDad guy on Yahoo of all places who'd finally admit that yes, Field Commander was pretty much an Advance Wars clone. This despite the fucking Gamespot review describing the game mechanics in such detail, and with such obviousness, that you could've edited out the words "Field Comander", and most any gamer with a GBA would assume it wa for an Advance Wars game.

4) Get some real bloody writers. Half the reason most game writing and reviewing is shit, is because they've got shit writers doing them. The entire games media is filled to the brim with a lot of rank amatuers who basically only got the jobs they had because they liked games a lot, and would basically do whatever they were told for pretty much free. And then everyone is astounded that they kneel and grovel so readily before the mighty game companies. Most of these guys are just overly vociferous fanboys who lucked into a dream job, and it shows. Badly. THe writing's atrocious, the reviews and press easily manipulated, because all you have to do is show these guys a pretty trailer and they're gobbling it up just like the rest of the slavering dorks that lurk Gametab. There's a reason the internet produced such a boom in games media, it's because a lot of sadly misguided individuals looked at what was already out there, realized it was amateur shit, and figured they ould get away with just as much mediocrity and be quite fine with it. As it turns out, they were right, and now a number of them are millionaires for doing little more than drooling on their keyboards to game company press releases.


Really, I don't think a one of the dedicated games media sites out there could really survive such demands. Which wouldn't necessarily be a bad thing, for anyone but the fanboys who work for them. The only vaguely respectable individuals I've seen coming out in recent years as games journalists have been, surprise surprise, from mainstream media outlets. Unfortunately, most of them are too busy trying to justify their chosen focus and ind up just oming across as ragingly pretentious, as they bend over backwards overanalyzing games to the point of tedium.

[Urizen]

I would say let'em spoil, but Roger Ebert manages to review films all the time without spoiling them. Say what you want about the guy, but he's a hell of a film reviewer.

I'm going to pick a bone here. Ebert is first and foremost a movie critic. He just happens to play the part of reviewer because that's how he manages to get his word across to the greatest number of people.

Yes, he dumbs things down the point where he is faced with recommending a movie to someone based on what's playing any given week. A movie is only as good as what it's being compared with most of the time in his field of work.

As a critic, he aspires to more. It's about preserving, edifying and enriching the culture and canon of watching feature films.

[Urizen]

Really, I don't think a one of the dedicated games media sites out there could really survive such demands. Which wouldn't necessarily be a bad thing, for anyone but the fanboys who work for them. The only vaguely respectable individuals I've seen coming out in recent years as games journalists have been, surprise surprise, from mainstream media outlets. Unfortunately, most of them are too busy trying to justify their chosen focus and ind up just oming across as ragingly pretentious, as they bend over backwards overanalyzing games to the point of tedium.

If charity begins at home, and this site can take some unfettered truth, I'll say a lot of what is said in this thread extends to CoG.

The reviews here are only better because the one or two hundred people who read those reviews have some idea of who the reviewers are. It's not that there's a new review or that it's a CoG review. It's that it's a Psykoboy or fitbabits review. There's nothing about the expressions, the language, the scoring, the diction or anything else that sets these reviews apart anything else on the net.

I'd say more but I'd be regurgitating stuff from the article in the link I posted above.

[Telefrog]

How to improve reviews?

(A bunch of awesome suggestions.)

Really, I don't think a one of the dedicated games media sites out there could really survive such demands. Which wouldn't necessarily be a bad thing, for anyone but the fanboys who work for them. The only vaguely respectable individuals I've seen coming out in recent years as games journalists have been, surprise surprise, from mainstream media outlets. Unfortunately, most of them are too busy trying to justify their chosen focus and ind up just oming across as ragingly pretentious, as they bend over backwards overanalyzing games to the point of tedium.

This man has it exactly right. And guess what? It's not going to change.

Video game journalism (whatever the fuck that means) is a long drawn out handjob for fans. It's a niche audience. It's the game equivelant to Car & Driver or Toy Review or even Cat Fancy. Game journalists live and die by the marketing money and access of the publishers. They can't piss off their only avenue of information so everything falls into the 7-10 scale unless it's outrageously bad or a small developer.

Roger Ebert and his ilk have the luxury of being noteworthy enough to say whatever he wants about movies and the studios have to suck it up or he'll blackball them. How did he get that way? By not getting into bed with the studios in the first place and never compromising.

[Variable Gear]

I'm fairly certain I liked Braid a lot more than you did, but perhaps you'll enjoy this (http://www.nintendojo.com/editorials/view_item.php?1222725730).

Let me know what you think.
That was an excellent editorial, DaX. Nice work on it.

I agree with all of the points you raised in that article. The established industry does not seem capable of discussing games on anything other than a technical level. Games that have a message, or are examples of masterful design, get brushed under the rug in order to clear the way for a useless preview about an upcoming title, which, as you mentioned, is not going to change much from outlet to outlet.

We have a long way to go.

[Urizen]

That was an excellent editorial, DaX. Nice work on it.

I do disagree with those who say nothing will ever change. It's our lot in life to be stuck with the worst writing possible. This thread, for instance, and Elliott's symposium are not errant shots in the dark.

This is an issue weighing on the minds of lots of people, some who have influence. I believe change is inevitable and now is a good time to get involved in the debate.

[Variable Gear]

I'm really late to the party on this, but...part one of the symposium went live in December 2008 (http://shawnelliott.blogspot.com/2008/12/symposium-part-one-review-scores.html).

Part two has also gone live recently (http://shawnelliott.blogspot.com/2009/02/symposium-part-two-review-policy.html). I'll post my thoughts on both parts once I've gotten a chance to pour over them myself.

[Ancalagon]

I think I'd like more transparency in reviews. If Bill the FPS nut reviews The Legend Of Zelda: The One Wherein There is a Plot Twist, I want to know what his stance towards RPGs is. I want to know whether this is the sort of thing he would normally play. If he states his bias at the outset (eg, I hate Zelda, and I hate swords, and I hate fantasy, and I hate elves), then I know how to interpret his review better.

Also, the reviewer should state how much time he spent playing the game, how far he got, and what portions of it he tried. Didnt touch the multiplayer? Mention that. Didnt finish the game? Mention that. got 50 hours into it, nearly died cos you forgot to eat and still didnt finish it? Mention that.

[Variable Gear]

I think I'd like more transparency in reviews. If Bill the FPS nut reviews The Legend Of Zelda: The One Wherein There is a Plot Twist, I want to know what his stance towards RPGs is. I want to know whether this is the sort of thing he would normally play. If he states his bias at the outset (eg, I hate Zelda, and I hate swords, and I hate fantasy, and I hate elves), then I know how to interpret his review better.
Your post would make more sense if Zelda was a RPG. ;)

[Ancalagon]

Your post would make more sense if Zelda was a RPG. ;)

Never played it! But hey, it was just an example. If.... NeCroG0THiRz3r reviews, say, Viva Pinata, I'd like him to state his bias so I know why he says it needs more shades of brown and black (and theres gonna be trouble if he compares it to Diablo 3)

[Variable Gear]

Never played it! But hey, it was just an example. If.... NeCroG0THiRz3r reviews, say, Viva Pinata, I'd like him to state his bias so I know why he says it needs more shades of brown and black (and theres gonna be trouble if he compares it to Diablo 3)
I understand your overall point. I agree that outstanding biases should be stated upfront.

However, most reviewers rate each game on its own merit, reducing the value of such a practice.

Still, it's nice to know their overall genre preferences.

[Telefrog]

Never played it! But hey, it was just an example. If.... NeCroG0THiRz3r reviews, say, Viva Pinata, I'd like him to state his bias so I know why he says it needs more shades of brown and black (and theres gonna be trouble if he compares it to Diablo 3)

I find this line of thinking odd. Movie, book, and art reviewers don't state upfront if they have a preference for a certain genre. Car reviewers don't have to specify that they like sedans more than coupes.

I get your point - that many game reviewers let their bias influence the outcome of their reviews, but that seems more an issue with the juvenile nature of most of the writers. Frankly, they just aren't professional enough to see beyond their own prejudices.

[J Arcane]

Actually, I've been finding its quite common for gun reviewers of all things to state any personal biases or experiences in a gun review.

And probably my favorite car reviewers of all time, the Top Gear crew, wear their biases on their sleeves.

[Telefrog]

And probably my favorite car reviewers of all time, the Top Gear crew, wear their biases on their sleeves.

Jeremy and co do admit to bias, but I haven't seen that it has ever interefered with giving a fair review. Besides, the show has a particular focus which does not encourage econoboxes or family wagons anyway. I don't think anyone would expect them to go nutty over a four-banger compact unless there was something really special going on under the hood.

[fitbabits]

Game Reviews: How Can the Practice be Improved?

By scrapping them altogether.

[Urizen]

Game Reviews: How Can the Practice be Improved?

By scrapping them altogether.


Correct me if I'm wrong. Don't you write reviews and publish them online for an internet community?

[quidmonkey]

Game Reviews: How Can the Practice be Improved?

By scrapping them altogether.


Reviewers should stop writing reviews and merely assign a number.

[fitbabits]

Correct me if I'm wrong. Don't you write reviews and publish them online for an internet community?
We do, yes.

Reviewers should stop writing reviews and merely assign a number.
No, not a number. A letter. A through F. Seems fairer to me.

[Ancalagon]

Actually, what would improve reviews immensely is if publishers stopped "advising" reviewers on how to review their games.

The 1% of the time where its noticeable or leads to an artificially inflated score erodes confidence in 99% of reviews. Publishers should themselves spend more time making better games and less time massaging review scores.

EDIT: That figure could be considerably higher than 1% I realize, I'm just trying not to be alarmist.

[quidmonkey]

No, not a number. A letter. A through F.

You mean an ASCII character. A $. ╟ through ▐.

[fitbabits]

You mean an ASCII character. A $. ╟ through ▐.
I would prefer A-F, like Entertainment Weekly. Or school grades, etc.

[quidmonkey]

I would prefer A-F, like Entertainment Weekly. Or school grades, etc.

As in hexadecimal? L4D = 0xAF. I r G4ym3 R3v13w3r.

[Telefrog]

I would prefer A-F, like Entertainment Weekly. Or school grades, etc.

Unfortunately, even that would get broken down into a numerical value by Metacritic and the like.

Speaking of which, here's a great Edge article (http://www.edge-online.com/features/is-metacritic-damaging-games-industry) about Metacritic:


Metacritic is still edited by just one man, ‬Marc Doyle. ‬But his focus remains very much on the reason why it was established in the first place. “‬I really see myself as a kind of gatekeeper to tell people that these are the games you should be paying attention to,” ‬he declares. ‬His role is to gather scores and comments for every game released in the ‬US, ‬choosing which publications are included and concocting the formula that ‬combines them into a single number.

[Wraitheist]

I think there are a lot of good review sites out there. A lot of them have different styles, etc., but it is easy to get a good idea of what a game is like by checking out multiple reviews from different sources.

[quidmonkey]

Unfortunately, even that would get broken down into a numerical value by Metacritic and the like.

Hence, no words, just a number.

I think there are a lot of good review sites out there. A lot of them have different styles, etc., but it is easy to get a good idea of what a game is like by checking out multiple reviews from different sources.

Hence, Metacritic: statistics as mean, median and mode.

[Johan]

Demos .

[Shrinn]

I don't like numbers. I value discussion on why a game was bad much more than just seeing an F.

[crazyD]

Reviewers should stop writing reviews and merely assign a number.

That is the exact opposite of what I'd want. I would rather read a description of what a reviewer thinks is good or bad about a game than a generic number with no backing. I think number values to qualify a game is complete bullshit, and the text should be enough. If need be, maybe something indicating whether or not they would recommend it to fans of the genre or gamers overall.

[J Arcane]

Jeremy and co do admit to bias, but I haven't seen that it has ever interefered with giving a fair review. Besides, the show has a particular focus which does not encourage econoboxes or family wagons anyway. I don't think anyone would expect them to go nutty over a four-banger compact unless there was something really special going on under the hood.
That's because a truly great reviewer knows his own biases, and knows how they affect his judgement, and how to set them aside when they need to be for the sake of looking at it from another angle.

One of the other things I like about the show are the competition reviews, where they take three cars out, and swap them between them to get a more even spread of opinion, something the bigger magazines/media really should do.

[jpublic]

I have worked as a reviewer for a magazine before - OGR (which is now long defunct). Let me tell you, the publisher provided review copy serves to put a lot of pressure on the reviewer - especially if the game is bad.

If you're honest, you run into the possibility of getting your magazine blacklisted. If you're overly generous, you're giving a false review. If - like happened when we were to review Battlecruiser - we refuse to review it until they patch it into something playable, we end up being very late.

That balancing act - being a good reviewer on my part vs pressure by the bean counters to not alienate an advertiser - was what made me quit. I just couldn't take the pressure to be dishonest.

[KamaItachi]

There is a phrase, which I'm paraphrasing, that people don't get the governemnt they need, but the government they deserve. I think the exact same could be said about game journalism.

What we, as gamers can do to improve the review process is stop enabling the dick waving and hystrionics, quit hating on those who have an opposing view on our favourite game, prefer a different machine, accept that other people besides the 'hardcore' might enjoy the wii and encouraging mature journalism as a whole.

[Tron]

With demos being so readily available I don't care too much for reviews any more. The opinion of a stranger rarely lines up with my own.

[Tron]

One of the other things I like about the show are the competition reviews, where they take three cars out, and swap them between them to get a more even spread of opinion, something the bigger magazines/media really should do.

Cars aren't subjective though...it's very easy to quantify their performance. 0-60 times, skid pad rating, 1/4 mile, track lap times, breaking distance, etc.

[Wilkz07]

One Game = 1 Review. IGN often has 3 different reviews for their games depending on location of release. They all ready the same though sometimes the UK review will score higher than the US review. I hate that... i guess i could stop looking at IGN.

Here's what IGN has multi-reviews for:
Street Fighter IV [AU] Fighting 9.0
Street Fighter IV [UK] Fighting 9.5
Street Fighter IV [US] Fighting 9.3

F.E.A.R. 2: Project Origin [US] Shooter 8.1
F.E.A.R. 2: Project Origin [AU] Shooter 8.6
F.E.A.R. 2: Project Origin [UK] Shooter 8.5

[mister slim]

We do, yes.


No, not a number. A letter. A through F. Seems fairer to me.

So, given that "fitabits" starts with the letter 'F', bottom of the scale would be 'A' right?

[Variable Gear]

There is a phrase, which I'm paraphrasing, that people don't get the governemnt they need, but the government they deserve. I think the exact same could be said about game journalism.

What we, as gamers can do to improve the review process is stop enabling the dick waving and hystrionics, quit hating on those who have an opposing view on our favourite game, prefer a different machine, accept that other people besides the 'hardcore' might enjoy the wii and encouraging mature journalism as a whole.
I agree with this. The majority of those who are involved in the digestion of game reviews/journalism are moronic, fanboyish people. Oh my god, Killzone 2 didn't get a 10/10 from Edge, let's riot! (Even if we agree with the reviewer that there are numerous flaws with the game.) People like this deserve Gerstman-gate, deserve 100-point IGN review scores, deserve shit. There is some minuscule percentage out there that wants a more mature discussion of gaming, and that population is growing, but there are still not enough of them to change the tone of the discussion.