Fanboys Wrecking Metacritic

[DoctorFinger]

We all know how heated the arguments over the console wars can get. Passion which used to be limited to discussions of religion and politics are now being directed into determining which console is The One.

The newest front in the console fanboy war? Review aggregator Metacritic (http://www.metacritic.com/), which in addition to compiling reviews of games, DVDs, movies and music into an aggregate score, allows users to review those products as well. But as 1UP points out (http://www.1up.com/do/newsStory?cId=3171112) when the game in question is a high profile console exclusive, the fanboys see those user reviews as a way to denigrate the opposing camp.

Gears of War 2 (http://www.metacritic.com/games/platforms/xbox360/gearsofwar2), which will not be released for several more days, has a meta score of 94 (based on 27 reviews) and a user score of 35 (based on 3700 reviews). Similar pre-release vandalism is befalling Resistance 2 (http://www.metacritic.com/games/platforms/ps3/resistance2?q=resistance%202), which has a meta score of 89 but a user score of 53 (based on 1200 reviews) and LittleBigPlanet (94/60).

This crap is, in short, pure second grade buffoonery. Yes, I'm sure some of those scores are from people who recieved early copies and are genuiely unhappy with the title, but the vast majority are simply platform adherents who somehow see good games on another platform as a direct attack on them. Notice how this zeal doesn't manifest itself as inflating the scores of their own platform's games, but by deflating the rating of the opposition. This doesn't happen in any other medium. Fans of Warner Bros movies don't go around knocking Universal's flicks. Sci-Fi fans don't knock down the scores of procedural crime shows. Country fans don't go out of their way to pull down the score of rap albums. Why do some small minority of gamers feel the need to turn their hobby into the Hundred Years War?

Source - 1UP (http://www.1up.com/do/newsStory?cId=3171112).

[GunnyMo]

The Law of Internet Fuckwadery is proven yet again. As long as the meta score is unchanged I wouldn't sweat it too much. Smart people know stupidity when they see it in those user "reviews".

[Mr. Murphy]

Many gamers do not have the funds to purchase more than one console. Some people feel the need to denigrate what they don't have, in order to make themselves feel better about not having it.

Add to that the search for identity that occurs in your teenage years, and you get young people putting false importance on brand loyalty.

Also: 100th post.

[Gorvi]

Yeah, this started with LBP, at least that's where I first saw a story on it. Completely pathetic.

[frederec]

This crap is, in short, pure second grade buffoonery. Yes, I'm sure some of those scores are from people who recieved early copies and are genuiely unhappy with the title, but the vast majority are simply platform adherents who somehow see good games on another platform as a direct attack on them. Notice how this zeal doesn't manifest itself as inflating the scores of their own platform's games, but by deflating the rating of the opposition. This doesn't happen in any other medium. Fans of Warner Bros movies don't go around knocking Universal's flicks. Sci-Fi fans don't knock down the scores of procedural crime shows. Country fans don't go out of their way to pull down the score of rap albums. Why do some small minority of gamers feel the need to turn their hobby into the Hundred Years War?


I think at least part of the reason this happens is because the different factions of gamers tend to go to the same places for their information. Rap fans and country fans will talk shit about each other, but don't frequently hang around and have conversations about it.

Whereas with gaming everyone's thrown together, mainly because the people that aren't fanboys are genuinely interested in all of these things. It's just the blind adherants that want to stir shit up.

Also, I didn't read the article (will have to do that later), so I didn't see. Do the reader reviews pollute the official reviews? If not, I would see it as an annoyance, but not a crisis. Maybe something to be worked out like a spam filter, but not a big deal. Geeks tend to overreact and overdefend things. That's just the way we are.

[Gorvi]

Also, I didn't read the article (will have to do that later), so I didn't see. Do the reader reviews pollute the official reviews? If not, I would see it as an annoyance, but not a crisis. Maybe something to be worked out like a spam filter, but not a big deal. Geeks tend to overreact and overdefend things. That's just the way we are.
They're completely seperate. It's just one of those stories that kinda makes you shake your head and wonder a bit about people.

[Purple Santa]

They're completely seperate. It's just one of those stories that kinda makes you shake your head and wonder a bit about people.

I certainly agree. It's all great and all that you "love" whatever console you have...but it's like all other beliefs...keep it to yourself and others of similar taste. Don't push your views on others and ruin it for everyone else. Stuff we learn in kindergarten...don't be rude to others.

[Mdot]

Truly pathetic. The fact that some people have enough time on their hands to go and do shit like this is lame.

[Deunnero]

I use my fellow CoG'rs as my basis for game "scores".

[roboninja]

Fanboys can ruin anything. Not PC fanboys, of course. We're cool ;)

[Purple Santa]

Who really looks at user reviews anyway? I think it's shitty and all people are asses...but other user reviews I care about are at least from "users" I know in some way. Like users on this forum, who i get to know from reading enough threads.

[KingGorilla]

I cannot thimk of any people in the industry or the media with anything but loathing for Metacritic. Valve may be closest, scores prominently featured on Steam after all. But people are paid, hired, fired based on this bullshit.

[Haemorrhage]

Metacritic created the problem by allowing for user "voting" prior to the games release. A user review score should at least require and actual "written review" with a minimal word count by registered user only.

Ebgames.com had a similar user reviews which was polluted by fanboys. But they eventually wised up and changed how reviews were submitted.

[Food Nipple]

Metacritic created the problem by allowing for user "voting" prior to the games release. A user review score should at least require and actual "written review" with a minimal word count by registered user only.

Ebgames.com had a similar user reviews which was polluted by fanboys. But they eventually wised up and changed how reviews were submitted.

I have to agree, the system is broken by design. If you're going to aggregate user submitted reviews, set some standards to avoid abuse.

[DoctorFinger]

The games editor of the site told 1UP that they're going to be revising their registration and pre-release review policies, but didn't specify how.

[itchyeyes]

Frankly, I never bother with the fan reviews on Metacritic. What bothers me more is that the site does not adjust for differing rating scales. For instance an "average" game would receive a 7 from IGN and Gamespot, a 5 from Edge, and a C from 1UP, which they for some unknown reason equate to a 5. Yet metacritic averages them all on the same scale with no weighting. So reviews from outlets like 1UP and Edge almost always skew the average downwards, even when the actual reviews may even be more favorable than the average score.

[KingGorilla]

It is not their fault that IGN and CNET editorial staffers fell asleep in 4th grade arithmitec.

[itchyeyes]

It is not their fault that IGN and CNET editorial staffers fell asleep in 4th grade arithmitec.
It's hardly just IGN and Gamespot that follow the 7=average scale. And some sites use a system with only 5 distinct ratings, in which an average review (ie 3 stars) equates to 60%. Besides, it's not any of the reviewers' responsibility to makes sure that their scale aligns with everyone else's. They're not reviewing these games for Metacritic's use, they're reviewing them for their readers who are all well aware of what the scales mean.

We're getting off topic though, it doesn't matter what you normalize the scales to. Metacritic can weight everyones' average down to 5, or 2 for that matter if they want. I don't care, so long as they normalize them to something. As it is, it's just meaningless and completely breaks their system.

[DoctorFinger]

On some level though, shouldn't sites like IGN and CNET who call their 7 'average' codify that scale somehow?

[itchyeyes]

On some level though, shouldn't sites like IGN and CNET who call their 7 'average' codify that scale somehow?
I can't link the pages because they're blocked at work, but I'm pretty sure just about every major review magazine and site has some section where they explain their scale.

[KingGorilla]

The topic is boring Doc, Some of us condone Review bombs for shit like DRM, but just because you want to be a Bill Gates/Ken K pole smoker is something not meriting discussion. It would be an insult to civil discourse even to play devil's advocate.

It is just anyother symptom in the disease that are reviews aggregators.

[bean]

I never look at user reviews unless they have accompanying text. It's very easy to see within one paragraph whether or not someone is a douche.

[quidmonkey]

The real question is who doesn't use Metacritic? If not for games, then movies, books and music? Everyone I know uses Metacritic at some point in collating info on a new product.

[Johan]

I don't know why anyone would bother with reviews of any sort without some kind of context to the source and some personal knowledge of the veracity of the reviewer. Find a reviewer whose likes/dislikes seem to track with your own and follow that person's reviews, OR find a forum where you can get to know some people a bit and you'll know whose opinion to reject out of hand and whose to accept.

Random, anonymous reviews and numbers are quite useless and prone to sabotage; sometimes deservedly so. But who cares? Don't rely upon them.

[Wraith]

My first thought was, "wait, Metacritic has a user score? I didn't even notice that." Does anyone else, apart from those actually writing the reviews, even look at Metacritic user scores?

As for the score disparity, yeah. It's like looking on Amazon for reviews about a product that has yet to come out with a high degree of fan loyalty for competing products. Think Zune and iPod. (Though Amazon might've locked down reviewing on unreleased products, but the same goes for newly-released products.) Or like reading the comments on automotive blogs (Autoblog, Jalopnik). Think GM, Ford, Toyota, etc. There's going to be a crapload of stupid comments from people who've never even used the alternative, and have no idea that it "sucks," apart from their general hatred of the brand, much of which is probably undeserved. The average prospective buyer can't depend on accuracy of user reviews/ratings/comments in cases like this. Fanboys are going to be fanboys, and site admins can only do so much to filter out what's general fanboy hatespew from genuine disapproving reviews. (Or blind fanboy cheerleading from genuine positive reviews.)

[quidmonkey]

My first thought was, "wait, Metacritic has a user score? I didn't even notice that." Does anyone else, apart from those actually writing the reviews, even look at Metacritic user scores?

All the time, especially for movie reviews, which are biased towards certain genres critically. The greater the divide between the critics and users scores, the more my interest piques.

[Goronmon]

The forums over there hurt my brain.

@vbm247,mate your quite clearly stupid or un able to read properly,im a gears of war fan boy hence why im making such a point of getting this info out,and would you like to wiki the phrase trolling please and get back to me

read the other thread mate think i got the sony fan boy shite sorted then, i have owned a ps3 but sold it cos at the time all gmaes were shit,best game they got is tekken,untill it comes to 360 LOL,soy are fucked mate i fucking hate em all the big sony titles are coming out on 360 and pc now

:eek:

[Wraith]

The forums over there hurt my brain.

:eek:Wow.

I don't know why, but I'm going to try clean that up for him.Vbm247, I look down on your intelligence and I question your claims of literacy. I'm making statements such as those in my previous replies because I'm a Gears Of War fanboy. I propose that you are a troll and suggest that you might seek out the definition of the term.

In the other thread, I believe that I, a Gears of War fanboy, successfully disproved the claims of the Sony fanboys. I bought a PS3 before there were any games I deemed worthy of my interest, apart from Tekken, so I sold it. I think Sony has some serious problems, notably due to formerly exclusive titles now coming out on the 360 and PC. These are some of the reasons that I strongly dislike Sony and its video game console, the Playstation 3.Better, but he still sounds like a douche.

[Telefrog]

I have to honestly say that I didn't even know about Metacritic's User reviews until this story first broke a couple of weeks ago. As Wraith says, what's the point when any douchebag can assign a random number to a product? I use Metacritic to look at the scores from the sites I like. If I want idiot scores, I'll just go to Gamefaqs and read the insanity.

[Gorvi]

The forums over there hurt my brain.



:eek:
Yeah, that's why I like it here.

So far as user reviews, I've actually found that GameFAQs has some of the best, believe it or not. Obviously you can just ignore the reviews that are clearly poorly written, but I've found some pretty good reviews on there. Of course, it may be that's just the games I'm choosing to look for that have those.

[quidmonkey]

So far as user reviews, I've actually found that GameFAQs has some of the best, believe it or not. Obviously you can just ignore the reviews that are clearly poorly written, but I've found some pretty good reviews on there. Of course, it may be that's just the games I'm choosing to look for that have those.

It's because GameFAQers (Did I just say that?) tend to finish the games they review, know every secret, cheat and glitch, and are well aware of the community's view of the game. So in terms of lore, yes, your average GameFAQer will review better.

[Raen]

The games editor of the site told 1UP that they're going to be revising their registration and pre-release review policies, but didn't specify how.

I remember reading a comment to this end on the LBP page on Metacritic. It was getting some incredibly low user scores before it reached the original street date. I've found that simply looking at the range of scores on Metacritic, and accompanying information, and ignoring the user scores works pretty well for most things.

I never look at user reviews unless they have accompanying text. It's very easy to see within one paragraph whether or not someone is a douche.

This is a very good point. Just reading a score is pretty much useless. Always read at least the quote that's shown on Metacritic.

Better, but he still sounds like a douche.

I think that's cause he sounds slightly upper class now. You've gone too far the other way.

[Variable Gear]

Professional reviews have already ruined Metacritic. The fanboys don't have much to wreck.
It's because GameFAQers (Did I just say that?) tend to finish the games they review, know every secret, cheat and glitch, and are well aware of the community's view of the game. So in terms of lore, yes, your average GameFAQer will review better.
Obviously, it takes one to know one.

[Mr. Murphy]

I never use metacritic, I go off of friends and CoGers, so that I have some sort of context. Numbers don't mean anything to me, I want to know what you liked and why, and what games you've liked in the past.

[Orca]

If it started with LBP, it's at least plausible it was done by people upset with the condition online was in.

[Gorvi]

If it started with LBP, it's at least plausible it was done by people upset with the condition online was in.
This was well before that. I think it was either before the beta started, or soon after.

[Food Nipple]

It's because GameFAQers (Did I just say that?) tend to finish the games they review, know every secret, cheat and glitch, and are well aware of the community's view of the game. So in terms of lore, yes, your average GameFAQer will review better.

GameFAQs user reviews are good, but you certainly wouldn't know it if you took a look at their forums, which are an absolute travesty.

As for your "Who doesn't use Metacritic?" question: I don't. I usually don't read any reviews, but will sometimes pick up a game I haven't heard of if it gets a strong recommendation from 1up, or see a movie if it got a glowing review from Roger Ebert. I can probably count the number of times I've been to metacrtic on one hand, and if you ignore the visits that were a direct result of this news story, you can halve that number.

[frederec]

Wow.

I don't know why, but I'm going to try clean that up for him.

I know why, because that was awesome. Now I'm going to wish I had a shitty English to snobby English translator handy any time I read forum posts.

[quidmonkey]

As for your "Who doesn't use Metacritic?" question: I don't. I usually don't read any reviews, but will sometimes pick up a game I haven't heard of if it gets a strong recommendation from 1up, or see a movie if it got a glowing review from Roger Ebert. I can probably count the number of times I've been to metacrtic on one hand, and if you ignore the visits that were a direct result of this news story, you can halve that number.

You don't read reviews...except for Ebert reviews.

Does anyone read reviews anymore? All hail 2008 - the year of the podcast.

Metacritic is useful.

[Variable Gear]

Does anyone read reviews anymore? All hail 2008 - the year of the podcast.
I agree with you. Podcasts are where it's at!

[rvieira]

Can we do great reviews for Duke nukem forever making it the best game ever, even before release? / Sarcasm
If the site accepts user reviews before the release date the user review system was already broken from the beginning.

I have fewer friends that are gamers, and only three that are hardcore, so I used to read a lot of reviews before buying a game, but nowadays I think that user feedback from high level gaming (Cog and PiRi) forum and game videos are much more useful.

[roboninja]

I agree with you. Podcasts are where it's at!

Never listened to one on any topic.

I like to actually research things, use lots of disparate sources, etc. I have glanced at Metacritic before, as well as gamerankings.com, but I also look other places, like here and at PIRI.

[LongStepMantis]

The entire mentality behind things like this is absurd. Also further proof that many gamers have nothing better to do than talk shit about a product they don't have, use, or have any stake in whatsoever.

All under the delusion that if they can somehow make the other guy look bad, it improves their own status. All they are really doing is making themselves look foolish and immature. Good job.

Voting on a game that they couldn't have even played yet...do they really think people will see the bullshit as anything other than that? Probably, because they're idiots. Why not simply block reviews until the game has at least been released?

[Variable Gear]

Never listened to one on any topic.
Why not? They are a lot of fun.

[Food Nipple]

You don't read reviews...except for Ebert reviews.

Does anyone read reviews anymore? All hail 2008 - the year of the podcast.

Metacritic is useful.

While Ebert may not be popular with gamers thanks to a couple uninformed comments, I'd say he's easily the best film critic I've ever read. I don't think I need to argue with you that you shouldn't discount a source based on their weakest aspects; after all you did just praise GameFAQs.

I don't see how you can simultaneously hyperbolize the irrelevance of reviews and the indispensability of these same reviews in aggregate. A pile of shit does not gain value by increasing its size. And as far as statistical aggregation goes, the consensus is that metacritic does a piss poor job, especially compared to rottentomatoes.

Podcasts are nice since I can't read while driving, and the format helps critics who aren't particularly good writers (http://www.giantbomb.com) or writers who can't abide by a page limit (http://www.ign.com), but it's no replacement for a concise, intelligent critique by a good writer (http://www.1up.com/do/reviewPage?cId=3163578).

[Variable Gear]

Podcasts are nice since I can't read while driving, and the format helps critics who aren't particularly good writers (http://www.giantbomb.com) or writers who can't abide by a page limit (http://www.ign.com), but it's no replacement for a concise, intelligent critique by a good writer (http://www.1up.com/do/reviewPage?cId=3163578).
Food, how dare you remind me that Shawn is no longer with 1UP. I miss his antics, and his reviews, so much...

[quidmonkey]

While Ebert may not be popular with gamers thanks to a couple uninformed comments, I'd say he's easily the best film critic I've ever read. I don't think I need to argue with you that you shouldn't discount a source based on their weakest aspects; after all you did just praise GameFAQs.

I couldn't interpret how you "usually don't read reviews" but will check out a movie if Ebert gives it a "glowing review". How do you know Ebert's "glowing" opinion if you usually don't read his review?

I don't see how you can simultaneously hyperbolize the irrelevance of reviews and the indispensability of these same reviews in aggregate. A pile of shit does not gain value by increasing its size. And as far as statistical aggregation goes, the consensus is that metacritic does a piss poor job, especially compared to rottentomatoes.

A single score is a personal opinion; an aggregate score is a community's opinion. Your ability to interpret both is determined by how long you've been reading either. In your case, you have a good handle on 1up personalities and how their tastes interact with yours; you can buy a game or write it off based on your knowledge of those personalities: the same goes for me with Metacritic. Both have their pros and cons.

As for RT, knowing if a community approves or disapproves is useful, but vague. Metacritic is attempting more and failing harder for it.

Podcasts are nice since I can't read while driving, and the format helps critics who aren't particularly good writers (http://www.giantbomb.com) or writers who can't abide by a page limit (http://www.ign.com), but it's no replacement for a concise, intelligent critique by a good writer (http://www.1up.com/do/reviewPage?cId=3163578).

Reading is boring and takes too long; and yes, quality is quality - neither of us should be stating the obvious.

[Disgustipated]

Does anyone actually read the user reviews on sites like those?

[quidmonkey]

Does anyone actually read the user reviews on sites like those?

Probably about as many as read user posts on sites like these.

[Food Nipple]

I couldn't interpret how you "usually don't read reviews" but will check out a movie if Ebert gives it a "glowing review". How do you know Ebert's "glowing" opinion if you usually don't read his review?

You're right, I didn't phrase that well. I may not read many reviews, but when I do read them it's usually to look for recommendations for things I might have overlooked. A positive review may make me look at a game I would have otherwise overlooked, but a negative one won't dissuade me from trying something out. If it wasn't for 1up's coverage of Kororinpa, I probably wouldn't have known it existed, but their 4/10 for Assassin's Creed didn't stop me from playing it. Similarly, I took notice when Ebert gave four stars to Mulholland Drive and Amelie, but his unfavorable review of Fight Club wouldn't have stopped me from seeing it.

[quidmonkey]

You're right, I didn't phrase that well. I may not read many reviews, but when I do read them it's usually to look for recommendations for things I might have overlooked. A positive review may make me look at a game I would have otherwise overlooked, but a negative one won't dissuade me from trying something out. If it wasn't for 1up's coverage of Kororinpa, I probably wouldn't have known it existed, but their 4/10 for Assassin's Creed didn't stop me from playing it. Similarly, I took notice when Ebert gave four stars to Mulholland Drive and Amelie, but his unfavorable review of Fight Club wouldn't have stopped me from seeing it.

Mercury Meltdown? Twin Peaks? The City of Lost Children?

I wonder how many games have gotten their shtick from that last one.

[Orca]

This was well before that. I think it was either before the beta started, or soon after.

I see that, after going to look at the site.

Still, I don't see how the people logging the game as a 0 are "wrecking" the site when there seems to be an equally delusional horde scoring any game they like a 10 without reservation.

Either way, it's not a reasonable score and is "wrecking" the integrity of the average.

[Food Nipple]

Mercury Meltdown? Twin Peaks? The City of Lost Children?

I wonder how many games have gotten their shtick from that last one.

List Wars!

I was in grade school when Twin Peaks was cancelled, I had no idea who David Lynch was.

[KamaItachi]

It's hardly just IGN and Gamespot that follow the 7=average scale. And some sites use a system with only 5 distinct ratings, in which an average review (ie 3 stars) equates to 60%. Besides, it's not any of the reviewers' responsibility to makes sure that their scale aligns with everyone else's. They're not reviewing these games for Metacritic's use, they're reviewing them for their readers who are all well aware of what the scales mean.

We're getting off topic though, it doesn't matter what you normalize the scales to. Metacritic can weight everyones' average down to 5, or 2 for that matter if they want. I don't care, so long as they normalize them to something. As it is, it's just meaningless and completely breaks their system.

If all you're doing is using Metacritic to look at the aggregate score, you're using it wrong. People are probably sick of me saying this, but abstract scores should mean very, very little without accompanying text. Metacritic is good as a means to have easy access to half a dozen to a dozen reviews, but really should not be trusted to accurately reflect popular opinion with the average score.

[01010s]

Maybe a solution to this would be to not allow user posted reviews until a week or two after the games release. Admittedly not an ideal solution but it would stop quite a few of these idiots in their tracks I would hope.

[KingGorilla]

I wonder if we can get Anonymous to take down MetaCritic?

[MagGnome]

The real question is who doesn't use Metacritic? If not for games, then movies, books and music? Everyone I know uses Metacritic at some point in collating info on a new product.

Me, for one.

I can honestly say that I never visit Metacritic.

I'd like to see the site have less of an influence. This frenzy over review scores is absolutely ridiculous. They are nearly meaningless.

[Variable Gear]

I was in grade school when Twin Peaks was cancelled, I had no idea who David Lynch was.
You have to remember that Baron is an old man... :p

[mister slim]

Frankly, I never bother with the fan reviews on Metacritic. What bothers me more is that the site does not adjust for differing rating scales. For instance an "average" game would receive a 7 from IGN and Gamespot, a 5 from Edge, and a C from 1UP, which they for some unknown reason equate to a 5. Yet metacritic averages them all on the same scale with no weighting. So reviews from outlets like 1UP and Edge almost always skew the average downwards, even when the actual reviews may even be more favorable than the average score.

*Please pretend this space contains Chapters 45, 46, and 55 of my extended rant about the inability of the Metacritic/GameRankings idiots to understand enough basic fucking statistics to normalize a score against the source's other reviews. Also includes Appendix H, on the idiocy of converting letter grades to a hundred point scale in a way that breaks their own metrics. I'm not drunk enough to do it justice.*