Sadly every element of the system is flawed I wrote 1400 words on the issue yesterday (However, the spam filter keeps removing the post as I put it on my blog). Basically people took the idea way to seriously way to fast and did not account for the logistics of implementing such a system. Therefore I will post it all here: The Curious Case of the PCMR Rating System. A critical evaluation of the 'PCMR' Rating System. After my initial 5 mins musings of problems with the PCMR system gained a relatively high percentage of upvotes I thought I would tackle the issue properly and critique every weakness of the system and ways to overcome them. I also decided to publish this on my Blog as it probably will drag on for quite a while.
Disclaimer: Although, I do not think this will influence by judgement I believe being transparent is essential when engaging in criticality. Therefore, wish to disclose I did ask to assist in the creation of 'PCMRRatings' and got no response. I also wish to future disclose my academic background is in fields outside of video games journalism therefore I will critique this as a data collection method.
The Need for 'PCMR Ratings'
The last couple of years has highlighted significant problems with the standards of games being released on the PC Platform.; with the steam platform especially been filled with unfinished, buggy and unoptimised games, both from independent developers and triple A publishers. For example, a plethora of Early access games are mocked by gamers for their sheer incompetence and Batman Arkham Knight recently had to be removed from sale due to its buggy state. As a result, a good platform to critique the quality of video games is essential. This role was traditional filled by video games journalists who published their thoughts on the quality of a video game. However, the 'GamerGate' Scandal combined with these websites hyping video games, been out of touch with the PC community and often focusing on the gameplay elements (over the inclusion of specific features) has led to individuals going to over media outlets such as youtubers. You tubers such as 'Totalbiscuit', "Angry Joe" and "Jim Sterling" fill an important role in teaching potential consumers about the quality of PC games while often been more in touch with the PC community, identifying the importance of 60 FPS and Field of View Sliders. However, these individual are often limited to a small section of machines therefore cannot assess performance across the board. PCMR Ratings aims to be a website where PC games can share their experience of the performance of video games on PC to gain a more accurate account of the port quality of a video game. However, despite this void in the market I believe the implementation of this system is severely flawed and will not work in its current state.
The Problems with the PCMR System (I Decided to review this implementation as it seemed the most complete/up to date:https://imgur.com/a/W407n#2)
The flaws of the implementation of this system are sadly far ranging. The first issue with this much of the system are extremely subjective in its nature as the creator does not disclosed what makes up each category. This leads to a significant variation in results that each 'reviewer' gives (just look at the original thread and the ratings people would give a game) therefore this metric is far from consistent. This combined with the PC platform having vendettas against certain games (Such as Assassins Creed) will lead to many games been unfairly penalised and overs been hyped. As a result this system needs to be made more objective. However, this is no easy task as the PC platform is not one size fits all is it alright to say that all games need controller support when their are RTS's out there? Is it right to compare the options menu of a 2D and 3D game? Is it right to compare the glitches in a linear 4hr experience to a 200hr RPG? and how can you assess the server quality of a offline Game? As a result this highlights the need for a different system created per GENRE even though this may mean that video games are only be accurately compared to other games in their category it will help to ensure consistent rankings across said categories. However, even if a different system existed for each genre the categories themselves are very bias. First off what classes as good optimisation (running on a intergraded graphics a 750ti? a 980?). This is a very difficult metric as not only will different people interpret it differently but performance is every changing is it right to suddenly raise the rankings at the end of the year? Should it be linked to the apparently graphic fidelity (if so this is still very subjective and the Genre system will need to be implemented). This is out of all the issues the most difficult to tackle. My initial thoughts would be hard descriptions for each level (Freq frame drops, often crashing, runs perfectly) but as this may be influenced by people running a game which is to demanding for their specs. As a result the min/rec requirements will need to be factored it. IE how does it perform at min specs - Max spec but this may lead to the Min specs been raised or having ridiculous requirements (for example BAK). Therefore a second metric may need to be introduced to identify how much of a percent of users will likely have hardware which can run it? However, this may still be incredibly subjective and clearly in the infancy of its design so the comity as a whole will need to debate this issue. The other issues which I cannot easily find an answer to is server quality. What defines a good connection Low ping? Percentage of uptake? Does every game need multiplayer access constantly? Does it vary around the world for example Hearthstone EU will be way lower than US. Can anyone think of categories to define each level? The issues around the subjectively of option menu could be overcome if we have a required set of options for each Genre and they lose X points for each missing. Clearly I could rabbit on this issue of hours but I decided to call it quits and move on to another topic as its late and I am not sure how many will even read this.
Another issue with the PCMr system is the absent of certain key components. For example in Resolution 1080p is all well and good but what about people with 16:10 monitors? When hard descriptions are created for this topic if it includes other resolutions options will be essential but is currently absent from the system. As previously discussed I also recommend completely scrapping the number system and replacing it more objective measures as its just to confusing. On a similar note the PCMR icons though impressive are not descriptive and therefore for this to have any reach outside of our community i.e. on steam pages they need to be reworked to shows what they entail or link to where people can find out.
Inaddition the PCMR system will be limited by other factors. Firstly users will be more likely to respond if they really enjoy it or it is amazing leading to the results not showing a wide spectrum of results. Secondly, different amount of reviews will exists on each game is it fair to compare 1 review to 100 so a certain amount of reviews (my rec is 20) will need to be implemented before we give it a ranking. Also trolls and 'Funny' reviews will need to be removed as they are already prevalent on the ratings site. This is again another controversial issue as unoptimised games may run on specific configuration and optimised games may work shit on a specific set up.
Now on to the most controversial aspect of my critique. I think the PCMR branding may need to be dropped. As I discussed in part A this is really require for PC gamers in this day and age. However, many hold negative views to our beloved community both from fear mongering by the traditional press (Ie Were all Nazis) and those spammers on youtube. Therfore only indies will likely want to feature are system or take any account of it. Therefore for this to have any impact outside of this sub (Which many people seem to want). We probably should slightly rebrand to avoid hate from the outside communities. To something around games aiding gamers.
Anyway I have waffled on for way to long, probably made several grammar errors and went of on a tangent and I am not sure if anyone will read this so I will end this here.
Thank you very much for reading,
The 1st Doctor. Edit 1: Corrected Several issues where Autocorrect as replaced a word.