The Rating Panel Information Thread

[Andail]

The Rating Panel Information Thread

What is the Rating Panel?
The Rating Panel is essentially a board of forum members that works with cataloguing, maintaining and rating the games in the AGS database. The daily work includes sorting games into the proper categories (genre, length etc) fixing or flagging broken links, rating the games and commenting on them, time permitting.

Who are in the Panel?
I work as coordinator and supervisor of the rating panel project. Progzmax is the current head of the panel.
The rest of the individual Rating Panel members are currently anonymous. This is to prevent coercion and abuse from game authors.

How does the Panel work?
The Panel works on the following premises:
1. Every member records which game he/she has rated in a special thread, along with a motivation. There is no anonymity within the panel.
2. Any rating can be challenged and discussed by fellow panel members.
3. The panel works under the supervision of each other and moderators.
4. The panel members are instructed and trained to rate according to certain guidelines and criteria that have been worked out in beforehand.
5. The panel members do not rate their own games.

What are the rating guidelines?
This is the official Rating Panel document.

Technical Rating
--------------------

1.  Is the game fully playable (can you complete the game without crashing or other severe bugs)?
2.  Are the puzzles and other gameplay elements competently designed and logical?
3.  Are the controls obvious or explained either by a readme or in-game tutorial?
4.  Is the language comprehensible enough to complete the game without confusion and/or difficulty?

Art/Music Rating
--------------------

1.  Are the sprite assets and animations original and in a style consistent with the rest of the game?
2.  Are the backgrounds original and in a style consistent with the rest of the game?
3.  Is the music original and in a style consistent with the rest of the game?
4.  Are the sound effects original and in a style consistent with the rest of the game?

Story Rating
---------------
1.  Is the story engrossing or thought provoking?
2.  Are the characters (and particularly the protagonist) engaging, with unique personalities and motivations?
3.  Do the characters and story elements remain consistent throughout the game?
4.  Is the dialogue engaging and consistent with the characters?

Gameplay Rating
---------------------
1.  Are the controls easy to learn?
2.  Is the user interface easy to understand and use?
3.  Is the game challenging without being punishing?
4.  Overall, is the game an enjoyable experience?

There is no mathematical function to decide the number of cups, but basically a game with 5 cups have only one or two no's. To achieve 2 cups at least half of the criteria must be met.


Why is my game so poorly rated?
First of all, it probably isn't. These ratings do not follow a typical bell shape diagram; very few games are awarded 4 cups, and only a handful have thus far earned 5 cups. Most games that are considered worth playing get 2 and 3 cups.

It's hard to be objective when you assess your own creation. Instead of getting upset, try to turn the c&c into something productive. With the right mindset, the Panel rating should be considered a useful tool.

Can a particular rating be appealed?
Yes, you can PM me, Andail, and state your case. As with any jury, judge or similar instance, it's unlikely that the Panel will revise their rating, but you're welcome to voice your opinion.

You can also demand that the Panel give you a more specified motivation for a particular rating. The Panel will try to formulate a longer comment to justify the rating.

Can I discuss the guidelines or rules pertaining to the rating?
Yes. But bear in mind that this project is ultimately sanctioned by CJ. This is his board, and you are using his program to make games that you publish on his site. Not everyone can have their saying. Everyone is bound to have different opinions, but not every single one of the thousands of game authors can ever be pleased.

Please remember to take the rating with a pinch of salt. At the end of the day, this is a way to bring order to the vast database.

Can I be a member of the Panel?
You may apply via a PM to me or Progzmax, but mind that the rotation of panelists is very slow.
It is required of panelists that
* They have been part of the AGS community for 2 years or more
* They are trusted, and not prone to cause controversy or violate forum policies
* They can set aside enough time - each panelist needs to play through several games every week to keep up
* They can sit for at least 4-6 months, to ensure consistent and fair ratings

Thank you for your attention.