Guidelines for Rating

Rating Categories
Terms of Use
Who can Rate?
Viewing Ratings and Rating Comments
How to Rate
Good Practice
Requesting Review of Inappropriate Content

1. Rating Categories

The PLoS Web site allows users to rate articles for subjective "quality". Scientific work can be measured on a number of scales. To reflect this, research articles can be rated in three separate categories: Insight, Reliability, and Style. Rating is done on a 1 to 5 scale with 1 being the lowest and 5 the highest rating.

Insight: This provides a measure of how thought-provoking a user found an article or how much it advances our scientific understanding. The scale for Insight ranges from 1, Bland, a report which provides no more than an incremental advance on the published literature; to 5, Profound, a report which substantially deepens or alters current thinking.
Reliability: This provides a measure of how secure a user feels the results and conclusion in a study are. The scale for Reliability ranges from 1, Tenuous, the study is preliminary and will need confirmation; to 5, Unassailable, the results are of high quality, the reasoning is tight, and the conclusions are completely solid.
Style: This provides a measure of how well performed and presented a user considers a study to be. The scale for Style ranges from 1, Crude, the technical accomplishment and presentation of the study is adequate at best; to 5, Elegant, the study satisfyingly presents the results of technically accomplished and expertly executed experiments.

Users can rate articles in any or all categories; the ratings given are combined to give an overall rating from that user. Note that Front Matter articles, such as Editorials, Perspectives, and Reviews, can only be given a single overall rating.

Ratings from all users are combined to give average ratings in each category and a combined overall rating. The number of users who have rated an article is also displayed.

Users can also supply comments to accompany their rating of an article. These can be used to explain the ratings given. Unlike other forms of comment, rating comments cannot be used as the beginning of a discussion thread. The comment field is optional; however, comments cannot be made unless the user also rates the article in at least one category.

2. Terms of Use

All Notes, Comments, Ratings, and other post-publication activity on the PLoS Web sites are subject to PLoS' Terms of Use.

3. Who Can Rate?

All registered users are able to rate any article. Anyone can register as a user. Users are required to unambiguously identify themselves and to supply a valid e-mail address in order to register. Registration information will be kept strictly confidential by PLoS staff unless otherwise indicated. PLoS reserves the right to suspend the privileges of any registered user. Rating is not anonymous.

4. Viewing Ratings and Rating Comments

The overall "Average Rating" for an article is displayed in the right hand navigation menu for all articles. Clicking on "See all categories" reveals the average rating in each category. Also displayed is the number of users who have rated the article; "User Ratings" links to a page containing the individual ratings from each user as well as any comments they have included.

Ratings in all and any category are displayed on articles whether they have been rated or not. However, when compiling lists based on ratings, PLoS will only include articles rated by more than 10 users.

5. How to Rate

To rate an article first make sure you are logged into the Web site. Then, click the "Rate This Article" link in the right hand navigation menu of the article to be rated. This will open a separate window within which you can select your desired number of rating stars for the study and enter any comments that you may wish to accompany your rating. Once completed, you will be returned to the article. Ratings and rating comments can be modified at any time by following the "edit my rating" link in the right hand navigation menu.

6. Good Practice

Rating comments are expected to be much shorter and pithier than other forms of commenting. Nevertheless they must conform to the same norms of civilized scientific discussion. Any contributions that do not meet these standards will be removed. Any users who consistently transgress these conventions will have their user privileges removed.

A partial list of commenting standards includes the following:

Language that is insulting, inflammatory, or obscene will not be tolerated.
Unsupported assertions or statements should be avoided. Comments must be evidence-based, not authority-based.
When previously published studies are cited, they must be accurately referenced and, where possible, a DOI and link to a publicly accessible version supplied.
Unpublished data should be provided with sufficient methodological detail for those data to be assessed. Alternatively, a permanent Web link to such information should be provided.
Arguments based on belief are to be avoided. For example the assertion, "I don't believe the results in Figure 2." must be supported.
Discussions should be confined to the demonstrable content of articles and should avoid speculation about the motivations or prejudices of authors.

PLoS is the final arbiter of the suitability of content for inclusion in the PLoS Web sites.

7. Requesting Review of Inappropriate Content

To request review of a comment you believe to be inappropriate, or if you suspect that the rating system is being abused, click the "Request review" link in the bottom-right corner of the rating. Indicate why the rating should be reviewed ("Spam," "Offensive," "Inappropriate," or "Other"), enter additional information in the text box, and click "Submit". PLoS staff will investigate the concern, which may involve consulting external experts if necessary. Any comments that PLoS staff deem to be inappropriate will be removed. Any users who consistently posts inappropriate material will have their user privileges removed.

PLoS is the final arbiter of the suitability of content for inclusion in the PLoS Web sites.