Peer review

From Testiwiki
Revision as of 07:18, 17 November 2009 by Mikko Pohjola (talk | contribs)
Jump to: navigation, search


<section begin=glossary/>

Peer review is a method for evaluating the scientific quality of a piece of information. In peer review a number of people that can be considered as reasonably acquainted with the topic that the piece of information addresses give their statement whether or not the piece of information is of good enough quality for publication in a scientific journal.

<section end=glossary/>

Most often peer review is considered in the context of publishing scientific articles that tend to be descriptions of scientific studies and their results. Peer review can also used as a means of controlling quality of assessments and their outputs. Basically peer review is actually about acceptability of the process of producing information, and thereby also acceptability of the outcomes of that process. However, peer review is usually not a systematic method, but rather a practice that builds on the assumption that peers can implicitly identify good works from bad ones based on their own expertise. Consequently peer review also often ends up addressing also questions of e.g. usability and relevance, in a relatively random fashion. Despite its shortcomings, peer review does have value in quality control, also in the context of assessment.

Technically the peer review can be done so that any piece of information is set available for peers to access and evaluate, and anyone who feels qualified to evaluate a given piece of information can go ahead and give her statement about its quality. The pieces of information can be assessments, or individual variables, or studies of any kind. The statement basically is whether the evaluator thinks that the piece of information is or is not of good enough quality that it could be published in a scientific journal. The levels of evaluation can then be e.g. 1) not reviewed, 2) reviewed, but not accepted, 3) reviewed and accepted. The amount of required acceptance statements can be agreed according to what is seen suitable for the system to be flexible, but still credible. Perhaps two or three, as in many scientific journals, is enough.

Peer review in Opasnet - an example of an open web-based review system

Basically, Opasnet is applying an open peer review process in its widest sense. It means that anyone can make a peer review about anything. However, a peer review is worthless unless the readers believe that the reviewer actually is a peer, which means a person who has enough relevant expertise, usually a fellow researcher. Therefore, the following guidance is advised:

  • If you need the information of a page in your assessment or other work and the page has not been reviewed yet, you should consider reviewing the page yourself before using it. Or, if you don't feel qualified, you should put some effort in finding a person who could review the page. This way, you increase the credibility of your own work, and you also help the Open Assessors' Network to evaluate and improve the contents of Opasnet.
  • You can peer review a page in Opasnet, if you have a credible record of expertise in the area of the page. It is advised that reviewers put enough information about this on their user page (maybe a brief curriculum vitae and a list of publications).
  • You should not be a major contributor of the page you review, i.e. you should not be one of those who have brought a substantive amount of scientific material to the page. Technical and linguistic edits can be done without limitation.
    • The roles of each contributor are clarified in the Acknowledgements of the page.

References


See also