Difference between revisions of "Peer review"

From Testiwiki
Jump to: navigation, search
m (See also)
Line 1: Line 1:
[[Category:Open assessment]]
 
[[Category:Quality control]]
 
[[Category:Glossary term]]
 
 
{{encyclopedia}}
 
{{encyclopedia}}
This page is about peer review in [[open assessment]]. For a full description, see the [[Peer review method]]. For other uses, see the [[:en:Peer review|Peer review]] page in Wikipedia.
 
  
<section begin=glossary />
+
<section begin=glossary/>
:'''Peer review''' in [[open assessment]] is a [[method]] for evaluating the scientific quality of the [[definition]] of a [[variable]] or an [[assessment]], or the generalisability of the scope of a [[study]]. Technically, it is a [[discussion]] on the Talk page of the object. In the case of a variable or an assessment, the discussion has the following [[statement]]:
+
:'''Peer review''' is a [[method]] for evaluating the scientific quality of a piece of information. In peer review a number of people that can be considered as reasonably acquainted with the topic that the piece of information addresses give their statement whether or not the piece of information is of good enough quality for publication in a scientific journal.
:: "The definition of this object is based on the state-of-the-art scientific knowledge and methods. The data used is representative and unbiased. The causalities are described in a well-founded way. The formula correctly describes how the result can be calculated based on the data and causalities. Overall, the information in the definition reflects the current scientific understanding and is unlikely to substantially change because of some existing information that is omitted."
+
<section end=glossary/>
:In the case of a variable, the definition (the quality of content of the data, causalities, and formula attributes) is evaluated against the scope, which is fixed. In the case of a study, the scope is evaluated against the definition (i.e. the scientific work performed), which is fixed. Thus, the question is about how much it is possible to generalise from the results of a study.
 
  
<section end=glossary />
+
Most often peer review is considered in the context of publishing scientific articles that tend to be descriptions of scientific studies and their results. Peer review can also used as a means of controlling quality of assessments and their outputs. Basically peer review is actually about acceptability of the process of producing information, and thereby also acceptability of the outcomes of that process. However, peer review is usually not a systematic method, but rather a practice that builds on the assumption that peers can implicitly identify good works from bad ones based on their own expertise. Consequently peer review also often ends up addressing also questions of e.g. usability and relevance, in a relatively random fashion. Despite its shortcomings, peer review does have value in quality control, also in the context of assessment.
  
====Peer review of the definition (for assessments and variables)====
+
Technically, it is a [[discussion]] on the Talk page of the object and has the following [[statement]]:
 
 
:'''Peer review''' in [[open assessment]] is a [[method]] for evaluating the scientific quality of the [[definition]] of a [[variable]] or an [[assessment]], or the generalisability of the scope of a [[study]]. Technically, it is a [[discussion]] on the Talk page of the object and has the following [[statement]]:
 
 
: "The definition of this object is based on the state-of-the-art scientific knowledge and methods. The data used is representative and unbiased. The causalities are described in a well-founded way. The formula correctly describes how the result can be calculated based on the data and causalities. Overall, the information in the definition reflects the current scientific understanding and is unlikely to substantially change because of some existing information that is omitted."
 
: "The definition of this object is based on the state-of-the-art scientific knowledge and methods. The data used is representative and unbiased. The causalities are described in a well-founded way. The formula correctly describes how the result can be calculated based on the data and causalities. Overall, the information in the definition reflects the current scientific understanding and is unlikely to substantially change because of some existing information that is omitted."
 
====Peer review of the scope (for a study)====
 
  
 
[[Study]] is a special kind of object in the sense that the definition typically describes a particular study that has been performed. Therefore, it is not possible to evaluate (in the sense of attempting to improve) the definition as such, because what was done was done. Instead, the interesting question is about the generalisability of the results.  
 
[[Study]] is a special kind of object in the sense that the definition typically describes a particular study that has been performed. Therefore, it is not possible to evaluate (in the sense of attempting to improve) the definition as such, because what was done was done. Instead, the interesting question is about the generalisability of the results.  
Line 23: Line 14:
 
Thus, the peer review of a study aims to answer this question: '''"To what questions do the study actually answer reliably, based on the current scientific understanding?"
 
Thus, the peer review of a study aims to answer this question: '''"To what questions do the study actually answer reliably, based on the current scientific understanding?"
  
====Who can and should do a peer review?====
+
====Who can and should do a peer review in Opasnet?====
  
 
Basically, Opasnet is applying an open peer review process in its widest sense. It means that anyone can make a peer review about anything. However, a peer review is worthless unless the readers believe that the reviewer actually is a peer, which means a person who has enough relevant expertise, usually a fellow researcher. Therefore, the following guidance is advised:
 
Basically, Opasnet is applying an open peer review process in its widest sense. It means that anyone can make a peer review about anything. However, a peer review is worthless unless the readers believe that the reviewer actually is a peer, which means a person who has enough relevant expertise, usually a fellow researcher. Therefore, the following guidance is advised:
Line 34: Line 25:
  
 
* [[Peer review method]]
 
* [[Peer review method]]
 +
* [[Template:Review]]
 +
* [[Quality assurance and quality control]]
 +
* [[Quality evaluation criteria]]
 +
* [[Template:Quality assessment]]
  
 
==References==
 
==References==
  
 
<references/>
 
<references/>
 +
 +
[[Category:Open assessment]]
 +
[[Category:Quality control]]
 +
[[Category:Glossary term]]

Revision as of 14:29, 16 November 2009


<section begin=glossary/>

Peer review is a method for evaluating the scientific quality of a piece of information. In peer review a number of people that can be considered as reasonably acquainted with the topic that the piece of information addresses give their statement whether or not the piece of information is of good enough quality for publication in a scientific journal.

<section end=glossary/>

Most often peer review is considered in the context of publishing scientific articles that tend to be descriptions of scientific studies and their results. Peer review can also used as a means of controlling quality of assessments and their outputs. Basically peer review is actually about acceptability of the process of producing information, and thereby also acceptability of the outcomes of that process. However, peer review is usually not a systematic method, but rather a practice that builds on the assumption that peers can implicitly identify good works from bad ones based on their own expertise. Consequently peer review also often ends up addressing also questions of e.g. usability and relevance, in a relatively random fashion. Despite its shortcomings, peer review does have value in quality control, also in the context of assessment.

Technically, it is a discussion on the Talk page of the object and has the following statement:

"The definition of this object is based on the state-of-the-art scientific knowledge and methods. The data used is representative and unbiased. The causalities are described in a well-founded way. The formula correctly describes how the result can be calculated based on the data and causalities. Overall, the information in the definition reflects the current scientific understanding and is unlikely to substantially change because of some existing information that is omitted."

Study is a special kind of object in the sense that the definition typically describes a particular study that has been performed. Therefore, it is not possible to evaluate (in the sense of attempting to improve) the definition as such, because what was done was done. Instead, the interesting question is about the generalisability of the results.

Thus, the peer review of a study aims to answer this question: "To what questions do the study actually answer reliably, based on the current scientific understanding?"

Who can and should do a peer review in Opasnet?

Basically, Opasnet is applying an open peer review process in its widest sense. It means that anyone can make a peer review about anything. However, a peer review is worthless unless the readers believe that the reviewer actually is a peer, which means a person who has enough relevant expertise, usually a fellow researcher. Therefore, the following guidance is advised:

  • If you need the information of a page in your assessment or other work and the page has not been reviewed yet, you should consider reviewing the page yourself before using it. Or, if you don't feel qualified, you should put some effort in finding a person who could review the page. This way, you increase the credibility of your own work, and you also help the Open Assessors' Network to evaluate and improve the contents of Opasnet.
  • You can peer review a page in Opasnet, if you have a credible record of expertise in the area of the page. It is advised that reviewers put enough information about this on their user page (maybe a brief curriculum vitae and a list of publications).
  • You should not be a major contributor of the page you review, i.e. you should not be one of those who have brought a substantive amount of scientific material to the page. Technical and linguistic edits can be done without limitation.
    • The roles of each contributor are clarified in the Acknowledgements of the page.

See also

References