Difference between revisions of "Peer review"

From Testiwiki
Jump to: navigation, search
(improvements, but half way)
(→‎See also)
 
(17 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
[[Category:Open assessment]]
 
[[Category:Open assessment]]
−
[[Category:Quality control]]
+
{{encyclopedia|moderator=Jouni
−
[[Category:Glossary term]]
+
| reference = {{publication
−
{{method}}
+
| authors        = Jouni T. Tuomisto, Mikko Pohjola
−
This page is about peer review in [[open assessment]]. For other uses, see [[:en:Peer review|Peer review]] in Wikipedia.
+
| page          = Peer review
 +
| explanation    =
 +
| publishingyear = 2010
 +
| urn            =
 +
| elsewhere      =
 +
}}
 +
}}
  
−
'''Peer review''' in [[open assessment]] is a [[method]] for evaluating [[uncertainty|uncertainties]] that are not explicitly captured in the [[definition]] of the object (typically an [[assessment]] or a [[variable]]). Technically, it is a [[discussion]] on the Talk page and has a statement about whether the [[definition]] isthe following [[statement]]:
+
<section begin=glossary/>
−
: "This [[object]] describes a [[phenomenon]] that is defined in the [[scope]]. The description is located in the [[result]]. How the description was derived is documented in the [[definition]]. The definition/[[formula]] is a scientifically fulfills the following criteria:
+
:'''Peer review''' is a [[method]] for evaluating the scientific quality of a piece of information. In peer review a number of people that can be considered as reasonably acquainted with the topic that the piece of information addresses give their statement whether or not the piece of information is of good enough quality for publication in a scientific journal.
−
:* The data used is representative and unbiased.
+
<section end=glossary/>
−
:*
 
−
:* The causalities described are a comprehensive list. to describingscription reflects the [[reality]] in such a precise way that the uncertainties related to the [[result]]s and/or [[conclusion]]s can effectively and truthfully be evaluated using the [[parameter uncertainty|parameter uncertainties]] described. In other words, there are no known [[model uncertainty|model uncertainties]] that 1) would bias the results and that 2) are currently omitted."
 
  
−
==Scope==
+
Most often peer review is considered in the context of publishing scientific articles that tend to be descriptions of scientific studies and their results. Peer review can also used as a means of controlling quality of assessments and their outputs. Basically peer review is actually about acceptability of the process of producing information, and thereby also acceptability of the outcomes of that process. However, peer review is usually not a systematic method, but rather a practice that builds on the assumption that peers can implicitly identify good works from bad ones based on their own expertise. Consequently peer review also often ends up addressing also questions of e.g. usability and relevance, in a relatively random fashion. Despite its shortcomings, peer review does have value in quality control, also in the context of assessment.
  
−
What is the method of gaining social acceptance to an [[object]]?
+
Technically the peer review can be done so that any piece of information is set available for peers to access and evaluate, and anyone who feels qualified to evaluate a given piece of information can go ahead and give her statement about its quality. The pieces of information can be assessments, or individual variables, or studies of any kind. The [[statement]] basically is whether the evaluator thinks that the piece of information is or is not of good enough quality that it could be published in a scientific journal. The levels of evaluation can then be e.g. 1) not reviewed, 2) reviewed, but not accepted, 3) reviewed and accepted. The amount of required acceptance statements can be agreed according to what is seen suitable for the system to be flexible, but still credible. Perhaps two or three, as in many scientific journals, is enough.
  
−
==Definition==
+
'''Peer review in Opasnet - an example of an open web-based review system
  
−
===Input===
+
Basically, Opasnet is applying an open peer review process in its widest sense. It means that anyone can make a peer review about anything. However, a peer review is worthless unless the readers believe that the reviewer actually is a peer, which means a person who has enough relevant expertise, usually a fellow researcher. Therefore, the following guidance is advised:
 +
* If you need the information of a page in your assessment or other work and the page has not been reviewed yet, you should consider reviewing the page yourself before using it. Or, if you don't feel qualified, you should put some effort in finding a person who could review the page. This way, you increase the credibility of your own work, and you also help the [[Open Assessors' Network]] to evaluate and improve the contents of [[Opasnet]].
 +
* You can peer review a page in Opasnet, if you have a credible record of expertise in the area of the page. It is advised that reviewers put enough information about this on their user page (maybe a brief curriculum vitae and a list of publications).
 +
* You should not be a major contributor of the page you review, i.e. you should not be one of those who have brought a substantive amount of scientific material to the page. Technical and linguistic edits can be done without limitation.
 +
** The roles of each contributor are clarified in the [[Acknowledgements]] of the page.
  
−
===Output===
+
== Research: Increasing value, reducing waste ==
  
−
===Rationale===
+
''Increasing value, reducing waste'' is a special issue in Lancet focussing on how to improve research and the evaluation processes of scientific work. It was published January 8, 2014.[http://www.thelancet.com/series/research]
  
−
==Result==
+
The Lancet presents a Series of five papers about research. In the first report Iain Chalmers et al discuss how decisions about which research to fund should be based on issues relevant to users of research. Next, John Ioannidis et al consider improvements in the appropriateness of research design, methods, and analysis. Rustam Al-Shahi Salman et al then turn to issues of efficient research regulation and management. Next, An-Wen Chan et al examine the role of fully accessible research information. Finally, Paul Glasziou et al discuss the importance of unbiased and usable research reports. These papers set out some of the most pressing issues, recommend how to increase value and reduce waste in biomedical research, and propose metrics for stakeholders to monitor the implementation of these recommendations.
  
−
===Procedure===
+
* Sabine Kleinert, Richard Horton. How should medical science change? [http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(13)62329-6/fulltext]
 +
* Malcolm R Macleod, Susan Michie, Ian Roberts, Ulrich Dirnagl, Iain Chalmers, John P A Ioannidis, Rustam Al-Shahi Salman, An-Wen Chan, Paul Glasziou. Biomedical research: increasing value, reducing waste. The Lancet, Volume 383, Issue 9912, Pages 101 - 104, 11 January 2014. {{doi|10.1016/S0140-6736(13)62329-6}} [http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(13)62329-6/fulltext]
 +
* Iain Chalmers, Michael B Bracken, Ben Djulbegovic, Silvio Garattini, Jonathan Grant, A Metin Gülmezoglu, David W Howells, John P A Ioannidis, Sandy Oliver. How to increase value and reduce waste when research priorities are set. The Lancet, Volume 383, Issue 9912, Pages 156 - 165, 11 January 2014. {{doi|10.1016/S0140-6736(13)62229-1}} [http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(13)62229-1/fulltext]
 +
* John P A Ioannidis, Sander Greenland, Mark A Hlatky, Muin J Khoury, Malcolm R Macleod, David Moher, Kenneth F Schulz, Robert Tibshirani. Increasing value and reducing waste in research design, conduct, and analysis. The Lancet, Volume 383, Issue 9912, Pages 166 - 175, 11 January 2014. {{doi|10.1016/S0140-6736(13)62227-8}} [http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(13)62227-8/fulltext]
 +
* Rustam Al-Shahi Salman, Elaine Beller, Jonathan Kagan, Elina Hemminki, Robert S Phillips, Julian Savulescu, Malcolm Macleod, Janet Wisely, Iain Chalmers. Increasing value and reducing waste in biomedical research regulation and management. The Lancet, Volume 383, Issue 9912, Pages 176 - 185, 11 January 2014. {{doi|10.1016/S0140-6736(13)62297-7}} [http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(13)62297-7/fulltext]
 +
* An-Wen Chan, Fujian Song, Andrew Vickers, Tom Jeff erson, Kay Dickersin, Peter C Gøtzsche, Harlan M Krumholz, Davina Ghersi,H Bart van der Worp. Increasing value and reducing waste: addressing inaccessible research. The Lancet, Volume 383, Issue 9913, Pages 257 - 266, 18 January 2014. {{doi|10.1016/S0140-6736(13)62296-5}} [http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(13)62296-5/fulltext]
 +
* Paul Glasziou, Douglas G Altman, Patrick Bossuyt, Isabelle Boutron, Mike Clarke, Steven Julious, Susan Michie, David Moher, Elizabeth Wager. Reducing waste from incomplete or unusable reports of biomedical research. The Lancet, Volume 383, Issue 9913, Pages 267 - 276, 18 January 2014. {{doi|10.1016/S0140-6736(13)62228-X}} [http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(13)62228-X/fulltext]
  
−
'''Peer review''' in [[open assessment]] is a [[method]] for evaluating [[model uncertainty]], i.e. [[uncertainty]] that is not captured by the explicit [[parameter uncertainty]], which is described by model parameters and distributions. Technically, it is a [[discussion]] about an [[object]] (typically an [[assessment]] or a [[variable]]), which has the following [[statement]]:
+
== References ==
−
: "This [[object]] describes a [[phenomenon]] that is defined in the [[scope]]. The description reflects the [[reality]] in such a precise way that the uncertainties related to the [[result]]s and/or [[conclusion]]s can effectively and truthfully be evaluated using the [[parameter uncertainty|parameter uncertainties]] described. In other words, there are no known [[model uncertainty|model uncertainties]] that 1) would bias the results and that 2) are currently omitted."
 
  
−
===Management===
+
<references/>
−
 
 
−
The [[peer review]] [[discussion]] has the following form:
 
−
 
 
−
<big>'''Peer review'''</big>
 
−
 
 
−
{{discussion
 
−
|Dispute= This [[object]] describes a [[phenomenon]] that is defined in the [[scope]] of the object. The description reflects the [[reality]] in such a precise way that the uncertainties related to the [[result]]s and/or [[conclusion]]s can effectively and truthfully be evaluated using the [[parameter uncertainty|parameter uncertainties]] described. In other words, there are no known [[model uncertainty|model uncertainties]] that 1) would bias the results and that 2) are currently omitted.
 
−
|Outcome=
 
−
|Argumentation =
 
−
{{defend|1|The [[definition]] reflects the state-of-the-art of this field and does not lack any such sources of information that would clearly deviate the result from the current result.|--[[User:Jouni|Jouni]] 11:37, 16 January 2009 (EET)}}
 
−
 
 
−
{{attack|3|The issue described in argument 2 is missing.|--[[User:Jouni|Jouni]] 11:37, 16 January 2009 (EET)}}
 
−
:{{defend|2|The issue of ...(describe the issue here)... is important and relevant for this object.|--[[User:Jouni|Jouni]] 11:37, 16 January 2009 (EET)}}
 
−
 
 
−
}}
 
  
 
==See also==
 
==See also==
  
−
==References==
+
* [http://opeer.org/ o'Peer website]
 +
** [https://www.phas.ubc.ca/users/jesse-brewer Jesse Brewer, the founder of o'Peer]
 +
* [http://blogs.biomedcentral.com/bmcblog/2014/04/11/are-journals-ready-to-abolish-peer-review-2/?utm_campaign=14_05_14_BMCUpdate_Newsletter Are journals ready to abolish peer review] (a BMC event and blog)
 +
* [http://blogs.biomedcentral.com/bmcblog/2014/04/08/peer-review-chipped-not-broken/ Peer review chipped, not broken]
 +
* [[Peer review method]]
 +
* [[Template:Review]]
 +
* [[Quality assurance and quality control]]
 +
* [[Quality evaluation criteria]]
 +
* [[Template:Quality assessment]]
 +
* [http://openwetware.org/wiki/Peer_Review_Simulation_Project#Project_Details Peer review protocol for OpenWetWare]
  
−
<references/>
+
[[Category:Open assessment]]
 +
[[Category:Quality control]]
 +
[[Category:Glossary term]]
 +
[[Category:THL publications 2009]]
 +
[[Category:THL publications 2010]]

Latest revision as of 09:17, 14 June 2014



<section begin=glossary/>

Peer review is a method for evaluating the scientific quality of a piece of information. In peer review a number of people that can be considered as reasonably acquainted with the topic that the piece of information addresses give their statement whether or not the piece of information is of good enough quality for publication in a scientific journal.

<section end=glossary/>

Most often peer review is considered in the context of publishing scientific articles that tend to be descriptions of scientific studies and their results. Peer review can also used as a means of controlling quality of assessments and their outputs. Basically peer review is actually about acceptability of the process of producing information, and thereby also acceptability of the outcomes of that process. However, peer review is usually not a systematic method, but rather a practice that builds on the assumption that peers can implicitly identify good works from bad ones based on their own expertise. Consequently peer review also often ends up addressing also questions of e.g. usability and relevance, in a relatively random fashion. Despite its shortcomings, peer review does have value in quality control, also in the context of assessment.

Technically the peer review can be done so that any piece of information is set available for peers to access and evaluate, and anyone who feels qualified to evaluate a given piece of information can go ahead and give her statement about its quality. The pieces of information can be assessments, or individual variables, or studies of any kind. The statement basically is whether the evaluator thinks that the piece of information is or is not of good enough quality that it could be published in a scientific journal. The levels of evaluation can then be e.g. 1) not reviewed, 2) reviewed, but not accepted, 3) reviewed and accepted. The amount of required acceptance statements can be agreed according to what is seen suitable for the system to be flexible, but still credible. Perhaps two or three, as in many scientific journals, is enough.

Peer review in Opasnet - an example of an open web-based review system

Basically, Opasnet is applying an open peer review process in its widest sense. It means that anyone can make a peer review about anything. However, a peer review is worthless unless the readers believe that the reviewer actually is a peer, which means a person who has enough relevant expertise, usually a fellow researcher. Therefore, the following guidance is advised:

  • If you need the information of a page in your assessment or other work and the page has not been reviewed yet, you should consider reviewing the page yourself before using it. Or, if you don't feel qualified, you should put some effort in finding a person who could review the page. This way, you increase the credibility of your own work, and you also help the Open Assessors' Network to evaluate and improve the contents of Opasnet.
  • You can peer review a page in Opasnet, if you have a credible record of expertise in the area of the page. It is advised that reviewers put enough information about this on their user page (maybe a brief curriculum vitae and a list of publications).
  • You should not be a major contributor of the page you review, i.e. you should not be one of those who have brought a substantive amount of scientific material to the page. Technical and linguistic edits can be done without limitation.
    • The roles of each contributor are clarified in the Acknowledgements of the page.

Research: Increasing value, reducing waste

Increasing value, reducing waste is a special issue in Lancet focussing on how to improve research and the evaluation processes of scientific work. It was published January 8, 2014.[2]

The Lancet presents a Series of five papers about research. In the first report Iain Chalmers et al discuss how decisions about which research to fund should be based on issues relevant to users of research. Next, John Ioannidis et al consider improvements in the appropriateness of research design, methods, and analysis. Rustam Al-Shahi Salman et al then turn to issues of efficient research regulation and management. Next, An-Wen Chan et al examine the role of fully accessible research information. Finally, Paul Glasziou et al discuss the importance of unbiased and usable research reports. These papers set out some of the most pressing issues, recommend how to increase value and reduce waste in biomedical research, and propose metrics for stakeholders to monitor the implementation of these recommendations.

  • Sabine Kleinert, Richard Horton. How should medical science change? [3]
  • Malcolm R Macleod, Susan Michie, Ian Roberts, Ulrich Dirnagl, Iain Chalmers, John P A Ioannidis, Rustam Al-Shahi Salman, An-Wen Chan, Paul Glasziou. Biomedical research: increasing value, reducing waste. The Lancet, Volume 383, Issue 9912, Pages 101 - 104, 11 January 2014. doi:10.1016/S0140-6736(13)62329-6 [4]
  • Iain Chalmers, Michael B Bracken, Ben Djulbegovic, Silvio Garattini, Jonathan Grant, A Metin Gülmezoglu, David W Howells, John P A Ioannidis, Sandy Oliver. How to increase value and reduce waste when research priorities are set. The Lancet, Volume 383, Issue 9912, Pages 156 - 165, 11 January 2014. doi:10.1016/S0140-6736(13)62229-1 [5]
  • John P A Ioannidis, Sander Greenland, Mark A Hlatky, Muin J Khoury, Malcolm R Macleod, David Moher, Kenneth F Schulz, Robert Tibshirani. Increasing value and reducing waste in research design, conduct, and analysis. The Lancet, Volume 383, Issue 9912, Pages 166 - 175, 11 January 2014. doi:10.1016/S0140-6736(13)62227-8 [6]
  • Rustam Al-Shahi Salman, Elaine Beller, Jonathan Kagan, Elina Hemminki, Robert S Phillips, Julian Savulescu, Malcolm Macleod, Janet Wisely, Iain Chalmers. Increasing value and reducing waste in biomedical research regulation and management. The Lancet, Volume 383, Issue 9912, Pages 176 - 185, 11 January 2014. doi:10.1016/S0140-6736(13)62297-7 [7]
  • An-Wen Chan, Fujian Song, Andrew Vickers, Tom Jeff erson, Kay Dickersin, Peter C Gøtzsche, Harlan M Krumholz, Davina Ghersi,H Bart van der Worp. Increasing value and reducing waste: addressing inaccessible research. The Lancet, Volume 383, Issue 9913, Pages 257 - 266, 18 January 2014. doi:10.1016/S0140-6736(13)62296-5 [8]
  • Paul Glasziou, Douglas G Altman, Patrick Bossuyt, Isabelle Boutron, Mike Clarke, Steven Julious, Susan Michie, David Moher, Elizabeth Wager. Reducing waste from incomplete or unusable reports of biomedical research. The Lancet, Volume 383, Issue 9913, Pages 267 - 276, 18 January 2014. doi:10.1016/S0140-6736(13)62228-X [9]

References


See also

Retrieved from "http://www.opasnet.org/testiwiki/index.php?title=Peer_review&oldid=31947"