Description of the peer review system
What is Peer Review?
Journal articles generally appear in print about a year after the work described in the article was completed. Once an article appears in print, it is widely available to researchers and clinicians. The printed article may be perceived as having a certain authoritarian air, and it might be assumed that the contents and conclusions are valid since they would otherwise be unlikely to have been published. However, this assumption may not be sound, hence the need for a critical approach to information including published articles. It is important that published material is not misleading, since research directions and clinical decisions may be based on information gleaned from such articles. For this reason a system of peer review is widely used. Jefferson et al (2002) note that the aims of peer review are not clearly outlined in the literature, and suggest that the system has the following aims:
- Selecting submissions for publication (by a particular journal) and rejecting those with irrelevant, trivial, weak, misleading, or potentially harmful content.
- Improving the clarity, transparency, accuracy, and utility of the selected submissions.
EBP Process
The system of peer review ensures that published articles have been read critically by anonymous reviewers prior to publication. The reviewers read and review the article independently, without influencing each other, and submit their reviews to an editor. Taking at least two such reviews into account, the editor decides whether the article should be published or not, and conveys this outcome to the author. If the article is not rejected, it is usual for minor or major revisions to be required before publication, based on reviewers’ comments. The peer-review system aims to ensure that published articles provide reliable information, by scrutinizing the article for flaws in methodology or argument, and for unjustified claims. Articles that are published without this type of scrutiny are more likely to include flaws, and are less likely to provide the reader with reliable information.
The peer-review system is not without fault, of course. In a discussion of the importance of truth as a central theme in science, Charlton (2009) points out that the peer review system in science is the main mechanism by which a scientist’s work is assessed for purposes of publication, funding and promotion. Charlton considers that the powerful and pervasive system of peer review has been responsible for “the disappearance of ‘real science’” and a lack of truth-seeking in modern scientific work, pointing to a culture of “hype” with scientists extremely keen to market their work to funding bodies, editors and other stakeholders. In a Cochrane review on the effect of peer review for research funding on quality of the funded research, Demicheli and Pietrantonj (2007) note that a range of peer review methods are currently used in assessments of applications for research funding. They found no studies investigating the impact of peer review of the quality of funded research, and noted that such studies are urgently needed, to appropriately guide the assessment of applications for research funding. Jefferson et al (2002), in a review of the quality of peer review for publications, also note a paucity of studies on effectiveness of the peer review system, and propose that work is needed in this area. It is possible, for example, that a reviewer misses a flaw in the article, or criticizes the work more heavily than might reasonably be justified. In this case, however, the two reviewers would probably disagree, so the system, while not perfect, does generally achieve its aim. The only recent paper to comment on the effectiveness of the peer review system is a recent Science special article by Bohannon (2013) found a lack of peer reviewed scrutiny when submitting a fabricated spoof paper to a number of open-access journals. This suggests that peer reviewed papers undergo more scrutiny than those that are not peer reviewed at all, yet suggests readers need to be cautious when appraising evidence regardless of peer review processes as they can vary across journals.
References
Bohannon, J. (2013) Who’s Afraid of Peer Review? Science 342: 60-65.
Charlton, B.G. (2009) The vital role of transcendental truth in science. Med Hypotheses 72(4): 373-376.
Demicheli, V. and Pietrantonj, C.D. (2008) Peer review for improving the quality of grant applications. Cochrane Database of Systematic Reviews. 18(2)
Jefferson, T., Wager, E. and Davidoff, F. (2002) Measuring the quality of editorial peer review. JAMA 287(21): 2786-2790.
Resource contributed by: Dr Catherine Suttle
Affiliation: University of New South Wales
Web-site:
E-mail: Catherine.Suttle.1@city.ac.uk
Notes: Please contact me if you would like more information about this resource.