As interest in global warming becomes embedded in almost all sectors of society, the nature of publications relevant to global warming has diversified.
communication-NewsImage.cropimg.width=700.crop=video.png)
As interest in global warming becomes embedded in almost all sectors of society, the nature of publications relevant to global warming has diversified. The foundations of our understanding of global warming exist in high quality international journal publications with rigorous peer review and independent advisory boards. However, now that global warming pervades almost all sectors of society, and because the peer reviewed science literature is not readily accessible to business, policy makers and the public, there has been a rapid expansion of more public-facing publications. These range from high quality technical whitepapers associated with reputable organizations through to advocacy documents that are understandably unbalanced in a strict scientific sense.
These reports are often written in far more accessible language than scientific papers, and their accessibility leads to their reporting by the mass media, sometimes with findings around climate risk projections that are not scientifically rigorous.
Recent examples include suggestions that Australia will “lose its AAA rating as soon as 2030” due to climate change1, or “suburbs where more than half of properties will be uninsurable by 2030” 2.
This raises two major issues. The first is around the presence, absence and rigor of the peer review process. The second is a separate issue, but one that often compounds the first, concerning the balance, and sometimes misreporting by mainstream media and indeed commenters on social media platforms. These issues, while distinct, result in the general public not being presented with a fair, balanced and reasonable grasp of climate risk. In particular, given the gravity of consequences within the context of climate science, this commentary aims to shed further light on the issue by addressing the following: the academic peer review process; challenges with science communication; unintended consequences when scientific findings lack robustness or are misrepresented, and suggestions for improvements.
Research funding is scarce and in Australia the success rate on winning major grants is below 20% - that means a small team of researchers writes a ~100 page grant proposal with an 80% chance of failure. One way to maximize the chances of winning funding is to have a superb publication record, and clear demonstration that the publications have had impact. One measure of impact is the number of citations a paper received and one way to increase the citation rate is to publish in high impact journals. Publishing in high impact or very high quality journals is extremely hard, and the required rigor means that the publications tend to be thorough, robust and reproducible.
However, the stresses on researchers to publish can have negative impacts. While extremely rare, cases of papers being withdrawn due to academic misconduct do occur. Pressures to publish occasionally lead to short-cuts although these are often identified in the peer review process leading to a paper being rejected. Reputational risks from academic misconduct are very real - while withdrawing a published paper due to an error being discovered is acceptable, having even a single paper withdrawn by a journal for plagiarism, fraud, misrepresentation of results and so on is likely to be severely career limiting. Indeed, at the highest level there are deterrents such as the Academy of Science requiring the declaration of withdrawn papers for Fellowship nomination.
A fundamental protection against mis-publication is the peer review process which has held science to a very high standard for many years. It is important to understand how the peer review process works in practice. Peer reviewed-journals contain articles that are not only written by experts, but also reviewed by fellow experts in the same field. Each journal will have their own reviewing process, but in general they follow a similar structure. For example, a new manuscript is first received by the journal editor. Elite journals reject some percentage - perhaps 40% - at this stage. If the Editor considers the paper worthy of peer review, they send the manuscript to two or three reviewers. These reviewers are usually experts in the field, and should have no direct links to the author(s) to minimize conflicts of interest. These experts review the article, provide feedback that the editor then sends back to the author(s) in an anonymised form. The feedback to the author will be “accept” (very rare), “minor revisions” (rare), “major revisions” (common) or “reject” (more common than most researchers like). This is not a democratic vote on the basis of the reviews. It is not uncommon to have two reviewers say “minor revisions” and one say “reject”. If the reasons for recommending “reject” are robust, the Editor will simply reject the paper. Depending on the nature of the feedback, the authors then make suggested revisions and the editor decides whether to seek a second (and sometimes third) round of reviews. Ultimately, the Editor decides whether the manuscript is accepted or rejected for publication.
The review load for a challenging paper could be 3-5 days work, is unpaid and is not properly recognised in academic workloads. As such, there is an incentive to review papers adequately, rather than thoroughly. In addition, an Editor of a journal is normally unpaid - and it is a great deal of work to find three good reviewers and get them to review the paper rigorously. Journals have databases of reviewers, and an Editor may seek reviewers by exploring the literature themselves. For cross-disciplinary fields such as climate economics, this is increasingly hard - an outstanding climate scientist may have no experience in (say) economics and the paper may blend physical climate science with economics. The lack of deep understanding across physical climate science and other domains has become evident in some of the recent literature published within economics, business and financial journals;
the work might be brilliant economics but it demonstrates a lack of understanding of basic principles underpinning climate science.
The same problem might even occur within the same broader discipline itself - say climate scientists working on aerosols may know far less about complex groundwater contamination issues for example. Ideally, cross-disciplinary papers should be reviewed by climate scientists to assess the robustness of the physical climate risk assessments, climate impact researchers, economists and so on - each hunting for errors. This places such an onerous demand on Editors and it is effectively infeasible.
This leads to a simple conclusion, but one that is not often always appreciated by a lay audience. Papers in the peer reviewed literature are not necessarily correct and it takes time for multiple lines of evidence to form that really allow robust conclusions to be reached. A single paper, in isolation, is very rarely the whole story and understanding the way one paper fits into the broad scientific understanding requires very significant domain expertise. It is very common that the mass media pick up on a sensational new result, and it is found with hindsight that the paper is not quite as sensational as the media thought. That said, a paper in the peer reviewed literature has been through a detailed process with independent and usually anonymous reviewers. The paper might not be right, but it is far more likely right than a blog.
The role of the mass media is not to robustly report climate science. Inevitably, the media cannot report the full complexity and nuances of publications and most media have editorial policies or guidelines. For example, The Guardian rarely publishes pieces that hint that global warming might not be an existential threat, while The Australian tends to publish pieces hinting that evidence that the Earth is warming is alarmist.
It is worth remembering that most media is trying to sell advertising, not report science without fear or favour.
If one only reads The Guardian, the perception you will have is that the summer of 2023 in Australia will be the hottest on record, with mass coral bleaching, drought, fire, dangerous heatwaves and so on. A perception on reading The Australian is that it might be a little warmer than average, but no one really knows. The truth is nuanced and it would take considerably more effort on the part of the reader to tease out the probability of one of these outlets being more right than the other on each issue, why, and what this means in the context of climate change as distinct from natural climate variability.
In many cases, the mass media is simply looking to fill column inches, albeit with a particular editorial agenda. It really is not core to the mass media to distinguish between various types of publications (i.e. peer reviewed high quality journals versus industry white papers versus opinion pieces (such as this one!). Media likes a debate and so seeks contrarian views, or only reports views from one side of the argument. Sometimes contrarian views can be illuminating, sometimes they are simply an individual prosecuting an agenda. Increasingly, the loss of specialized science journalism means the media often lack the capacity to tease out the legitimacy of something leading to a nice headline and this can lead to the conveyance of a false sense of integrity in the message.
The unintended consequences of scientific findings being spurious, or misrepresented are far reaching, persistent and difficult to unwind. With respect to spurious findings, these can conveniently be exploited by those with vested interests to further their own agendas. For example, some of the methodologies deployed in the burgeoning area of climate services may already be erroneously used to support real world business strategy3. Some climate services, such as the Australian Climate Service led by the Bureau of Meteorology, are founded in high quality science, and importantly is not trying to sell you something - they strive to provide the best information they can. This contrasts with some other commercial climate service providers that are selling you a product and decline to fully disclose the uncertainties for fear of losing out to another competitor’s overstated climate prediction capabilities.
Failure by some to separate scientifically rigorous from visually appealing, or failing to understand the difference between accuracy and precision risks an erosion of trust in what researchers can provide in the context of climate risk.
This erosion of trust can lead to the risk of being desensitized to ‘doomsday predictions’ and prevent real, required and urgent action to address climate change, as well as increasing risks of ‘fake news’ or social media being relied upon as a credible alternative information source to the peer reviewed literature.
Another critical issue manifests in the development of misinformed policy. In fact, an unfortunate precedent was established when the IMF (and others) turned to the latest research from influential academic economists including Kenneth Rognoff and Carmen Reinhart during the Euro zone debt crisis around 2011 - 2012. Rognoff and Reinhart advocated for more austerity to avoid higher debt ratios which ostensibly would lead to further GDP decline. However, it then transpired that the findings of the now infamous paper "Growth in a Time of Debt"4 was critically compromised by an Excel spreadsheet error, thus invalidating the raison d'être for austerity. More recently, and within the climate context, physical climate risk stress tests conducted around the world have mostly returned benign economic impacts even under the most severe of climate scenarios, something highly likely to be erroneous5. This has been linked to the prescription of physical risk damage functions that are detached from reality and poses a grave risk as it could further delay climate inaction and collectively lull society as a whole into a false sense of security6.
There are several ways to address the issue at hand, with preventative measures favored over post-publication interventions. For example, specific to the lack of cross-disciplinary expertise within the peer review process; journal editors need support to encourage them to seek expertise from different disciplines to unlock access to expertise from different knowledge pools. There could also be the idea of twin/paired reviewers rather than individuals - prompting expert reviewers to better follow the flow of logic and test if insights generated in one discipline but being applied in another are indeed 'fit for purpose'. To encourage broader engagement, ‘crowd-sourcing’ methods such as releasing pre-prints to give the public and other interested parties a chance to provide feedback should be considered, particularly in the domain of applied research with high impact, contentious results - such as geographical areas deemed ‘unlivable’ by a certain time in the future due to climate change.
While the publication of factually incorrect or erroneous research should be (and almost always is) prevented in the first instance, the review process will never be completely fool-proof. As such, to ensure full accountability, factually or methodologically incorrect papers uncovered after publication should be retracted from literature - something that is implemented in some journals, but not in all.
For the media, upholding high standards of science reporting requires a mindset shift. Instead of fixating on ‘breaking news’ and article quantity, adopting more of an investigative journalism approach when covering new research would result in more nuanced narratives that reduce the risk of oversimplifying and misrepresenting the science. Other ideas include co-writing with academics (such as ‘The Conversation’) or having either in-house or a network of reviewer academics to comment on the veracity of new findings will help to boost credibility in lay reporting. Partnerships between business and researchers and co-authorship of reports is becoming increasingly common and is to be encouraged but remember that these reports are rarely truly independently peer-reviewed. We acknowledge that it is likely naive to think the mainstream media will move back to employing scientifically well informed journalists, setting deadlines that allow systematic fact checking and so on. However, unless they do, readers should actively treat mainstream media coverage of climate risk with a suitable degree of skepticism.
As alluded to earlier, the onus of overcoming the science communication gap does not rest solely with researchers and the media. There needs to be broader awareness that a peer-reviewed journal does not mean the paper is correct, rather that it is a single paper that adds to the body of knowledge that needs to be assessed to develop a good understanding. At the same time, awareness that a report from a business is not independently peer reviewed to the same standard as scientific journals can be helpful. We acknowledge that detecting mis-information, whether in a science paper, a report or in the media can be a challenge to those without the relevant subject matter expertise. Broadly, researchers that have been around a while tend to develop an eye for mis-representation of results, a good sense of when something is published that is inconsistent with a body of knowledge, and have a deep understanding of tools and techniques that have withstood the test of time within a discipline. Some businesses and policy units are setting up expert panels, relationships with teams of researchers and so on to act as advisory groups in specific areas. Where this is not possible in-house, developing a strategic relationship with appropriate groups are a good alternative to ensure business and policy decisions are grounded in robust science rather than greenwash.
1https://www.smh.com.au/politics/federal/trillion-dollar-cost-of-climate-change-will-challenge-federal-budget-20230818-p5dxji.html
2https://www.theguardian.com/australia-news/2022/nov/26/australias-unraveling-climate-risk-leaving-more-homes-uninsurable-against-flooding-expert-warns
3 Pitman A. et. al, Treating climate uncertainties as knowable risks - a recipe for greenwash, Energetics, 2022
4 Reinhart and Rognoff, Growth in a Time of Debt, American Economic Review, 2010
5 https://theconversation.com/have-some-economists-severely-underestimated-the-financial-hit-from-climate-change-recent-evidence-suggests-yes-214579
6 Pui and Werner, Financial Risks of Climate Change: Piranhas or Red Herrings? UNSW Press, 2023
