Review of the Methods

An independent expert in evidence synthesis methods was appointed to lead the development of ‘fit for purpose’ methods for the 2022 Scientific Consensus Statement. Advice on the draft Methods was provided by a Methods Working Group. The draft synthesis methods were formally peer reviewed by three independent evidence synthesis experts working in environmental policy/management including two experts from international organisations.

Review of the primary 2022 Scientific Consensus Statement outputs

Editorial Board

Based on advice from Australia’s Chief Scientist and the Reef Water Quality Independent Science Panel (ISP), a formal independent Editorial Board was established to run the peer review process for the three primary outputs of the 2022 Scientific Consensus Statement, specifically, the 30 syntheses of evidence, Summary document and Conclusions. Prospective Editorial Board members were primarily identified by searching the Editorial Boards of relevant scientific journals and were invited to apply to join the 2022 Scientific Consensus Statement Editorial Board. Applicants were screened against pre-determined selection criteria and assessed for potential conflicts of interest prior to their appointment. Following this process, seven Editorial Board members were confirmed, one Editor-in-Chief to oversee the process, and six Editors who had considerable editorial experience with indexed scientific journals from a variety of relevant disciplines.

Peer review of the syntheses of evidence

The formal peer review of the 30 syntheses of evidence involved 63 external and independent expert reviewers. Each question had a minimum of two reviewers, one with Great Barrier Reef-relevant expertise, and a second ‘external’ reviewer (i.e., international or from elsewhere in Australia). Three questions were identified as potentially contentious, and these questions were assigned three reviewers. Reviewers completed a structured peer review form which included questions about the quality, rigour and content of the synthesis, and provided a recommendation (i.e. accept, minor revisions, major revisions). A Lead and Second Editor endorsed the synthesis once question authors had adequately addressed peer reviewer feedback. Lead and Second Editors were also asked to provide assurance that there was a clear line of sight between the body of evidence and the higher-level Evidence Statement to ensure that all Evidence Statements were supported by the evidence base. The Editorial Board then collectively checked the Evidence Statements for use of non-technical language, clarity and for any inconsistencies among questions. Once satisfied with the Evidence Statements, the Editorial Board formally signed off the synthesis of evidence.

Eminent review of the 2022 Scientific Consensus Statement Conclusions and Summary

In addition to the 30 syntheses of evidence, the Editorial Board was also responsible for managing the peer review process for the Summary and Conclusions outputs. These two outputs were derived from the evidence base provided by the syntheses of evidence, with the Summary containing Theme-level insights, and the Conclusions containing high-level conclusions that covered the breadth of the evidence base. Given the diversity of topics that were covered in these two outputs, eminent reviewers were required who could bring considerable expertise and high-level experience to determine whether the products were fit for purpose. The Editorial Board developed selection criteria and identified an initial list of potential eminent experts. A meeting was subsequently held between the Editor-in-Chief and Australia’s Chief Scientist to discuss potential eminent reviewers and finalise the list of candidates. Given the importance of the role of eminent reviewers, Australia’s Chief Scientist approached prospective candidates to gauge their interest. Interested candidates were screened for conflicts of interest prior to their formal appointment. Three eminent reviewers were appointed, one with experience of Great Barrier Reef issues, one national and working outside of the Great Barrier Reef, and a third international reviewer with an agricultural background. The Scientific Consensus Statement’s Coordination Team was responsible for addressing the reviews in a similar manner to the synthesis of evidence review process. The Editor-in-Chief and a supporting Editor checked the preliminary revisions, and once satisfied, these documents were shared with the question authors for formal endorsement. Following this step, the Editorial Board met to discuss the eminent review process and formally sign off this stage.