This paper examines what it means to conduct a systematic review, given that such reviews are rapidly becoming the ‘gold standard’ in evidence synthesis, and draws insights from a recent World Bank report.
Systematic reviews are becoming a widely accepted ‘gold standard’ in evidence synthesis for evidence-based and – informed policy and practice. Many organizations exist to coordinate the registration, conduct and publication of systematic reviews across a range of disciplines, including medicine, international development, and environmental management and biodiversity conservation.
As the term ‘systematic review’ becomes more widely recognized, however, there is a risk that stakeholders may have only partial understanding of the rigorous methods required to produce a reliable systematic review. This article highlights one such example from the field of education and international development, where a World Bank report claimed to ‘systematically review’ six ‘systematic reviews’ that found divergent results.
The article critically appraises the six included reviews and the World Bank report itself using an a priori quality assessment tool. The analysis shows that none of the six included reviews are classifiable as systematic reviews according to widely accepted criteria. It also finds that the World Bank report failed to use true systematic review methods to synthesize the included reviews findings.
The study demonstrates the risks associated with partial understanding of the added value associated with systematic reviews and highlights a need for improved awareness of what systematic reviews are.
Design and development by Soapbox.