In Metaphysics, Aristotle declared that there are many things with a plurality of parts which ‘are not merely a complete aggregate but instead some kind of whole beyond its parts’. The French post-impressionist painter Georges Seurat expressed a similar sentiment with his painting technique ‘Pointillism’, in which monochromatic dots of pure colour are placed beside one another in order to either heighten or diminish each other’s intensity, producing a more vibrant image. Both Aristotle’s theory of the universe and Seurat’s vision of painting offer a principle that has endured through the ages. Disparate elements when fused together and surveyed collectively provide us with a greater understanding than if those parts are viewed separately, and this is nowhere more evident than in the process by which systematic reviews are carried out.
Undergirding this process is the knowledge that researchers are expected to strive exhaustively in their effort to gather a number of studies and research findings before rigorously assessing them for their quality and then presenting the conclusions in a reasoned, fair, and impartial summary. In surveying a vast sum of separate studies, researchers seek to arrive at the question of clinical effectiveness regarding a particular medical research question, striving to either illuminate knowledge gaps in the particular area of interest, or to indicate where the path forward lies. Governed by a peer-reviewed protocol and a process that is entirely transparent, systematic reviews aim to be repeatable in their conclusions, thus transcending the limitations of traditional reviews. They provide us with a reliable look at the research area in question at this moment in time, granting experts a vital tool for their arsenal when making an informed judgement on a particular healthcare intervention.
Elliott M. Antman et al. describe in their paper ‘A Comparison of Results of Meta-analyses of Randomized Control Trials and Recommendations of Clinical Experts. Treatments for Myocardial Infarction’, published in JAMA in 1992, how there were ‘thrombolytic drugs that did not begin to be recommended even for specific indications by more than half the experts until 13 years after they could have been shown to be effective’ (Antman, E. et al, 1992) due largely to the fact that there had been a ‘tendency to ignore the thrombolytics that had been proven to reduce mortality and to enthuse about lidocaine that had not’ (Antman, E. et al, 1992). In response to instances such as this, where particular scientific viewpoints were being pushed without the backing of sufficient evidence at the expense of a more dependable alternative, systematic reviews were introduced and became essential aspects of the scientific literature so as to provide those in the particular field with a reliable overview of the evidence with regards to the healthcare intervention in question.
This is further evident in the case of a particular paper titled ‘Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children’, by Wakefield et al. Published in 1998, the authors implied that the MMR vaccine may be associated with ‘gastrointestinal disease and developmental regression in a group of previously normal children’ (Wakefield, A.J. et al, 1998). As a widespread panic ensued, 10 of the original 12 authors retracted the paper, while the study ‘received wide publicity, and MMR vaccination rates began to drop because parents were concerned about the risk of autism after vaccination’ (Sathyanarayana Rao, T.S. & Chittaranjan, A., 2011). In response to public concern, an incredibly thorough systematic review was carried out by Demicheli et al., published originally in 2005. Assessing 64 studies for their methodological quality and risk of bias, researchers arrived at the conclusion that ‘no significant association could be assessed between MMR immunisation and the following conditions: autism, asthma, leukaemia, hay fever, type 1 diabetes, gait disturbance, Crohn’s disease, demyelinating diseases, bacterial or viral infections’ (Demicheli, V. et al, 2005). With this conclusion, the importance of systematic reviews is underlined once again – they help dispel unreliable evidence while providing an informed overview of the issue, clarifying any concerns that either the public or medical professionals may harbour.
Following the rigour of Demicheli et al., all researchers of systematic reviews are likewise expected to be thorough in selecting and interrogating studies, evaluating their merits and translating the findings into an informed review. All forms of publication bias and ‘cherry-picking’ are to be avoided, certifying that the work they have produced is a fair and reliable assessment of the research evidence. In the pursuit of reliability, researchers ought to explicate their search strategy for finding studies; their process for selecting studies to use in their review; their method of extracting data from reports; their principal summary measures; their methods of handling data and combining the results of studies; and their assessment of the risk of bias that may affect the integrity of the evidence. To ensure the high standard of systematic reviews, a stringent checklist must be followed such as the one here, provided by PRISMA.
As an organisation at the forefront of child psychology, psychiatry, and mental health, we ask all researchers to submit their systematic reviews to either one of ACAMH’s journals – Child and Adolescent Mental Health (CAMH) and the Journal of Child Psychology and Psychiatry (JCPP). We consistently publish systematic reviews, such as this one featured in JCPP in January 2019, which focuses on the test-retest reliability in published studies on standardized diagnostic interviews (SDI). To ensure the rigorous standard and high quality of systematic reviews, both of our journals require authors to conform to the guidelines set out in the PRISMA Statement and strongly encourage the pre-registration of review protocols on publicly accessible platforms.
Bibliography
- Antman, E., Lau, J., Kupelnick, B., et al. (1992). A Comparison of Results of Meta-analyses of Randomized Control Trials and Recommendations of Clinical Experts. Treatments for Myocardial Infarction. JAMA, 268(2).
- Wakefield, A. J., Murch, S. H., Anthony, A., et al. (1998). Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. Lancet, 351(9103). doi:https://doi.org/10.1016/S0140-6736(97)11096-0.
- Sathyanarayana Rao, T. S., Chittaranjan, A. (2011). The MMR vaccine and autism: Sensation, refutation, retraction, and fraud. Indian Journal of Psychiatry, 53(2), pp. 95-96. doi:10.4103/0019-5545.82529.
- Demicheli, V., Rivetti, A., Debalini, M. G., Di Pietrantonj, C. (2005). Vaccines for measles, mumps and rubella in children. Evidence-Based Child Health: A Cochrane Review Journal, 8(6). doi:https://doi.org/10.1002/ebch.1948
