Living systematic reviews: towards real-time evidence for health-care decision making
Systematic reviews aim to provide an accurate summary of available evidence for specific health questions. In practice, an increase in methodological expectations and an increasing deluge of primary studies challenges the ability of many review teams to produce timely, high-quality systematic reviews and to keep them up to date. Only a minority of reviews are updated within two years and as new research is published in the intervening period, these delays lead to significant inaccuracy. One estimate is that 7% of systematic reviews are inaccurate the day they are published and after two years 23% of reviews that are not updated will present incorrect conclusions. The difficulties faced by review teams in keeping reviews up to date leads to considerable inaccuracy and to some extent undermines the value created through the use of rigorous methods.
Living systematic review (LSR) is an emerging approach to the updating of systematic reviews in which the review is updated frequently, typically at least each month, and usually published as online-only systematic reviews. New processes and technologies can help to keep review findings constantly up to date, even in fast-moving research areas. These include the use of text mining, crowdsourcing, and linked data to enable more efficient evidence surveillance and review production. A number of LSR pilot projects (Badgett et al, Cnossen et al, Synnot et al) have demonstrated the feasibility of this approach for intervention and other review types and its application in network meta-analysis has been proposed.
The key areas in which LSRs differ from conventionally updated systematic reviews are team and workflow management, meta-analysis, and publication. Instead of the intense, sporadic effort of conventional SRs and SR updates, LSRs require a continuous workflow, with a moderate amount of effort coordinated over long periods of time, and gradual evolution in the review team. The updating of a review often leads to repeated meta-analysis, which may increase the rate of false-positive findings. Sequential methods can control this risk, but are controversial. Alternatively, a Bayesian approach has also been proposed. These issues apply in all updates, but are particularly relevant in LSRs given the frequency of updating. The publication of LSRs also requires some adaptation of existing norms. When a search does not identify any new studies for inclusion only the published search date need be updated. If new studies are identified, a new publication is generally required with a new digital object identifier (DOI), bibliographic database listing, and citation.
This approach to systematic review updating also benefits other evidence processes, particularly guideline development and decision support systems. The supply of up-to-date evidence from LSRs enables living recommendations and decision support rules but, more significantly, it helps to create an environment for more dynamic, interlinked synthesis and use of the data generated by health research.
Authors: Chris Mavergames and Julian Elliott
These views are our own and do not represent the views of our employer, Cochrane.
Chris Mavergames is Head of Informatics and Knowledge Management for The Cochrane Collaboration. He leads on Cochrane’s technology and knowledge management infrastructure, including the Cochrane Linked Data Project, and provides vision and leadership for Cochrane’s emerging technology strategy to 2020.
Julian Elliott is Senior Research Fellow at the Australasian Cochrane Centre and Head of Clinical Research in the Department of Infectious Diseases, Alfred Hospital and Monash University. He is leading Cochrane’s development of new evidence systems, including Project Transform, a major Cochrane project that is using new technologies and processes to improve the production of systematic reviews. He is also the co-founder and CEO of Covidence, a nonprofit online platform for efficient systematic review production.