What is Systematic Review?
A systematic review is a comprehensive, structured analysis of existing research on a specific topic. It uses predefined criteria to identify, evaluate, and synthesize relevant studies, aiming to provide an unbiased summary of the current evidence.
The explicit and systematic approach of a systematic review distinguishes it from traditional reviews and commentaries.
How systematic reviews differ from narrative reviews:
Systematic reviews are time-intensive and need a research team with multiple skills and contributions. There are some cases where systematic reviews are unable to meet the necessary objectives of the review question.
In these cases,scoping reviews(which are sometimes called scoping exercises/scoping studies) may be more useful to consider.
Scoping reviews are different from systematic reviewsbecause they may not include a mandatory critical appraisal of the included studies or synthesize the findings from individual studies.
Assessing The Need For A Systematic Review
When assessing the need for a systematic review, one must first check if any existing or ongoing reviews already exist and determine if a new review is justified.
Scoping reviews frequently serve as preliminary steps before conducting full systematic reviews. They help assess the available literature’s breadth, identify key concepts, and determine the feasibility of a more comprehensive review.
This initial exploration guides researchers in refining their approach for subsequent in-depth analyses.
Resources to consider searching include:
If an existing review addressing the question of interest is found, its quality should be assessed to determine its suitability for guiding policy and practice.
If a high-quality, relevant review is located, but its completion date is some time ago, updating the review might be warranted.
Assessing current relevance is vital, especially in rapidly evolving research fields. Collaboration with the original research team might be beneficial during the update process, as they could provide access to their data.
If the review is deemed to be of adequate quality and remains relevant, undertaking another systematic review may not be necessary.
When a new systematic review or an update is deemed necessary, the subsequent step involves establishing a review team and potentially an advisory group, who will then develop the review protocol.
How To Conduct A Systematic Review
PRISMA(Preferred Reporting Items for Systematic Reviews and Meta-Analyses) is a reporting guideline designed to improve the transparency and completeness of systematic review reporting.
PRISMA was created to tackle the issue of inadequate reporting often found in systematic reviews:
Step 1: write a research protocol
A protocol in the context of systematic reviews is a detailed plan that outlines the methodology to be employed throughout the review process.
The protocol serves as a roadmap, guiding researchers through each stage of the review in a transparent and replicable manner.
This document should provide specific details about every stage of the research process, including the methodology for identifying, selecting, and analyzing relevant studies.
For example, the protocol should specify search strategies for relevant studies, including whether the search will encompass unpublished works.
The protocol should be created before beginning the research process to ensure transparency and reproducibility.
This pre-determined plan ensures that decisions made during the review are objective and free from bias, as they are based on pre-established criteria.
Protocol modifications are sometimes necessary during systematic reviews. While adhering to the protocol is crucial for minimizing bias, there are instances where modifications are justified. For instance, a deeper understanding of the research question that emerges from examining primary research might necessitate changes to the protocol.
Systematic reviews should be registered at inception (at the protocol stage) for these reasons:
This registration prevents duplication (research waste) and makes the process easy when the full systematic review is sent for publication.
PROSPEROis an international database of prospectively registered systematic reviews in health and social care. Non-Cochrane protocols should be registered on PROSPERO.
Research Protocol
Citation
Rasika Jayasekara, Nicholas Procter. The effects of cognitive behaviour therapy for major depression in older adults: a systematic review. PROSPERO 2012 CRD42012003151 Available from:https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42012003151
Review question
How effective is CBT compared with other interventions, placebo or standard treatment in achieving relapse prevention and improving mental status for older adults with major depression?
Searches
The search strategy aims to find both published and unpublished studies and publications. The search will be limited to English language papers published from 2002 to 2012.
A three-step search strategy will be developed using MeSH terminology and keywords to ensure that all materials relevant to the review are captured.
Thirdly, the reference list of all identified reports and articles will be searched for additional studies.
The databases to be searched included:
The search for unpublished studies will include:
Experts in the field will be contacted for ongoing and unpublished trials. Experts will be identified through journal publications.
Types of study to be included
All randomised controlled trials (RCTs) assessing the effectiveness of CBT as a treatment for older adults with major depression when compared to standard care, specific medication, other therapies and no intervention will be considered.
In the absence of RCTs, other research designs such as quasi-experimental studies, case-controlled studies and cohort studies will be examined. However, descriptive studies and expert opinion will be excluded.
Condition or domain being studied
Major depression is diagnosed according to DSM IV or ICD 10 criteria.
The trials including participants with an explicit diagnosis of dementia or Parkinson’s disease and other mental illnesses will be excluded.
The review will include trials conducted in primary, secondary, community, nursing homes and in-patient settings.
Participants/population
The review will include trials in which patients are described as elderly, geriatric, or older adults, or in which all patients will be aged 55 or over (many North American trials of older adult populations use a cut-off of 55 years).
The review will include trials with subjects of either sex. Where possible, participants will be categorised as community or long term care residents.
Intervention(s), exposure(s)
The review will focus on interventions designed to assess the effects of CBT for older adults with major depression.
The label cognitive behavioural therapy has been applied to a variety of interventions and, accordingly, it is difficult to provide a single, unambiguous definition.
In order to be classified as CBT the intervention must clearly demonstrate the following components:
In addition, all therapies that do not meet these criteria (or that provide insufficient information) but are labelled as ‘CBT’ or ‘Cognitive Therapy’ will be included as ‘less well defined’ CBT.
Comparator(s)/control
other interventions, placebo or standard treatment
Main outcome(s)
Primary outcomes
Measures of effect
The review will categorise outcomes into those measured in the shorter term (within 12 weeks of the onset of therapy), medium term (within 13 to 26 weeks of the onset of therapy) and longer term (over 26 weeks since the onset of therapy).
Additional outcome(s)
Secondary outcomes
Data extraction (selection and coding)
Data will be extracted from papers included in the review using JBI-MAStARI. In this stage, any relevant studies will be extracted in relation to their population, interventions, study methods and outcomes.
Where data are missing or unclear, authors will be contacted to obtain information.
Risk of bias (quality) assessment
Since the review will evaluate the experimental studies only, The Joanna Briggs Institute Meta Analysis of Statistics Assessment and Review Instrument (JBI-MAStARI) will be used to evaluate each study’s methodological validity.
If there is a disagreement between the two reviewers, there will be a discussion with the third reviewer to solve the dissimilarity.
Strategy for data synthesis
Where possible quantitative research study results will be pooled in statistical meta-analysis using Review Manager Software from the Cochrane Collaboration.
Odds ratio (for categorical outcome data) or standardised mean differences (for continuous data) and their 95% confidence intervals will be calculated for each study.
Heterogeneity will be assessed using the standard Chi-square. Where statistical pooling is not possible the findings will be presented in narrative form.
Step 2: formulate a research question
Developing a focused research question is crucial for a systematic review, as it underpins every stage of the review process.
The question defines the review’s nature and scope, guides the identification of relevant studies, and shapes the data extraction and synthesis processes.
It’s essential that the research question is answerable and clearly stated in the review protocol, ensuring that the review’s boundaries are well-defined.
A narrow question may limit the number of relevant studies and generalizability, while a broad question can make it challenging to reach specific conclusions.
PICO Framework
The PICO framework is a model for creating focused clinical research questions. The acronym PICO stands for:
Using the PICO format when designing research helps to minimize bias because the questions and methods of the review are formulated before reviewing any literature.
The PICO elements are also helpful in defining the inclusion criteria used to select sources for the systematic review.
The PICO framework is commonly employed in systematic reviews that primarily analyze data fromrandomized controlled trials.
Not every element of PICO is required for every research question. For instance, it is not always necessary to have a comparison
Types of questions that can be answered using PICO:
“In patients with a recent acute stroke (less than 6 weeks) with reduced mobility (P), is any specific physiotherapy approach (I) more beneficial than no physiotherapy (C) at improving independence in activities of daily living and gait speed (O)?
“For women who have experienced domestic violence (P), how effective are advocacy programmes (I) compared to other treatments (C) on improving the quality of life (O)?”
Are women with a history of pelvic inflammatory disease (PID) (P) at higher risk for gynecological cancers (O) than women with no history of PID (C)?
Among asymptomatic adults at low risk of colon cancer (P), is fecal immunochemical testing (FIT) (I) as sensitive and specific for diagnosing colon cancer (O) as colonoscopy (C)?
Among adults with pneumonia (P), do those with chronic kidney disease (CKD) (I) have a higher mortality rate (O) than those without CKD (C)?
Alternative Frameworks
Step 3: Search Strategy
PRISMA(Preferred Reporting Items for Systematic reviews and Meta-Analyses) provide appropriate guidance for reporting quantitative literature searches.
Present the full search strategies for all databases, registers and websites, including any filters and limits used.
PRISMA 2020 Checklist
A search strategy is a comprehensive and reproducible plan for identifying all relevant research studies that address a specific research question.
This systematic approach to searching helps minimize bias and distinguishes systematic reviews from other types of literature reviews.
It’s important to be transparent about the search strategy and document all decisions for auditability. The goal is to identify all potentially relevant studies for consideration.
Here’s a breakdown of a search strategy:
Search String Construction
It is recommended to consult topic experts on the review team and advisory board in order to create as complete a list of search terms as possible for each concept.
To retrieve the most relevant results, a search string is used. This string is made up of:
Information Sources
The primary goal is to find all published and unpublished studies that meet the predefined criteria of the research question. This includes considering various sources beyond typical databases
Information sources for systematic reviews can include a wide range of resources like scholarly databases, unpublished literature, conference papers, books, and even expert consultations.
Specify all databases, registers, websites, organisations, reference lists and other sources searched or consulted to identify studies. Specify the date when each source was last searched or consulted.
An exhaustive, systematic search strategy is developed with the assistance of an expert librarian.
It is important to note that this may not be an exhaustive list of all potential databases.
Example:
A systematic computerized search was performed for publications that appeared between 1974 and 2018 in English language journals. Four databases were searched including PsychINFO, Embase, OVOID MEDLINE, and AMED. The databases were searched with combinations of search terms relating to attachment (“attachment” OR “working model” OR “safe haven” OR “secure base” OR “felt security”) AND romantic couples (“dyad” OR “couple” OR “spous” OR “partner” OR “romantic” OR “wife” OR “husband” OR “close relationship” OR “interpersonal” OR “intimate” OR “mari”) AND social support (“support prov” OR “caregiving” OR “support giv” OR “social support” OR “enacted support” OR “support received” OR “receiv* support” OR “prov support” OR “dyadic coping” OR “interpersonal coping” OR “collaborative coping” OR “help‐seeking” OR “emotional support” OR “tangible support” OR “instrumental support” OR “perceived support” OR “responsive” OR “buffer” OR “partner support” OR “Support avail*” OR “available support”). The reference lists of the retrieved studies were checked to find other relevant publications, which were not identified in the computerized database searches.
Inclusion Criteria
Specify the inclusion and exclusion criteria for the review. PRISMA 2020 Checklist
Before beginning the literature search, researchers should establish clear eligibility criteria for study inclusion.
Inclusion criteria are used to select studies for a systematic review and should be based on the study’s research method and PICO elements.
To maintain transparency and minimize bias, eligibility criteria for study inclusion should be established a priori. Ideally, researchers should aim to include only high-qualityrandomized controlled trialsthat adhere to the intention-to-treat principle.
The selection of studies should not be arbitrary, and the rationale behind inclusion and exclusion criteria should be clearly articulated in the research protocol.
Studies that met the following criteria were included: (a) empirical studies of couples (of any gender) who are in a committed romantic relationship, whether married or not; (b) measurement of the association between adult attachment and support in the context of this relationship; (c) the article was a full report published in English; and (d) the articles were reports of empirical studies published in peer‐reviewed journals, dissertations, review papers, and conference presentations.
Iterative Process
The iterative nature of developing a search strategy for systematic reviews stems from the need to refine and adapt the search process based on the information encountered at each stage.
A single attempt rarely yields the perfect final strategy. Instead, it is an evolving process involving a series of test searches, analysis of results, and discussions among the review team.
Here’s how the iterative process unfolds:
The iterative nature of developing a search strategy is crucial for ensuring that the systematic review is comprehensive and unbiased.
By constantly refining the search strategy based on the results and feedback, researchers can be more confident that they have identified all relevant studies.
This iterative process ensures that the applied search strategy is sensitive enough to capture all relevant studies while maintaining a manageable scope.
Throughout this process, meticulous documentation of the search strategy, including any modifications, is crucial for transparency and future replication of the systematic review.
Step 4: Search the Literature
Conduct a systematic search of the literature using clearly defined search terms and databases.
Applying the search strategy involves entering the constructed search strings into the respective databases’ search interfaces. These search strings, crafted using Boolean operators, truncation symbols, wildcards, and database-specific syntax, aim to retrieve all potentially relevant studies addressing the research question.
The researcher, during this stage, interacts with the database’s features to refine the search and manage the retrieved results.
Applying the search strategy is not merely a mechanical process of inputting terms; it demands a thorough understanding of database functionalities and a discerning eye to adjust the search based on the nature of retrieved results.
Step 5: screening and selecting research articles
Once the search strategy is finalized, it is applied to the selected databases, yielding a set of search results.
These search results are then screened against pre-defined inclusion criteria to determine their eligibility for inclusion in the review.
The goal is to identify studies that are both relevant to the research question and of sufficient quality to contribute to a meaningful synthesis.
Studies meeting the inclusion criteria are usually saved into electronic databases, such asEndnoteorMendeley, and include title, authors, date and publication journal along with an abstract (if available).
Study Selection
Specify the methods used to decide whether a study met the inclusion criteria of the review, including how many reviewers screened each record and each report retrieved, whether they worked independently, and if applicable, details of automation tools used in the process.
The selection process in a systematic review involves multiple reviewers to ensure rigor and reliability.
To minimize bias and enhance the reliability of the study selection process, it is recommended that at least two reviewers independently assess the eligibility of each study. This independent assessment helps reduce the impact of individual biases or errors in judgment.
First, the search results from separate databases were combined, and any duplicates were removed. The lead author (S. M.) and a postgraduate researcher (F. N.) applied the described inclusion criteria in a standardized manner. First, both the titles and abstracts of the articles were evaluated for relevance. If, on the basis of the title and/or abstract, the study looked likely to meet inclusion criteria hard copies of the manuscripts were obtained. If there was doubt about the suitability of an article, then the manuscript was included in the next step. The remaining articles were obtained for full‐text review, and the method and results sections were read to examine whether the article fitted the inclusion criteria. If there was doubt about the suitability of the manuscripts during this phase, then this article was discussed with another author (C. H.). Finally, the reference lists of the eligible articles were checked for additional relevant articles not identified during the computerized search. For the selected articles (n = 43), the results regarding the relationship between attachment and support were included in this review (see Figure 1, for PRISMA flowchart).
PRISMA Flowchart
ThePRISMA flowchart isa visual representation of the study selection process within a systematic review.
This systematic and transparent approach, as visualized in the PRISMA flowchart, ensures a robust and unbiased selection process, enhancing the reliability of the systematic review’s findings.
The flowchart serves as a visual record of the decisions made during the study selection process, allowing readers to assess the rigor and comprehensiveness of the review.
Source: McLeod, S., Berry, K., Hodgson, C., & Wearden, A. (2020). Attachment and social support in romantic dyads: A systematic review.Journal of clinical psychology,76(1), 59-101.
Step 6: Criticallay Appraising the Quality of Included Studies
Quality assessment provides a measure of the strength of the evidence presented in a review.
High-quality studies with rigorous methodologies contribute to a more robust and reliable evidence base, increasing confidence in the review’s conclusions.
Conversely, including low-quality studies with methodological weaknesses can undermine the review’s findings and potentially lead to inaccurate recommendations.
To judge the quality of studies included in a systematic review, standardized instruments, such as checklists and scales, are commonly used. These tools help to ensure a transparent and reproducible assessment process.
The choice of tool should be justified and aligned with the study design and the level of detail required. Using quality scores alone is discouraged; instead, individual aspects of methodological quality should be considered.
Evidence Tables
Aspects of the appraisal of studies included in the review should be recorded as evidence tables (NICE 2009): simple text tables where the design and scope of studies are summarised.
The reader of the review can use the evidence tables to check the details, and assess the credibility and generalisability of findings, of particular studies.
Critical appraisal of the quality of included studies may be combined with data extraction tables.
Step 7: extracting data from studies
To effectively extract data from studies that meet your systematic review’s inclusion criteria, you should follow a structured process that ensures accuracy, consistency, and minimizes bias.
1. Develop a data extraction form:
2. Extract the data:
3. Dual independent review:
By following these steps, you can effectively extract data from studies that meet your inclusion criteria, forming a solid foundation for the analysis and synthesis phases of your systematic review.
Step 8: synthesize the extracted data
The key element of a systematic review is the synthesis: that is the process that brings together the findings from the set of included studies in order to draw conclusions based on the body of evidence.
Data synthesis in a systematic review involves collating, combining, and summarizing findings from the included studies.
The data synthesis will be presented in theresults sectionof the systematic review.
Identifying patterns, trends, and differences across studies
Narrative synthesis uses a textual approach to analyze relationships within and between studies to provide an overall assessment of the evidence’s robustness. All systematic reviews should incorporate elements of narrative synthesis, such as tables and text.
Source: Head, B. A., Schapmire, T. J., & Zheng, Y. (2017). Telehealth in palliative care: a systematic review of patient-reported outcomes.Journal of Hospice & Palliative Nursing,19(2), 130-139.
Step 9: discussion section and conclusion
Minimizing Bias
To reduce bias in a systematic review, it is crucial to establish a systematic and transparent review process that minimizes bias at every stage. Sources provide insights into strategies and methods to achieve this goal.
Reading List
Olivia Guy-Evans, MSc
BSc (Hons) Psychology, MSc Psychology of Education
Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.
Saul McLeod, PhD
BSc (Hons) Psychology, MRes, PhD, University of Manchester
Saul McLeod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.
