Systematic Reviews Introduction

• Systematic Reviews:

  • Each individual trial represents an estimate of the true effect of a treatment
  • Seek to identify and synthesise all of the available evidence for the most accurate answer
  • Take a very deliberate systematic approach
  • Allows pooling of results (meta-analysis)
• Systematic Review

  1. Formulation of a specific question to be addressed – (PICO structure)
  2. Retrieval of ALL relevant literature; systematic search, selection of studies based upon pre-defined criteria
  3. Tabulation of study characteristics/assessment of study quality/risk of bias
  4. Synthesis/numerical aggregation of study findings
  5. Hypothesis testing? Presentation of effect size? Narrative synthesis?
• The importance of A PRIORI

  • Design methods before you have seen the data, with a detailed protocol
  • Post hoc provides opportunity for data mining
• PICOS:

  • Participants
  • Demographics, condition, stage of disease etc
  • Intervention
  • Broad or specific, dose, parameters
  • Comparison
  • Explanatory (placebo/sham), pragmatic, comparative efficacy
  • Outcomes
  • categorical or continuous, validated scales, MCID?
  • Study designs
  • randomised controlled trials, maybe include non-randomised?
• 2 Independent reviewers (+1 arbiter) must concurrently:

  • Scrutinize the searches
  • Apply inclusion/exclusion criteria
  • Assess risk of bias/study quality
  • Extract data
• Searching the literature:

  • Electronic databases
  • Handsearching
  • Published / unpublished
  • Grey literature
  • Language?
  • Search strategies
• Cochrane Risk of Bias assessment: From high to unclear to low:

  • Random sequence generation (selection bias)
  • Allocation concealment (selection bias)
  • Blinding of participants and personnel (performance bias)
  • Blinding of outcome assessment (detection bias)
  • Incomplete outcome data addressed (attrition bias)
  • Selective reporting (reporting bias)
• Different types of biases that can be reviewed:

  • Bias arising from the randomisation process
  • Bias due to deviations from intended intervention
  • Bias due to missing outcome data
  • Bias in the measurement of the outcome
  • Bias in selection of the reported results
• Meta-Analysis:

  • Meta-analysis measures size & consistency of treatment effect across more than 1 study
  • By pooling data from studies we can increase the accuracy and power of our estimates
  • We can estimate both the treatment effect and the consistency between studies
• Meta-analysis:

  • Estimates the effect size for each study
  • Weights each study (usually on size & precision)
  • Pools each study to generate one overall effect size
  • Done properly this should be more precise than that from individual studies
• Meta-analysis diagram:

  • the vertical line represent the line of no effect (of treatment)
  • the circles are the point estimates, representing different studies and their average result, the bigger the circle, the larger the study
  • the horizontal lines represent the confidence intervals of each study, the longer the lines, the better the confidence
  • the diamond represents the pooled estimate of effect of all the studies put together to formulate an average

  
• Heterogeneity:

  • trials differ in many ways
  • patients
  • methods
  • treatments
  • doses
  • time points
  • bias
  • it is not always appropriate to pool data if trials are very different
• Heterogeneity:

  • When looking at a meta-analysis ask yourself - were the studies similar enough to justify lumping them together?
  • It depends on the question
  • Did the authors look at whether effects differed between more tightly defined subgroups (through subgroup analysis or “meta-regression”)