meta analyze

• Meta-Analysis

  An Analysis of Analyses
• Meta-analysis

  The systematic consolidation of individual studies, also known as primary studies, that focus on the same research question
• Meta-analysis


  • It involves the quantitative accumulation and aggregation of individual studies
• Objectives / Functions of Meta-Analysis

  • Describing a field of research (systematic overview)
  • Describing (causal) relationships (with longitudinal or experimental data)
  • Development and testing of theories
• Meta-analyses are necessary for statistical control of heterogeneous individual results from single studies and to provide an overview of specific research questions with many publications
• Steps Involved in a Meta-Analysis

  1. Defining the Research Question

  2. Literature Review

  3. Selection of Primary Studies

  4. Coding of the Study Characteristics

  5. Computing the Effect Sizes

  6. Testing for Homogeneity

  7. Summarizing the Effect Parameters

  8. Testing for Moderators, Bias

  9. Interpretation of the Effect Sizes
• Study Scope


  Balancing the manageability and relevance of literature when there is an exponential growth in the number of publications in many research fields
• Existing Meta-Analyses

  The number of meta-analyses in management research has grown significantly, so it's likely that one or several meta-analyses on many topics of high scholarly interest already exist
• Selection of Primary Studies

  1. Utilize multiple search strategies

  2. Review the literature cited in other relevant studies

  3. Browse different databases

  4. Consider additional sources (grey literature)
• Selection of Studies

  1. Consideration of All Discoverable Studies

  2. Justified Selection Based on Criteria (availability of relevant statistical information, minimum study quality, comparability of variables)
• Avoiding studies with methodological flaws is important in the selection of studies for meta-analysis
• Coding of the Study Characteristics

  Noting or assessing specific study characteristics like sample size and other moderator variables of interest
• Computing the Effect Sizes and Weighting Factors


  1. Calculation of Effect Sizes in Primary Studies

  2. Calculation of a Weighting Factor for Each Primary Study

  3. Estimation of the Common Population Effect Size
• Effect Size Using Cohen's d


  Measure of the standardized mean difference between the experimental group and control group
• Effect Size Using r-index

  Pearson Product-Moment Correlation, a general measure of the strength of association between two variables
• It's possible to convert the d-index into r
• Aggregation of Individual Effect Sizes is done to estimate the common population effect size
• d


  Mean difference between the experimental group (e) and control group (c)
• SDpooled

  Pooled standard deviation
• Hedges' g

  Measure that considers sample sizes in its calculation in addition to the mean difference
• Glass' δ

  Measure that is experimentally oriented, only considering the standard deviation of the control group
• 1. index


  Pearson Product-Moment Correlation, a general measure of the strength of association between two variables
• Aggregation of Individual Effect Sizes

  1. Calculate the weighted mean population effect size

  2. Give more weight to studies with larger sample sizes as they are typically more precise and reliable
• Confidence interval

  • Evaluates the statistical significance of the overall effect size
  • A 95% confidence interval means that about 95 of 100 confidence intervals would contain the true mean value
• If the confidence interval around the overall effect size (D) includes 0, the overall effect across all studies is not statistically significant
• Correcting for Artefacts


  1. Correcting for Measurement Reliability

  2. Correcting for Limited Range in Primary Studies
• Homogeneity


  The observed variance in effect sizes is due to sampling error (all primary studies examine the same phenomenon)
• Heterogeneity


  The observed variance is greater and implies differences between the studies
• Testing for Homogeneity


  Use the Q statistic to check if the observed effect sizes show greater variance than would be "expected by chance" (due to sampling errors)
• If the empirical Q (Qemp) is greater than the critical Q (Qkrit), it means the individual studies do not just reflect the same phenomenon
• If Homogeneity is Not Present


  1. Outlier Analysis

  2. Moderator Analysis

  3. Q Random Effects Model
• Fixed Effect Meta-Analysis

  • Assumption: The estimated parameter is the same in all studies, but the studies measure this parameter with varying accuracy
  • Limited generalizability (only to test subjects, not to study designs)
  • Higher power
• Random Effects Meta-Analysis

  • Assumption: The estimated parameter varies across studies
  • Additional influencing factors are specific details in the study procedures
  • Unrestricted generalizability (also to study designs)
  • Lower power, more conservative in assumptions
• Potential issues in Meta-Analysis

  • Comparability of Studies
  • Selection Bias/Publication Bias
  • Quality of Included Studies
  • Poor Documentation in Primary Studies
  • Statistical Dependencies
  • Coding Agreement
• The research question is "Does unemployment lead to diminished mental health?"
• The effect size measure used is Cohen's d, using a random effects model