ANOVA

• Analysis of Variance (ANOVA)


  A statistical method used to analyze the difference between three or more conditions

  
• Analysis of three or more conditions

  1. Descriptive statistics (means, confidence intervals, medians, standard deviations, graphical illustrations)

  2. Effect size (magnitude of difference between conditions, d and partial eta2)

  3. ANOVA

  
• ANOVA


  • Serves the same purpose as t-tests
  • Tests for differences in group means
  • More flexible than t-tests as it can handle any number of groups
• ANOVA


  • Looks at the different sources of variability in a dataset
  • Uses sum of squares to calculate the sources of variability

  
• ANOVA


  1. Looks to see whether there are differences in the means of the groups

  2. Determines the grand mean and sees how different the group means are from it
• ANOVA


  • Tests whether there is a significant difference between some or all of the means of the conditions by comparing them with the grand mean
  • The t-test generalized to more than two groups, so there is a direct relationship between them

  
• Meaning of ANOVA


  Analyzes the different sources from which variation in the scores arises

  
• Independent ANOVA


  Used when participants perform in only one condition of several (between-participants design)
• Related ANOVA


  Used when the participants perform in all conditions (within-participants design)
• Visualizing the design


  • One-way design: between groups (one factor with three levels, participants only in one level)
  • One-way design: repeated measures (one factor with three levels, participants in all levels)
• Grouping variable


  The predictor or independent variable, made up of k groups

  
• Outcome variable


  The variable on which people differ, that we are trying to explain or account for based on group membership
• Individual group means


  The means for each of the k groups (M1, M2, M3)
• Grand mean (MG)


  The single mean representing the average of all participants across all groups
• Sample sizes (n)


  The number of participants in each of the k groups (n1, n2, n3)
• Variability


  Differences in individual data or group means, can be due to random chance or actual differences
• Between-groups sum of squares


  Calculates the variability arising from the differences between groups

  
• Between-groups variance


  Arises from treatment effects, individual differences, and experimental error
• Within-groups sum of squares

  Calculates the variability arising from differences within each group

  
• Within-groups variance


  Arises from individual differences and experimental error

  
• Total sum of squares


  The total variability in the dataset
• ANOVA table


  Presents the mean square, F statistic, and other key information

  
• F statistic


  The test statistic for ANOVA, a signal to noise ratio that assesses the overall fit of the model
• Partitioning the variance


  ANOVA partitions the total variability into between-groups and within-groups components

  
• ANOVA and t-test


  There is a direct relationship between them, as ANOVA generalizes the t-test to more than two groups
• ANOVA and Type 1 error


  Using multiple t-tests increases the probability of committing a Type 1 error, so ANOVA is preferred
• Hypothesis in ANOVA


  H0: all group means are equal, HA: at least one mean is different
• Balanced vs unbalanced designs


  Balanced has equal sample sizes in all groups, unbalanced has different sample sizes
• Study designs


  • Determine differences between independent groups
  • Determine differences between conditions/treatments with no pre-test
  • Determine differences in change scores
• Assumptions of ANOVA: continuous DV, categorical IV with 2+ groups, independence of observations, no significant outliers, normality, homogeneity of variances
• Homogeneity of variance


  If violated, can use adjusted F statistics like Brown-Forsythe or Welch's F
• ANOVA is not as robust as commonly believed, violations of assumptions can impact power and error rates
• Planned contrasts


  Specific pairwise comparisons to test hypotheses, either simple or complex
• Post hoc tests


  All pairwise comparisons when there were no specific hypotheses, increases familywise error rate
• Nonparametric alternative
• Conduct a sensitivity analysis


  Apply a robust test to check that your conclusion does not change
• Planned contrasts (aka planned comparisons)


  Specific pairwise comparisons
• Planned contrasts


  • Can be specific pairwise comparisons (i.e., not all pairwise comparisons), which are called simple contrasts (e.g., a comparison between the "Moderate" vs. "High" groups)
  • Can represent a comparison that involves more than two groups (e.g., a comparison of the "Moderate" and "Low" groups combined versus the "High" group) and are called complex contrasts
• Planned contrasts


  Done to test specific hypotheses
• Post hoc tests


  Used when there were no specific hypotheses