Experimental Control

• Requirements for a Causal Claim

  • Covariation: cause and effect are related
  • Temporal Precedence: cause comes before effect
  • Elimination of confounds: no alternative causes
• The Alternative Explanation Mega Lists

  • Classical threats (Shadish, Cook, & Campbell, 1979)
  • Maturation
  • History
  • Regression to the mean
  • Attrition
  • Testing
  • Instrumentation
  • Other threats to internal validity
  • Selection threats
  • Design confound
  • Order effect
  • Observer bias
  • Demand characteristics
  • Placebo effect
• Classical Threats


  • Originally outlined by Shadish, Cook, and Campbell (1979)
  • General threats to experiments without control groups
  • All things that can lead to changes in the experimental group instead of the treatment under study
• History


  • Changes in the experimental group over time…because of an external factor
• History


  • News report increases/decreases anxiety in participants
• Maturation


  • Changes in the experimental group over time… due to natural changes or development
• Maturation


  • Development
  • Fatigue and boredom
• Regression to the mean


  • Changes in the experimental group over time… because extreme scores are unlikely to occur again
• Regression to the mean


  • Madden curse
  • Therapy
  • Genetic
• Testing


  • Changes in the experimental group over time… due to repeated testing
• Testing


  • Improvements due to practice
  • Participants become more familiar with the reaction time test and get faster
  • Also fatigue
• Instrumentation


  • Actual or apparent changes in the experimental group over time …that result from changes in the instrument or measure
• Instrumentation


  • In different groups: lever wears down over time, machine goes less reliable, different machines, different RA
  • Word meaning changes over time (awful, hussy, gay)
  • Across cultures (quite, state, chips)
• Attrition


  • Changes in the experimental group over time …that result from the failure of the participants to complete the study or measure
• Attrition


  • Is there a reason people are dropping out (MNAR)
  • Particular participants didn't want to answer a particular question
  • Particularly stressed participants stop coming in
  • Mortality
• Selection effects


  • When the experimental group differs from the control group because of systemic bias in group assignment
• Selection effects can be combined with all of the classical threats
• Classical threats can be found even in experiments with control groups
• Selection and History


  • People who sign up first are in control group, an event happens, then everyone else is assigned to the treatment group
• Design Confound


  • When the experimental group experiences something different from the control group because of bad design
• Design confounds can be combined with all of the classical threats
• Design Confound


  • Instrumentation, groups have different measurement devices, interact with different RAs
• Order Effects


  • In a repeated measures design, when the effect of the independent variable is confounded with carry-over effects from one level to another
• Between-subjects manipulation


  Each participant experiences one condition (aka., independent groups)
• Within-subjects manipulation


  Each participant experiences multiple conditions (aka repeated measures)
• Practice effects


  • An improvement in performance as a result of repeated exposure to the DV
• Fatigue effects


  • A decrement in performance as a result of repeated exposure to the DV
• Contrast effects


  • When the response to the second condition is affected because the participant contrasted the conditions to each other
• Counterbalance


  A control used within-subjects where participants are randomly assigned to take the conditions in different orders
• Time delay


  Long gaps where forgetting takes place may remove order effects
• Placebo Effects


  • When experimental group differs from the control group because the experiment group EXPECTS to differ (improve)
• Psychology doesn't always use placebos, but we do use placebo-like treatments
• Placebo-like treatments


  • Pretrial publicity: negative articles, positive articles, unrelated articles
  • Cognitive load: easy memory task, hard memory task, no memory task
  • Terror management: write about own death, write about dental pain
  • Trauma: look at traumatic photos, look at happy photos
• Hawthorn effects


  • When participants behave differently because they know they are being watched
• Experimental expectancy effects


  When experimental group differs from the control group because the experimenter EXPECTS them to differ (improve)
• Clever Hans


  • Experimenter with knowledge = 8% correct, Experimenter without knowledge = 98% correct
• Rosenthal & Fode, 1963


  • Rosenthal told his students that some mice were "Dull" or "Smart", and the "Smart" mice learned the maze faster
• Demand Characteristics


  • Participants differ because they are conforming to the experimental demands; any aspect of the experiment that allows participants to guess what is under investigation
• Cover story


  The ostensible reason for the study, what we tell participants is going on
• Confederate


  A member of the research staff who posses as another participant, or in another role, to help surreptitiously deliver experimental stimuli