Threats to Validity

Cards (26)

  • Two general types of research designs:
    • Internal Validity
    • External Validity
  • Internal Validity - the extent to which the changes observed in the dependent variables are caused by the independent variables.
  • Questions of internal validity - cannot be answered positively unless the design provides adequate control of extraneous variables.
  • Internal validity is essentially a problem of control.
  • Anything that contributes to the control of a research design contributes to its internal validity.
  • External validity - refers to generalizability or representativeness of the findings.
  • Questions addressed in external validity:
    • What groups, settings, experimental variables, and measurement variables can these findings be generalized?
  • Threats to internal validity:
    • History
    • Maturation
    • Testing
    • Instrumentation
    • Regression
    • Differential selection of participants
    • Experimental mortality
  • History - specific events or conditions, other than the treatment, may occur between the 1st and 2nd measurements of the participants to produce changes in the DV.
  • Maturation - processes that operate within the participants simply as a function of the passage of time
  • Testing - exposure to a pretest may affect participants' performance on a 2nd test, regardless of the independent variable
  • Instrumentation - changes in the measuring instruments, in the scores, or in the observers used may produce changes in the obtained measures
  • Regression - if groups are selected on the basis of extreme scores, statistical regression may operate to produce an effect that could be mistakenly interpreted as an experimental effect
  • Differential selection of participants - important differences may exist between the groups before the independent variable is applied
  • Experimental mortality - occurs when there is a differential loss of respondents from the comparison groups
  • Ways to control threats to internal validity:
    • Random assignment
    • Randomized matching
    • Homogeneous selection
    • Building variables in the design
    • Analysis of covariance
    • Using participants as their own controls
  • Randomized matching - match on as many variables as possible and then randomly assign one member of the pair to the independent variables, other goes to the control group
  • Homogeneous selection - select samples that are as similar as possible on some extraneous variable.
    • Example: IQ, age
  • Building variables in the design - include the extraneous variable as one of the independent variables examined
  • Analysis of covariance - removing portion of performance that is systematically related to an extraneous variable
  • Using participants as their own controls - participants are in each of the experimental conditions, one at a time.
  • Types of external validity:
    • Population external validity
    • Ecological external validity
    • External validity of operations
  • Population external validity - identifying the population to which results may be generalizable
  • Ecological external validity - concerned with generalizing experimental effects to other environmental conditions
    • Settings
  • External validity of operations - concerned with how well the operational definitions and the experimental procedures represent the constructs of interest
    • Would the same relationships be found if a different researcher used different operations in investigating the same question?
  • Rothman's Causal Pies - attempt to account for the multifactorial nature of causation.