Internal Validity - the extent to which the changes observed in the dependent variables are caused by the independent variables.
Questions of internal validity - cannot be answered positively unless the design provides adequate control of extraneous variables.
Internal validity is essentially a problem of control.
Anything that contributes to the control of a research design contributes to its internal validity.
External validity - refers to generalizability or representativeness of the findings.
Questions addressed in external validity:
What groups, settings, experimental variables, and measurement variables can these findings be generalized?
Threats to internal validity:
History
Maturation
Testing
Instrumentation
Regression
Differential selection of participants
Experimental mortality
History - specific events or conditions, other than the treatment, may occur between the 1st and 2nd measurements of the participants to produce changes in the DV.
Maturation - processes that operate within the participants simply as a function of the passage of time
Testing - exposure to a pretest may affect participants' performance on a 2nd test, regardless of the independent variable
Instrumentation - changes in the measuring instruments, in the scores, or in the observers used may produce changes in the obtained measures
Regression - if groups are selected on the basis of extreme scores, statistical regression may operate to produce an effect that could be mistakenly interpreted as an experimental effect
Differential selection of participants - important differences may exist between the groups before the independent variable is applied
Experimental mortality - occurs when there is a differential loss of respondents from the comparison groups
Ways to control threats to internal validity:
Random assignment
Randomized matching
Homogeneous selection
Building variables in the design
Analysis of covariance
Using participants as their own controls
Randomized matching - match on as many variables as possible and then randomly assign one member of the pair to the independent variables, other goes to the control group
Homogeneous selection - select samples that are as similar as possible on some extraneous variable.
Example: IQ, age
Building variables in the design - include the extraneous variable as one of the independent variables examined
Analysis of covariance - removing portion of performance that is systematically related to an extraneous variable
Using participants as their own controls - participants are in each of the experimental conditions, one at a time.
Types of external validity:
Population external validity
Ecological external validity
External validity of operations
Population external validity - identifying the population to which results may be generalizable
Ecological external validity - concerned with generalizing experimental effects to other environmental conditions
Settings
External validity of operations - concerned with how well the operational definitions and the experimental procedures represent the constructs of interest
Would the same relationships be found if a different researcher used different operations in investigating the same question?
Rothman's Causal Pies - attempt to account for the multifactorial nature of causation.